2024-05-27T09:47:39.234 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T09:47:39.234 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} --archive /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706 --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/orig.config.yaml 2024-05-27T09:47:39.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T09:47:39.391 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706 branch: wip-vshankar-testing-20240521.105740-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7727706' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240521.105740-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: a6507388dd5057528934822c0163b0c347ef1d5d subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a6507388dd5057528934822c0163b0c347ef1d5d kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240521.105740-debug sha1: a6507388dd5057528934822c0163b0c347ef1d5d owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2274 sha1: a6507388dd5057528934822c0163b0c347ef1d5d sleep_before_teardown: 0 subset: 442/1024 suite: fs suite_branch: wip-vshankar-testing-20240521.105740-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a6507388dd5057528934822c0163b0c347ef1d5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK3ERCgYpk6aTC0hcB0soqIlvKfrtagQneb5TLpkbAGEsDfpxUleV2diyQUFN6TnQhuttiJPA+IlE+ljqn4QxoQ= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCRLp4pfePYyY6HdYs4QUFiQaTg2DTCMLsw0gnJqV+P3TXNx3sG2e7XfhvUaTrBPKnGb7K/AzsGr2Y8/F1jHmKo= smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO5owZBN0kyNCKv9DTasDev5qgo6Mx8ih1Tow6rTlQ66ZOZaEtOjqOgwg2PpwvNH64Qefgav1GgCQw1ggFwTpts= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_06:46:53 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T09:47:39.392 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa; will attempt to use it 2024-05-27T09:47:39.392 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks 2024-05-27T09:47:39.393 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T09:47:39.394 INFO:teuthology.task.internal:Checking packages... 2024-05-27T09:47:39.416 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6507388dd5057528934822c0163b0c347ef1d5d' 2024-05-27T09:47:39.416 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T09:47:39.416 INFO:teuthology.packaging:ref: None 2024-05-27T09:47:39.416 INFO:teuthology.packaging:tag: None 2024-05-27T09:47:39.416 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:47:39.416 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:47:39.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:47:39.708 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3868.ga6507388 2024-05-27T09:47:39.710 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T09:47:39.716 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T09:47:39.717 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T09:47:39.746 INFO:teuthology.task.internal:Saving configuration 2024-05-27T09:47:39.767 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T09:47:39.774 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T09:47:39.797 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 09:42:39.315781', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK3ERCgYpk6aTC0hcB0soqIlvKfrtagQneb5TLpkbAGEsDfpxUleV2diyQUFN6TnQhuttiJPA+IlE+ljqn4QxoQ='} 2024-05-27T09:47:39.817 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 09:42:39.312666', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCRLp4pfePYyY6HdYs4QUFiQaTg2DTCMLsw0gnJqV+P3TXNx3sG2e7XfhvUaTrBPKnGb7K/AzsGr2Y8/F1jHmKo='} 2024-05-27T09:47:39.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 09:42:39.314026', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO5owZBN0kyNCKv9DTasDev5qgo6Mx8ih1Tow6rTlQ66ZOZaEtOjqOgwg2PpwvNH64Qefgav1GgCQw1ggFwTpts='} 2024-05-27T09:47:39.837 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T09:47:39.843 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-27T09:47:39.843 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-27T09:47:39.843 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-27T09:47:39.843 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T09:47:39.952 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f242b03fac0>, signals=[15]) 2024-05-27T09:47:39.952 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T09:47:39.959 INFO:teuthology.task.internal:Opening connections... 2024-05-27T09:47:39.959 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T09:47:39.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:47:40.036 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-05-27T09:47:40.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:47:40.112 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-05-27T09:47:40.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:47:40.188 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T09:47:40.194 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-05-27T09:47:40.213 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-05-27T09:47:40.213 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-05-27T09:47:40.271 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-05-27T09:47:40.271 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:47:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:47:40.273 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-05-27T09:47:40.294 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-05-27T09:47:40.314 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-05-27T09:47:40.314 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:47:40.371 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-05-27T09:47:40.372 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:47:40.372 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:47:40.372 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:47:40.372 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:47:40.372 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:47:40.372 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:47:40.372 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-05-27T09:47:40.394 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-05-27T09:47:40.412 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-05-27T09:47:40.412 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:47:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-05-27T09:47:40.470 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:47:40.470 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:47:40.470 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:47:40.470 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:47:40.470 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:47:40.470 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:47:40.470 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-05-27T09:47:40.494 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T09:47:40.504 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T09:47:40.527 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T09:47:40.527 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:47:40.529 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:47:40.532 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:47:40.548 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T09:47:40.554 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T09:47:40.554 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:47:40.587 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:47:40.589 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:47:40.693 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T09:47:40.769 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T09:47:40.784 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T09:47:40.784 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.c': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-27T09:47:40.784 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:47:40.785 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:47:40.786 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:47:40.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:47:40.803 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.803 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T09:47:40.803 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-05-27T09:47:40.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:47:40.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:47:40.804 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.804 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T09:47:40.805 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-05-27T09:47:40.805 DEBUG:teuthology.task.kernel:sha1 for mon.c is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.805 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T09:47:40.805 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-05-27T09:47:40.859 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-437.el9.x86_64 2024-05-27T09:47:40.860 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.860 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T09:47:40.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.865 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-437.el9.x86_64 2024-05-27T09:47:40.865 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-437.el9.x86_64 2024-05-27T09:47:40.865 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.865 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T09:47:40.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.866 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:40.866 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T09:47:40.866 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:47:41.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum upgrade -y linux-firmware 2024-05-27T09:47:41.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum upgrade -y linux-firmware 2024-05-27T09:47:41.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum upgrade -y linux-firmware 2024-05-27T09:47:43.308 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-27T09:47:44.455 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-27T09:47:44.549 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.1 MB 00:01 2024-05-27T09:47:44.976 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-05-27T09:47:46.339 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-27T09:47:49.201 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-27T09:47:50.565 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-27T09:47:50.844 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 17 kB 00:00 2024-05-27T09:47:51.674 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-05-27T09:47:52.024 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 17 kB 00:00 2024-05-27T09:47:52.639 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-27T09:47:53.367 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-27T09:47:55.740 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-27T09:47:57.233 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 17 kB 00:00 2024-05-27T09:47:57.845 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-27T09:47:58.693 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-05-27T09:47:59.175 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-27T09:47:59.818 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-05-27T09:47:59.819 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-05-27T09:47:59.819 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-05-27T09:47:59.929 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T09:47:59.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:48:00.155 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T09:48:00.155 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T09:48:00.157 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi143.front.sepia.ceph.com to complete... 2024-05-27T09:48:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-05-27T09:48:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-05-27T09:48:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-05-27T09:48:00.951 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T09:48:00.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:48:01.092 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T09:48:01.092 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T09:48:01.095 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi042.front.sepia.ceph.com to complete... 2024-05-27T09:48:04.433 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-27T09:48:06.428 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-05-27T09:48:06.429 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-05-27T09:48:06.429 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-05-27T09:48:06.519 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T09:48:06.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:48:06.664 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T09:48:06.664 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T09:48:06.666 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi104.front.sepia.ceph.com to complete... 2024-05-27T09:48:34.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27 09:48:34 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T09:48:34.067 INFO:teuthology.orchestra.run.smithi143.stderr:FINISHED --2024-05-27 09:48:34-- 2024-05-27T09:48:34.067 INFO:teuthology.orchestra.run.smithi143.stderr:Total wall clock time: 34s 2024-05-27T09:48:34.067 INFO:teuthology.orchestra.run.smithi143.stderr:Downloaded: 1 files, 340M in 34s (10.2 MB/s) 2024-05-27T09:48:34.069 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T09:48:34.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T09:48:36.162 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying... ######################################## 2024-05-27T09:48:36.309 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing... ######################################## 2024-05-27T09:48:36.396 INFO:teuthology.orchestra.run.smithi143.stdout:Updating / installing... 2024-05-27T09:48:42.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27 09:48:42 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T09:48:42.679 INFO:teuthology.orchestra.run.smithi104.stderr:FINISHED --2024-05-27 09:48:42-- 2024-05-27T09:48:42.679 INFO:teuthology.orchestra.run.smithi104.stderr:Total wall clock time: 36s 2024-05-27T09:48:42.679 INFO:teuthology.orchestra.run.smithi104.stderr:Downloaded: 1 files, 340M in 36s (9.58 MB/s) 2024-05-27T09:48:42.682 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T09:48:42.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T09:48:44.344 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T09:48:44.960 INFO:teuthology.orchestra.run.smithi104.stdout:Verifying... ######################################## 2024-05-27T09:48:45.073 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing... ######################################## 2024-05-27T09:48:45.200 INFO:teuthology.orchestra.run.smithi104.stdout:Updating / installing... 2024-05-27T09:48:53.153 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T09:48:59.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27 09:48:59 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T09:48:59.306 INFO:teuthology.orchestra.run.smithi042.stderr:FINISHED --2024-05-27 09:48:59-- 2024-05-27T09:48:59.307 INFO:teuthology.orchestra.run.smithi042.stderr:Total wall clock time: 58s 2024-05-27T09:48:59.307 INFO:teuthology.orchestra.run.smithi042.stderr:Downloaded: 1 files, 340M in 58s (5.90 MB/s) 2024-05-27T09:48:59.309 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T09:48:59.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T09:49:01.502 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying... ######################################## 2024-05-27T09:49:01.689 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing... ######################################## 2024-05-27T09:49:01.794 INFO:teuthology.orchestra.run.smithi042.stdout:Updating / installing... 2024-05-27T09:49:10.151 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T09:49:21.640 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T09:49:21.640 DEBUG:teuthology.orchestra.run.smithi143:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T09:49:21.720 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T09:49:21.721 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T09:49:21.722 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T09:49:21.723 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T09:49:21.724 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T09:49:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T09:49:21.726 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T09:49:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T09:49:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T09:49:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T09:49:21.730 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T09:49:21.731 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T09:49:21.732 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T09:49:21.733 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T09:49:21.734 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T09:49:21.735 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T09:49:21.736 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T09:49:21.737 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T09:49:21.738 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T09:49:21.739 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T09:49:21.740 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T09:49:21.741 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T09:49:21.742 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T09:49:21.743 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T09:49:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T09:49:21.745 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T09:49:21.746 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T09:49:21.747 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T09:49:21.748 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T09:49:21.749 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T09:49:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T09:49:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T09:49:21.752 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T09:49:21.753 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T09:49:21.754 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T09:49:21.755 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T09:49:21.756 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T09:49:21.757 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T09:49:21.758 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T09:49:21.759 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T09:49:21.760 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T09:49:21.761 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T09:49:21.762 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T09:49:21.763 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T09:49:21.764 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T09:49:21.765 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T09:49:21.766 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T09:49:21.767 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T09:49:21.768 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T09:49:21.769 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T09:49:21.770 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T09:49:21.771 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T09:49:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T09:49:21.773 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T09:49:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T09:49:21.775 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T09:49:21.776 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T09:49:21.777 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T09:49:21.778 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T09:49:21.779 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T09:49:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T09:49:21.781 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T09:49:21.782 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T09:49:21.783 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T09:49:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T09:49:21.785 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T09:49:21.786 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T09:49:21.787 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T09:49:21.788 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T09:49:21.789 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T09:49:21.790 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T09:49:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T09:49:21.792 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T09:49:21.793 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T09:49:21.794 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T09:49:21.795 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T09:49:21.796 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T09:49:21.797 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T09:49:21.798 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T09:49:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T09:49:21.800 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T09:49:21.801 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T09:49:21.802 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T09:49:21.803 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T09:49:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T09:49:21.805 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T09:49:21.806 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T09:49:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T09:49:21.808 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T09:49:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T09:49:21.810 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T09:49:21.811 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T09:49:21.812 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T09:49:21.813 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T09:49:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T09:49:21.815 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T09:49:21.816 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T09:49:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T09:49:21.818 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T09:49:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T09:49:21.820 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T09:49:21.821 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T09:49:21.822 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T09:49:21.823 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T09:49:21.824 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T09:49:21.825 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T09:49:21.826 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T09:49:21.827 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T09:49:21.828 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T09:49:21.829 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T09:49:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T09:49:21.831 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T09:49:21.832 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T09:49:21.833 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T09:49:21.834 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T09:49:21.835 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T09:49:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T09:49:21.837 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T09:49:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T09:49:21.839 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T09:49:21.840 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T09:49:21.841 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T09:49:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T09:49:21.843 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T09:49:21.844 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T09:49:21.845 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T09:49:21.846 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T09:49:21.847 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T09:49:21.848 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T09:49:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T09:49:21.850 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T09:49:21.851 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T09:49:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T09:49:21.853 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T09:49:21.854 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T09:49:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T09:49:21.856 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T09:49:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T09:49:21.858 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T09:49:21.859 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T09:49:21.860 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T09:49:21.861 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T09:49:21.862 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T09:49:21.863 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T09:49:21.864 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:49:21.868 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T09:49:21.868 DEBUG:teuthology.orchestra.run.smithi143:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout:for file in vmlinuz System.map config; do 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout: fi 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout:done 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T09:49:21.891 INFO:teuthology.orchestra.run.smithi143.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T09:49:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-05-27T09:49:21.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2024-05-27T09:49:21.965 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2024-05-27T09:49:21.965 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2024-05-27T09:49:21.965 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.06 2024-05-27T09:49:21.965 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 68.el9 2024-05-27T09:49:21.965 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2024-05-27T09:49:21.965 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Group : Unspecified 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 8264254 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : builder@centos.org 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : CentOS 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T09:49:21.966 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T09:49:21.967 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T09:49:21.967 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T09:49:21.967 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2024-05-27T09:49:21.967 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T09:49:21.967 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T09:49:21.968 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T09:49:21.968 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 6.9.0-gb9459a26b088 2024-05-27T09:49:21.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T09:49:22.492 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-05-27T09:49:23.930 INFO:teuthology.orchestra.run.smithi143.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T09:49:23.961 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-05-27T09:49:23.963 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-05-27T09:49:23.980 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.RSLbGMfYm7 2024-05-27T09:49:23.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RSLbGMfYm7 2024-05-27T09:49:24.047 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.RSLbGMfYm7 2024-05-27T09:49:24.188 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.RSLbGMfYm7 is 6KB 2024-05-27T09:49:24.236 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.RSLbGMfYm7 2024-05-27T09:49:24.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T09:49:24.312 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T09:49:24.312 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T09:49:24.312 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T09:49:24.312 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T09:49:24.313 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T09:49:24.457 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2024-05-27T09:49:29.875 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi104.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T09:49:29.876 DEBUG:teuthology.orchestra.run.smithi104:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T09:49:29.952 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T09:49:29.953 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T09:49:29.954 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T09:49:29.955 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T09:49:29.956 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T09:49:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T09:49:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T09:49:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T09:49:29.960 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T09:49:29.961 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T09:49:29.962 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T09:49:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T09:49:29.964 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T09:49:29.965 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T09:49:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T09:49:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T09:49:29.968 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T09:49:29.969 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T09:49:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T09:49:29.971 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T09:49:29.972 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T09:49:29.973 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T09:49:29.974 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T09:49:29.975 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T09:49:29.976 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T09:49:29.977 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T09:49:29.978 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T09:49:29.979 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T09:49:29.980 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T09:49:29.981 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T09:49:29.982 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T09:49:29.983 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T09:49:29.984 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T09:49:29.985 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T09:49:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T09:49:29.987 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T09:49:29.988 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T09:49:29.989 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T09:49:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T09:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T09:49:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T09:49:29.993 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T09:49:29.994 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T09:49:29.995 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T09:49:29.996 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T09:49:29.997 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T09:49:29.998 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T09:49:29.999 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T09:49:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T09:49:30.001 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T09:49:30.002 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T09:49:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T09:49:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T09:49:30.005 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T09:49:30.006 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T09:49:30.007 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T09:49:30.008 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T09:49:30.009 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T09:49:30.010 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T09:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T09:49:30.012 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T09:49:30.013 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T09:49:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T09:49:30.015 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T09:49:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T09:49:30.017 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T09:49:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T09:49:30.019 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T09:49:30.020 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T09:49:30.021 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T09:49:30.022 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T09:49:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T09:49:30.024 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T09:49:30.025 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T09:49:30.026 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T09:49:30.027 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T09:49:30.028 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T09:49:30.029 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T09:49:30.030 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T09:49:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T09:49:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T09:49:30.033 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T09:49:30.034 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T09:49:30.035 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T09:49:30.036 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T09:49:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T09:49:30.038 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T09:49:30.039 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T09:49:30.040 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T09:49:30.041 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T09:49:30.042 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T09:49:30.043 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T09:49:30.044 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T09:49:30.045 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T09:49:30.046 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T09:49:30.047 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T09:49:30.048 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T09:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T09:49:30.050 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T09:49:30.051 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T09:49:30.052 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T09:49:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T09:49:30.054 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T09:49:30.055 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T09:49:30.058 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T09:49:30.059 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T09:49:30.060 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T09:49:30.061 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T09:49:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T09:49:30.063 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T09:49:30.064 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T09:49:30.065 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T09:49:30.066 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T09:49:30.067 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T09:49:30.068 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T09:49:30.069 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T09:49:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T09:49:30.071 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T09:49:30.072 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T09:49:30.073 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T09:49:30.074 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T09:49:30.075 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T09:49:30.076 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T09:49:30.077 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T09:49:30.078 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T09:49:30.079 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T09:49:30.080 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T09:49:30.081 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T09:49:30.082 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T09:49:30.083 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T09:49:30.084 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T09:49:30.085 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T09:49:30.086 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T09:49:30.087 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T09:49:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T09:49:30.089 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T09:49:30.090 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T09:49:30.091 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T09:49:30.092 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T09:49:30.093 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T09:49:30.094 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T09:49:30.095 INFO:teuthology.orchestra.run.smithi104.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:49:30.098 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T09:49:30.099 DEBUG:teuthology.orchestra.run.smithi104:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:fi 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:for file in vmlinuz System.map config; do 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout: fi 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:done 2024-05-27T09:49:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:fi 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:fi 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T09:49:30.122 INFO:teuthology.orchestra.run.smithi104.stdout:fi 2024-05-27T09:49:30.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo rpm -qi grub2-tools 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Name : grub2-tools 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Epoch : 1 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Version : 2.06 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Release : 68.el9 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: x86_64 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Group : Unspecified 2024-05-27T09:49:30.194 INFO:teuthology.orchestra.run.smithi104.stdout:Size : 8264254 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:License : GPLv3+ 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Packager : builder@centos.org 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Vendor : CentOS 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Summary : Support tools for GRUB. 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:Description : 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T09:49:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:hardware devices. 2024-05-27T09:49:30.196 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T09:49:30.196 INFO:teuthology.orchestra.run.smithi104.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T09:49:30.196 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T09:49:30.196 INFO:teuthology.task.kernel:Updating grub on smithi104 to boot 6.9.0-gb9459a26b088 2024-05-27T09:49:30.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T09:49:30.790 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-05-27T09:49:32.139 INFO:teuthology.orchestra.run.smithi104.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T09:49:32.175 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-05-27T09:49:32.178 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-05-27T09:49:32.207 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.WPCUFxRAwB 2024-05-27T09:49:32.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WPCUFxRAwB 2024-05-27T09:49:32.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.WPCUFxRAwB 2024-05-27T09:49:32.413 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.WPCUFxRAwB is 6KB 2024-05-27T09:49:32.432 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.WPCUFxRAwB 2024-05-27T09:49:32.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T09:49:32.506 INFO:teuthology.orchestra.run.smithi104.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T09:49:32.507 INFO:teuthology.orchestra.run.smithi104.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T09:49:32.507 INFO:teuthology.orchestra.run.smithi104.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T09:49:32.507 INFO:teuthology.orchestra.run.smithi104.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T09:49:32.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T09:49:32.684 DEBUG:teuthology.orchestra.run.smithi104:> sudo shutdown -r now 2024-05-27T09:49:48.366 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T09:49:48.367 DEBUG:teuthology.orchestra.run.smithi042:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T09:49:48.449 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T09:49:48.450 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T09:49:48.451 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T09:49:48.452 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T09:49:48.453 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T09:49:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T09:49:48.455 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T09:49:48.456 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T09:49:48.457 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T09:49:48.458 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T09:49:48.459 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T09:49:48.460 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T09:49:48.461 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T09:49:48.462 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T09:49:48.463 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T09:49:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T09:49:48.465 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T09:49:48.466 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T09:49:48.467 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T09:49:48.468 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T09:49:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T09:49:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T09:49:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T09:49:48.472 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T09:49:48.473 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T09:49:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T09:49:48.475 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T09:49:48.476 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T09:49:48.477 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T09:49:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T09:49:48.479 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T09:49:48.480 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T09:49:48.481 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T09:49:48.482 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T09:49:48.483 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T09:49:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T09:49:48.485 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T09:49:48.486 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T09:49:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T09:49:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T09:49:48.489 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T09:49:48.490 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T09:49:48.491 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T09:49:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T09:49:48.493 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T09:49:48.494 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T09:49:48.495 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T09:49:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T09:49:48.497 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T09:49:48.498 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T09:49:48.499 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T09:49:48.500 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T09:49:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T09:49:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T09:49:48.503 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T09:49:48.504 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T09:49:48.505 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T09:49:48.506 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T09:49:48.507 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T09:49:48.508 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T09:49:48.509 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T09:49:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T09:49:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T09:49:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T09:49:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T09:49:48.514 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T09:49:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T09:49:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T09:49:48.517 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T09:49:48.518 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T09:49:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T09:49:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T09:49:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T09:49:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T09:49:48.523 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T09:49:48.524 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T09:49:48.525 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T09:49:48.526 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T09:49:48.527 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T09:49:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T09:49:48.529 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T09:49:48.530 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T09:49:48.531 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T09:49:48.532 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T09:49:48.533 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T09:49:48.534 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T09:49:48.535 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T09:49:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T09:49:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T09:49:48.538 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T09:49:48.539 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T09:49:48.540 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T09:49:48.541 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T09:49:48.542 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T09:49:48.543 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T09:49:48.544 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T09:49:48.545 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T09:49:48.546 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T09:49:48.547 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T09:49:48.548 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T09:49:48.549 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T09:49:48.550 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T09:49:48.551 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T09:49:48.552 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T09:49:48.553 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T09:49:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T09:49:48.555 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T09:49:48.556 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T09:49:48.557 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T09:49:48.558 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T09:49:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T09:49:48.560 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T09:49:48.561 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T09:49:48.562 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T09:49:48.563 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T09:49:48.564 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T09:49:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T09:49:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T09:49:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T09:49:48.568 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T09:49:48.569 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T09:49:48.570 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T09:49:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T09:49:48.572 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T09:49:48.573 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T09:49:48.574 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T09:49:48.575 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T09:49:48.576 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T09:49:48.577 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T09:49:48.578 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T09:49:48.579 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T09:49:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T09:49:48.581 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T09:49:48.582 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T09:49:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T09:49:48.584 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T09:49:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T09:49:48.586 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T09:49:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T09:49:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T09:49:48.589 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T09:49:48.590 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T09:49:48.591 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T09:49:48.592 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T09:49:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T09:49:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T09:49:48.595 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T09:49:48.596 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T09:49:48.597 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T09:49:48.598 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T09:49:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T09:49:48.600 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T09:49:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T09:49:48.602 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T09:49:48.603 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:49:48.607 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T09:49:48.607 DEBUG:teuthology.orchestra.run.smithi042:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T09:49:48.631 INFO:teuthology.orchestra.run.smithi042.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T09:49:48.631 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:49:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:49:48.631 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-05-27T09:49:48.631 INFO:teuthology.orchestra.run.smithi042.stdout:for file in vmlinuz System.map config; do 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: fi 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:done 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T09:49:48.632 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:49:48.633 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T09:49:48.633 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-05-27T09:49:48.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 68.el9 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T09:49:48.718 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8264254 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T09:49:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T09:49:48.720 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T09:49:48.720 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T09:49:48.720 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-05-27T09:49:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T09:49:48.720 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T09:49:48.721 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T09:49:48.721 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 6.9.0-gb9459a26b088 2024-05-27T09:49:48.721 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T09:49:49.253 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-05-27T09:49:50.541 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T09:49:50.569 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-05-27T09:49:50.571 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-05-27T09:49:50.612 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.JyNuF9ZRwO 2024-05-27T09:49:50.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JyNuF9ZRwO 2024-05-27T09:49:50.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.JyNuF9ZRwO 2024-05-27T09:49:50.831 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.JyNuF9ZRwO is 6KB 2024-05-27T09:49:50.885 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.JyNuF9ZRwO 2024-05-27T09:49:50.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T09:49:50.964 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T09:49:50.964 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T09:49:50.964 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T09:49:50.964 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T09:49:50.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T09:49:51.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-05-27T09:49:54.462 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-27T09:49:54.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-27T09:49:54.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:50:02.694 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-27T09:50:02.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-27T09:50:02.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:50:13.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-05-27T09:50:21.126 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-27T09:50:21.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-27T09:50:21.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:50:21.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-05-27T09:50:22.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-27T09:50:22.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:50:30.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-27T09:50:30.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:50:39.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-05-27T09:50:48.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-27T09:50:48.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:51:22.061 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T09:51:30.223 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T09:51:34.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-27T09:51:34.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:51:34.512 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-05-27T09:51:35.166 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-27T09:51:35.166 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-gb9459a26b088"... 2024-05-27T09:51:35.166 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-05-27T09:51:35.219 INFO:teuthology.orchestra.run.smithi143.stdout:6.9.0-gb9459a26b088 2024-05-27T09:51:35.220 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T09:51:35.220 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T09:51:35.220 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T09:51:36.221 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-27T09:51:36.221 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T09:51:36.355 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2024-05-27T09:51:36.416 DEBUG:teuthology.parallel:result is None 2024-05-27T09:51:42.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-27T09:51:42.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:51:42.695 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-05-27T09:51:43.321 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-27T09:51:43.321 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-27T09:51:43.321 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-05-27T09:51:43.374 INFO:teuthology.orchestra.run.smithi104.stdout:6.9.0-gb9459a26b088 2024-05-27T09:51:43.374 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T09:51:43.374 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T09:51:43.374 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T09:51:44.375 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-27T09:51:44.375 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T09:51:44.503 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2024-05-27T09:51:44.548 DEBUG:teuthology.parallel:result is None 2024-05-27T09:51:48.657 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T09:52:00.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-27T09:52:00.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:52:01.061 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-05-27T09:52:01.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-27T09:52:01.720 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-27T09:52:01.720 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-05-27T09:52:01.776 INFO:teuthology.orchestra.run.smithi042.stdout:6.9.0-gb9459a26b088 2024-05-27T09:52:01.776 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T09:52:01.776 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T09:52:01.777 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T09:52:02.778 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-27T09:52:02.778 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T09:52:02.920 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-05-27T09:52:02.980 DEBUG:teuthology.parallel:result is None 2024-05-27T09:52:02.981 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T09:52:02.998 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T09:52:02.998 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:52:03.006 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:52:03.016 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:52:03.074 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T09:52:03.081 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T09:52:03.107 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T09:52:03.108 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:52:03.115 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:52:03.122 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:52:03.193 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T09:52:03.200 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T09:52:03.200 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:52:03.207 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:52:03.214 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:52:03.300 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:52:03.300 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:52:03.301 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:52:03.332 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:52:03.332 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:52:03.333 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:52:03.338 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T09:52:03.344 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T09:52:03.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:52:03.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:52:03.359 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:52:03.439 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T09:52:03.481 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T09:52:03.481 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:52:03.489 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:52:03.496 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:52:03.553 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:52:03.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:52:03.764 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:52:03.855 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:52:03.949 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T09:52:03.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:52:04.035 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:52:04.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:52:04.210 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:52:04.295 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:52:04.386 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T09:52:04.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:52:04.489 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:52:04.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:52:04.673 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:52:04.755 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:52:04.840 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T09:52:04.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:52:04.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-05-27T09:52:04.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-05-27T09:52:04.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-05-27T09:52:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:52:05.047 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:52:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:52:05.532 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T09:52:05.565 INFO:teuthology.task.internal:Starting timer... 2024-05-27T09:52:05.565 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T09:52:05.589 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T09:52:05.627 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T09:52:05.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-05-27T09:52:05.759 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-05-27T09:52:05.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-05-27T09:52:05.918 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-05-27T09:52:05.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-05-27T09:52:06.046 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-05-27T09:52:06.051 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T09:52:06.051 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-05-27T09:52:06.108 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-05-27T09:52:06.109 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-05-27T09:52:06.163 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-05-27T09:52:06.164 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-05-27T09:52:06.231 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-05-27T09:52:06.231 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi143.front.sepia.ceph.com': 'permissive'} 2024-05-27T09:52:06.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:52:06.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:52:06.315 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:52:06.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:52:06.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:52:06.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:52:06.464 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T09:52:06.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:52:06.539 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:52:06.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:52:06.687 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T09:52:06.698 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T09:52:06.711 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T09:52:06.712 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T09:58:34.038 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2024-05-27T09:58:34.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-27T09:58:34.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:58:34.189 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-05-27T09:58:34.352 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-27T09:58:34.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-27T09:58:34.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:58:34.474 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-05-27T09:58:34.637 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-27T09:58:34.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-27T09:58:34.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:58:34.743 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-05-27T09:58:34.895 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-27T09:58:34.896 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T09:58:34.908 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T09:58:34.908 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:58:34.909 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:58:34.916 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:58:34.916 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:58:34.924 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:58:34.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:58:35.011 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:58:35.012 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:58:35.015 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:58:35.053 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:58:35.057 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:58:35.059 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:58:35.127 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-05-27T09:58:35.130 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-05-27T09:58:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-05-27T09:58:35.158 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-05-27T09:58:35.159 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-05-27T09:58:35.176 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-05-27T09:58:35.198 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:58:35.199 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:58:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:58:35.237 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:58:35.240 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:58:35.264 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:58:35.375 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:58:35.382 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:58:35.384 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:58:35.384 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-05-27T09:58:35.384 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.384 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.385 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.385 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:58:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-05-27T09:58:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.414 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:58:35.424 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:58:35.424 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-05-27T09:58:35.424 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.424 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.425 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.425 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:58:35.426 INFO:teuthology.run_tasks:Running task install... 2024-05-27T09:58:35.436 DEBUG:teuthology.task.install:project ceph 2024-05-27T09:58:35.436 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}} 2024-05-27T09:58:35.436 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'} 2024-05-27T09:58:35.436 INFO:teuthology.task.install:Using flavor: default 2024-05-27T09:58:35.442 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T09:58:35.442 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T09:58:35.442 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': None, 'wait_for_package': False} 2024-05-27T09:58:35.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:58:35.446 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': None, 'wait_for_package': False} 2024-05-27T09:58:35.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:58:35.447 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': None, 'wait_for_package': False} 2024-05-27T09:58:35.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:58:35.595 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/ 2024-05-27T09:58:35.596 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga6507388 2024-05-27T09:58:35.599 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/ 2024-05-27T09:58:35.599 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga6507388 2024-05-27T09:58:35.668 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/ 2024-05-27T09:58:35.668 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga6507388 2024-05-27T09:58:35.713 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:58:35.714 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T09:58:35.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:58:35.767 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:58:35.768 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T09:58:35.768 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:58:35.785 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:58:35.786 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T09:58:35.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:58:35.808 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T09:58:35.808 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6507388dd5057528934822c0163b0c347ef1d5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:58:35.857 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T09:58:35.857 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6507388dd5057528934822c0163b0c347ef1d5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:58:35.873 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T09:58:35.873 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6507388dd5057528934822c0163b0c347ef1d5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:58:35.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:58:35.985 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:58:36.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:58:36.057 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:58:36.090 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:58:36.107 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:58:36.134 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-05-27T09:58:36.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-05-27T09:58:36.164 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-05-27T09:58:36.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-05-27T09:58:36.186 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-05-27T09:58:36.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-05-27T09:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout:45 files removed 2024-05-27T09:58:36.472 INFO:teuthology.orchestra.run.smithi143.stdout:45 files removed 2024-05-27T09:58:36.505 INFO:teuthology.orchestra.run.smithi104.stdout:45 files removed 2024-05-27T09:58:36.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T09:58:36.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T09:58:36.543 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T09:58:37.335 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 231 kB/s | 84 kB 00:00 2024-05-27T09:58:37.382 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 193 kB/s | 84 kB 00:00 2024-05-27T09:58:37.427 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 179 kB/s | 84 kB 00:00 2024-05-27T09:58:37.810 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-27T09:58:37.866 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-27T09:58:37.936 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-27T09:58:38.327 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-05-27T09:58:38.417 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-05-27T09:58:38.461 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-05-27T09:58:38.982 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-27T09:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-27T09:58:40.767 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.1 MB 00:02 2024-05-27T09:58:42.644 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-27T09:58:42.829 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-27T09:58:45.516 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 19 MB 00:05 2024-05-27T09:58:47.610 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-05-27T09:58:47.613 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-05-27T09:58:49.380 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 17 kB 00:00 2024-05-27T09:58:50.277 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 17 kB 00:01 2024-05-27T09:58:50.917 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-05-27T09:58:51.039 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-27T09:58:51.363 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-27T09:58:53.742 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 17 kB 00:01 2024-05-27T09:58:55.825 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-27T09:58:57.453 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-27T09:58:57.532 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-27T09:58:59.879 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:58:59.881 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:58:59.901 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T09:58:59.903 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T09:58:59.904 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T09:58:59.905 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T09:58:59.908 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T09:58:59.911 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T09:58:59.912 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T09:58:59.985 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.0.0-3868.ga6507388.el9 ceph 6.6 k 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.4 M 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 863 k 2024-05-27T09:59:00.005 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga6507388.el9 ceph 147 k 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga6507388.el9 ceph 1.5 M 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 143 k 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 3.5 M 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 7.4 M 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 49 k 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 10 M 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.0.0-3868.ga6507388.el9 ceph 43 M 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 273 k 2024-05-27T09:59:00.006 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 764 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 224 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga6507388.el9 ceph 27 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 18 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 32 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 756 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 127 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T09:59:00.007 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga6507388.el9 ceph 161 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.0.0-3868.ga6507388.el9 ceph 321 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 301 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 100 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 86 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.0 M 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 170 k 2024-05-27T09:59:00.008 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T09:59:00.009 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.3 M 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.1 M 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 20 M 2024-05-27T09:59:00.010 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 34 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.0.0-3868.ga6507388.el9 ceph 2.3 M 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 274 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.0.0-3868.ga6507388.el9 ceph 4.9 M 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 18 M 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 16 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga6507388.el9 ceph 25 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T09:59:00.011 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:59:00.012 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:59:00.012 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga6507388.el9 ceph 157 k 2024-05-27T09:59:00.012 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:59:00.012 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:59:00.012 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:59:00.012 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 497 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.2 M 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:59:00.013 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 45 k 2024-05-27T09:59:00.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 136 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:59:00.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:59:00.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T09:59:00.017 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:59:00.018 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:59:00.019 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:59:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout:Install 148 Packages 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 11 Packages 2024-05-27T09:59:00.021 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T09:59:00.022 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 219 M 2024-05-27T09:59:00.022 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-05-27T09:59:00.086 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:59:00.088 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:59:00.107 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T09:59:00.108 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T09:59:00.109 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T09:59:00.111 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T09:59:00.113 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T09:59:00.117 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T09:59:00.118 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T09:59:00.194 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-3868.ga6507388.el9 ceph 6.6 k 2024-05-27T09:59:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.4 M 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 863 k 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga6507388.el9 ceph 147 k 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga6507388.el9 ceph 1.5 M 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 143 k 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 3.5 M 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 7.4 M 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 49 k 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 10 M 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-3868.ga6507388.el9 ceph 43 M 2024-05-27T09:59:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 273 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 764 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 224 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga6507388.el9 ceph 27 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 18 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 32 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 756 k 2024-05-27T09:59:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 127 k 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga6507388.el9 ceph 161 k 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-3868.ga6507388.el9 ceph 321 k 2024-05-27T09:59:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 301 k 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 100 k 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 86 k 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.0 M 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 170 k 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T09:59:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.3 M 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.1 M 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T09:59:00.244 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 20 M 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 34 k 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-3868.ga6507388.el9 ceph 2.3 M 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 274 k 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-3868.ga6507388.el9 ceph 4.9 M 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 18 M 2024-05-27T09:59:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 16 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga6507388.el9 ceph 25 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:59:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga6507388.el9 ceph 157 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 497 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.2 M 2024-05-27T09:59:00.247 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T09:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 45 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 136 k 2024-05-27T09:59:00.249 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:59:00.250 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:59:00.251 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T09:59:00.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:59:00.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T09:59:00.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-05-27T09:59:00.255 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout:Install 148 Packages 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 11 Packages 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T09:59:00.256 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 219 M 2024-05-27T09:59:00.257 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-05-27T09:59:00.996 INFO:teuthology.orchestra.run.smithi104.stdout:(1/159): ceph-19.0.0-3868.ga6507388.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-05-27T09:59:01.376 INFO:teuthology.orchestra.run.smithi042.stdout:(1/159): ceph-19.0.0-3868.ga6507388.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-27T09:59:01.944 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-27T09:59:02.205 INFO:teuthology.orchestra.run.smithi104.stdout:(2/159): ceph-fuse-19.0.0-3868.ga6507388.el9.x8 715 kB/s | 863 kB 00:01 2024-05-27T09:59:02.404 INFO:teuthology.orchestra.run.smithi104.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 740 kB/s | 147 kB 00:00 2024-05-27T09:59:02.459 INFO:teuthology.orchestra.run.smithi042.stdout:(2/159): ceph-base-19.0.0-3868.ga6507388.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-05-27T09:59:02.493 INFO:teuthology.orchestra.run.smithi042.stdout:(3/159): ceph-fuse-19.0.0-3868.ga6507388.el9.x8 773 kB/s | 863 kB 00:01 2024-05-27T09:59:02.535 INFO:teuthology.orchestra.run.smithi042.stdout:(4/159): ceph-immutable-object-cache-19.0.0-386 1.9 MB/s | 147 kB 00:00 2024-05-27T09:59:02.785 INFO:teuthology.orchestra.run.smithi042.stdout:(5/159): ceph-mgr-19.0.0-3868.ga6507388.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-05-27T09:59:03.385 INFO:teuthology.orchestra.run.smithi042.stdout:(6/159): ceph-mon-19.0.0-3868.ga6507388.el9.x86 8.1 MB/s | 4.9 MB 00:00 2024-05-27T09:59:03.561 INFO:teuthology.orchestra.run.smithi042.stdout:(7/159): ceph-mds-19.0.0-3868.ga6507388.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-05-27T09:59:03.613 INFO:teuthology.orchestra.run.smithi104.stdout:(4/159): ceph-common-19.0.0-3868.ga6507388.el9. 6.7 MB/s | 20 MB 00:02 2024-05-27T09:59:03.955 INFO:teuthology.orchestra.run.smithi104.stdout:(5/159): ceph-mgr-19.0.0-3868.ga6507388.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-05-27T09:59:04.305 INFO:teuthology.orchestra.run.smithi104.stdout:(6/159): ceph-mds-19.0.0-3868.ga6507388.el9.x86 1.2 MB/s | 2.3 MB 00:01 2024-05-27T09:59:04.336 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:59:04.338 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:59:04.358 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T09:59:04.359 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T09:59:04.360 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T09:59:04.362 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T09:59:04.364 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T09:59:04.367 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T09:59:04.369 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T09:59:04.439 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:19.0.0-3868.ga6507388.el9 ceph 6.6 k 2024-05-27T09:59:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.4 M 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 863 k 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga6507388.el9 ceph 147 k 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga6507388.el9 ceph 1.5 M 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 143 k 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 3.5 M 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 7.4 M 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 49 k 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 10 M 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:19.0.0-3868.ga6507388.el9 ceph 43 M 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 273 k 2024-05-27T09:59:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 764 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 224 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga6507388.el9 ceph 27 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 18 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 32 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 756 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 127 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T09:59:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga6507388.el9 ceph 161 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:19.0.0-3868.ga6507388.el9 ceph 321 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 301 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 100 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 86 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.0 M 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 170 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T09:59:04.464 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.3 M 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.1 M 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T09:59:04.465 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 20 M 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 34 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:19.0.0-3868.ga6507388.el9 ceph 2.3 M 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 274 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:19.0.0-3868.ga6507388.el9 ceph 4.9 M 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 18 M 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 16 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga6507388.el9 ceph 25 k 2024-05-27T09:59:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga6507388.el9 ceph 157 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:59:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 497 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.2 M 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:59:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 45 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 136 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:59:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:59:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:59:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:59:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:59:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:59:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-05-27T09:59:04.475 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-05-27T09:59:04.476 INFO:teuthology.orchestra.run.smithi143.stdout:Install 148 Packages 2024-05-27T09:59:04.476 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 11 Packages 2024-05-27T09:59:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T09:59:04.477 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 219 M 2024-05-27T09:59:04.477 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-05-27T09:59:04.822 INFO:teuthology.orchestra.run.smithi104.stdout:(7/159): ceph-mon-19.0.0-3868.ga6507388.el9.x86 5.6 MB/s | 4.9 MB 00:00 2024-05-27T09:59:05.097 INFO:teuthology.orchestra.run.smithi143.stdout:(1/159): ceph-19.0.0-3868.ga6507388.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-05-27T09:59:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:(8/159): ceph-osd-19.0.0-3868.ga6507388.el9.x86 9.4 MB/s | 18 MB 00:01 2024-05-27T09:59:05.386 INFO:teuthology.orchestra.run.smithi042.stdout:(9/159): ceph-selinux-19.0.0-3868.ga6507388.el9 218 kB/s | 25 kB 00:00 2024-05-27T09:59:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:(10/159): ceph-radosgw-19.0.0-3868.ga6507388.el 5.1 MB/s | 10 MB 00:01 2024-05-27T09:59:05.604 INFO:teuthology.orchestra.run.smithi143.stdout:(2/159): ceph-fuse-19.0.0-3868.ga6507388.el9.x8 1.7 MB/s | 863 kB 00:00 2024-05-27T09:59:05.678 INFO:teuthology.orchestra.run.smithi042.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga6507388.e 1.9 MB/s | 224 kB 00:00 2024-05-27T09:59:05.713 INFO:teuthology.orchestra.run.smithi143.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 1.3 MB/s | 147 kB 00:00 2024-05-27T09:59:05.804 INFO:teuthology.orchestra.run.smithi042.stdout:(12/159): cephfs-shell-19.0.0-3868.ga6507388.el 217 kB/s | 27 kB 00:00 2024-05-27T09:59:05.962 INFO:teuthology.orchestra.run.smithi042.stdout:(13/159): libcephfs-devel-19.0.0-3868.ga6507388 201 kB/s | 32 kB 00:00 2024-05-27T09:59:06.505 INFO:teuthology.orchestra.run.smithi104.stdout:(8/159): ceph-radosgw-19.0.0-3868.ga6507388.el9 6.0 MB/s | 10 MB 00:01 2024-05-27T09:59:06.571 INFO:teuthology.orchestra.run.smithi042.stdout:(14/159): libcephfs2-19.0.0-3868.ga6507388.el9. 1.2 MB/s | 756 kB 00:00 2024-05-27T09:59:06.589 INFO:teuthology.orchestra.run.smithi104.stdout:(9/159): ceph-selinux-19.0.0-3868.ga6507388.el9 307 kB/s | 25 kB 00:00 2024-05-27T09:59:06.680 INFO:teuthology.orchestra.run.smithi042.stdout:(15/159): libcephsqlite-19.0.0-3868.ga6507388.e 1.4 MB/s | 157 kB 00:00 2024-05-27T09:59:06.797 INFO:teuthology.orchestra.run.smithi042.stdout:(16/159): librados-devel-19.0.0-3868.ga6507388. 1.1 MB/s | 127 kB 00:00 2024-05-27T09:59:06.813 INFO:teuthology.orchestra.run.smithi143.stdout:(4/159): ceph-mds-19.0.0-3868.ga6507388.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-05-27T09:59:06.895 INFO:teuthology.orchestra.run.smithi104.stdout:(10/159): ceph-base-19.0.0-3868.ga6507388.el9.x 891 kB/s | 5.4 MB 00:06 2024-05-27T09:59:06.914 INFO:teuthology.orchestra.run.smithi042.stdout:(17/159): libradosstriper1-19.0.0-3868.ga650738 4.2 MB/s | 497 kB 00:00 2024-05-27T09:59:07.181 INFO:teuthology.orchestra.run.smithi104.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga6507388.e 787 kB/s | 224 kB 00:00 2024-05-27T09:59:07.213 INFO:teuthology.orchestra.run.smithi143.stdout:(5/159): ceph-base-19.0.0-3868.ga6507388.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-27T09:59:07.282 INFO:teuthology.orchestra.run.smithi104.stdout:(12/159): cephfs-shell-19.0.0-3868.ga6507388.el 272 kB/s | 27 kB 00:00 2024-05-27T09:59:07.398 INFO:teuthology.orchestra.run.smithi104.stdout:(13/159): libcephfs-devel-19.0.0-3868.ga6507388 274 kB/s | 32 kB 00:00 2024-05-27T09:59:07.689 INFO:teuthology.orchestra.run.smithi143.stdout:(6/159): ceph-mgr-19.0.0-3868.ga6507388.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-05-27T09:59:08.090 INFO:teuthology.orchestra.run.smithi104.stdout:(14/159): libcephfs2-19.0.0-3868.ga6507388.el9. 1.1 MB/s | 756 kB 00:00 2024-05-27T09:59:08.282 INFO:teuthology.orchestra.run.smithi104.stdout:(15/159): libcephsqlite-19.0.0-3868.ga6507388.e 824 kB/s | 157 kB 00:00 2024-05-27T09:59:08.474 INFO:teuthology.orchestra.run.smithi104.stdout:(16/159): librados-devel-19.0.0-3868.ga6507388. 665 kB/s | 127 kB 00:00 2024-05-27T09:59:08.680 INFO:teuthology.orchestra.run.smithi042.stdout:(18/159): librgw2-19.0.0-3868.ga6507388.el9.x86 2.9 MB/s | 5.2 MB 00:01 2024-05-27T09:59:08.722 INFO:teuthology.orchestra.run.smithi143.stdout:(7/159): ceph-common-19.0.0-3868.ga6507388.el9. 5.0 MB/s | 20 MB 00:03 2024-05-27T09:59:08.798 INFO:teuthology.orchestra.run.smithi042.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga6 387 kB/s | 45 kB 00:00 2024-05-27T09:59:08.834 INFO:teuthology.orchestra.run.smithi143.stdout:(8/159): ceph-mon-19.0.0-3868.ga6507388.el9.x86 3.0 MB/s | 4.9 MB 00:01 2024-05-27T09:59:08.849 INFO:teuthology.orchestra.run.smithi104.stdout:(17/159): libradosstriper1-19.0.0-3868.ga650738 1.3 MB/s | 497 kB 00:00 2024-05-27T09:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga650 1.1 MB/s | 136 kB 00:00 2024-05-27T09:59:08.956 INFO:teuthology.orchestra.run.smithi143.stdout:(9/159): ceph-selinux-19.0.0-3868.ga6507388.el9 203 kB/s | 25 kB 00:00 2024-05-27T09:59:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:(21/159): python3-cephfs-19.0.0-3868.ga6507388. 1.3 MB/s | 161 kB 00:00 2024-05-27T09:59:09.257 INFO:teuthology.orchestra.run.smithi042.stdout:(22/159): python3-rados-19.0.0-3868.ga6507388.e 1.5 MB/s | 321 kB 00:00 2024-05-27T09:59:09.508 INFO:teuthology.orchestra.run.smithi042.stdout:(23/159): python3-rbd-19.0.0-3868.ga6507388.el9 1.2 MB/s | 301 kB 00:00 2024-05-27T09:59:09.616 INFO:teuthology.orchestra.run.smithi042.stdout:(24/159): python3-rgw-19.0.0-3868.ga6507388.el9 928 kB/s | 100 kB 00:00 2024-05-27T09:59:09.750 INFO:teuthology.orchestra.run.smithi042.stdout:(25/159): rbd-fuse-19.0.0-3868.ga6507388.el9.x8 650 kB/s | 86 kB 00:00 2024-05-27T09:59:10.956 INFO:teuthology.orchestra.run.smithi143.stdout:(10/159): ceph-radosgw-19.0.0-3868.ga6507388.el 4.5 MB/s | 10 MB 00:02 2024-05-27T09:59:10.967 INFO:teuthology.orchestra.run.smithi042.stdout:(26/159): rbd-mirror-19.0.0-3868.ga6507388.el9. 2.5 MB/s | 3.0 MB 00:01 2024-05-27T09:59:11.981 INFO:teuthology.orchestra.run.smithi143.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga6507388.e 219 kB/s | 224 kB 00:01 2024-05-27T09:59:12.067 INFO:teuthology.orchestra.run.smithi042.stdout:(27/159): rbd-nbd-19.0.0-3868.ga6507388.el9.x86 155 kB/s | 170 kB 00:01 2024-05-27T09:59:12.140 INFO:teuthology.orchestra.run.smithi143.stdout:(12/159): cephfs-shell-19.0.0-3868.ga6507388.el 171 kB/s | 27 kB 00:00 2024-05-27T09:59:12.167 INFO:teuthology.orchestra.run.smithi042.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 345 kB/s | 34 kB 00:00 2024-05-27T09:59:12.275 INFO:teuthology.orchestra.run.smithi143.stdout:(13/159): libcephfs-devel-19.0.0-3868.ga6507388 237 kB/s | 32 kB 00:00 2024-05-27T09:59:12.325 INFO:teuthology.orchestra.run.smithi042.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3868.ga650738 910 kB/s | 143 kB 00:00 2024-05-27T09:59:12.591 INFO:teuthology.orchestra.run.smithi143.stdout:(14/159): libcephfs2-19.0.0-3868.ga6507388.el9. 2.3 MB/s | 756 kB 00:00 2024-05-27T09:59:12.709 INFO:teuthology.orchestra.run.smithi143.stdout:(15/159): libcephsqlite-19.0.0-3868.ga6507388.e 1.3 MB/s | 157 kB 00:00 2024-05-27T09:59:12.825 INFO:teuthology.orchestra.run.smithi143.stdout:(16/159): librados-devel-19.0.0-3868.ga6507388. 1.1 MB/s | 127 kB 00:00 2024-05-27T09:59:13.325 INFO:teuthology.orchestra.run.smithi143.stdout:(17/159): libradosstriper1-19.0.0-3868.ga650738 997 kB/s | 497 kB 00:00 2024-05-27T09:59:13.699 INFO:teuthology.orchestra.run.smithi104.stdout:(18/159): librgw2-19.0.0-3868.ga6507388.el9.x86 1.1 MB/s | 5.2 MB 00:04 2024-05-27T09:59:13.813 INFO:teuthology.orchestra.run.smithi104.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga6 400 kB/s | 45 kB 00:00 2024-05-27T09:59:13.868 INFO:teuthology.orchestra.run.smithi042.stdout:(30/159): ceph-common-19.0.0-3868.ga6507388.el9 1.5 MB/s | 20 MB 00:12 2024-05-27T09:59:13.982 INFO:teuthology.orchestra.run.smithi104.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga650 804 kB/s | 136 kB 00:00 2024-05-27T09:59:14.166 INFO:teuthology.orchestra.run.smithi104.stdout:(21/159): python3-cephfs-19.0.0-3868.ga6507388. 886 kB/s | 161 kB 00:00 2024-05-27T09:59:14.508 INFO:teuthology.orchestra.run.smithi104.stdout:(22/159): python3-rados-19.0.0-3868.ga6507388.e 942 kB/s | 321 kB 00:00 2024-05-27T09:59:14.710 INFO:teuthology.orchestra.run.smithi042.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3868.ga6507 1.5 MB/s | 3.5 MB 00:02 2024-05-27T09:59:14.850 INFO:teuthology.orchestra.run.smithi104.stdout:(23/159): python3-rbd-19.0.0-3868.ga6507388.el9 884 kB/s | 301 kB 00:00 2024-05-27T09:59:14.994 INFO:teuthology.orchestra.run.smithi042.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga6 972 kB/s | 274 kB 00:00 2024-05-27T09:59:15.076 INFO:teuthology.orchestra.run.smithi104.stdout:(24/159): python3-rgw-19.0.0-3868.ga6507388.el9 522 kB/s | 100 kB 00:00 2024-05-27T09:59:15.103 INFO:teuthology.orchestra.run.smithi042.stdout:(33/159): ceph-mgr-rook-19.0.0-3868.ga6507388.e 459 kB/s | 49 kB 00:00 2024-05-27T09:59:15.151 INFO:teuthology.orchestra.run.smithi104.stdout:(25/159): rbd-fuse-19.0.0-3868.ga6507388.el9.x8 799 kB/s | 86 kB 00:00 2024-05-27T09:59:15.211 INFO:teuthology.orchestra.run.smithi042.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3868.ga 153 kB/s | 16 kB 00:00 2024-05-27T09:59:15.425 INFO:teuthology.orchestra.run.smithi143.stdout:(18/159): librgw2-19.0.0-3868.ga6507388.el9.x86 2.5 MB/s | 5.2 MB 00:02 2024-05-27T09:59:15.519 INFO:teuthology.orchestra.run.smithi042.stdout:(35/159): ceph-volume-19.0.0-3868.ga6507388.el9 888 kB/s | 273 kB 00:00 2024-05-27T09:59:15.534 INFO:teuthology.orchestra.run.smithi143.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga6 420 kB/s | 45 kB 00:00 2024-05-27T09:59:15.659 INFO:teuthology.orchestra.run.smithi143.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga650 1.1 MB/s | 136 kB 00:00 2024-05-27T09:59:15.776 INFO:teuthology.orchestra.run.smithi143.stdout:(21/159): python3-cephfs-19.0.0-3868.ga6507388. 1.4 MB/s | 161 kB 00:00 2024-05-27T09:59:15.986 INFO:teuthology.orchestra.run.smithi143.stdout:(22/159): python3-rados-19.0.0-3868.ga6507388.e 1.5 MB/s | 321 kB 00:00 2024-05-27T09:59:16.086 INFO:teuthology.orchestra.run.smithi042.stdout:(36/159): cephadm-19.0.0-3868.ga6507388.el9.noa 1.3 MB/s | 764 kB 00:00 2024-05-27T09:59:16.177 INFO:teuthology.orchestra.run.smithi143.stdout:(23/159): python3-rbd-19.0.0-3868.ga6507388.el9 1.5 MB/s | 301 kB 00:00 2024-05-27T09:59:16.213 INFO:teuthology.orchestra.run.smithi042.stdout:(37/159): cephfs-top-19.0.0-3868.ga6507388.el9. 145 kB/s | 18 kB 00:00 2024-05-27T09:59:16.285 INFO:teuthology.orchestra.run.smithi143.stdout:(24/159): python3-rgw-19.0.0-3868.ga6507388.el9 924 kB/s | 100 kB 00:00 2024-05-27T09:59:16.362 INFO:teuthology.orchestra.run.smithi042.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 544 kB/s | 80 kB 00:00 2024-05-27T09:59:16.369 INFO:teuthology.orchestra.run.smithi143.stdout:(25/159): rbd-fuse-19.0.0-3868.ga6507388.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-27T09:59:16.545 INFO:teuthology.orchestra.run.smithi042.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-27T09:59:16.612 INFO:teuthology.orchestra.run.smithi042.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-27T09:59:16.704 INFO:teuthology.orchestra.run.smithi042.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-27T09:59:16.963 INFO:teuthology.orchestra.run.smithi042.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 4.8 MB/s | 1.2 MB 00:00 2024-05-27T09:59:17.186 INFO:teuthology.orchestra.run.smithi042.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 480 kB/s | 106 kB 00:00 2024-05-27T09:59:17.247 INFO:teuthology.orchestra.run.smithi042.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-05-27T09:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-27T09:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-27T09:59:17.515 INFO:teuthology.orchestra.run.smithi042.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-05-27T09:59:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-05-27T09:59:17.600 INFO:teuthology.orchestra.run.smithi104.stdout:(26/159): rbd-mirror-19.0.0-3868.ga6507388.el9. 1.2 MB/s | 3.0 MB 00:02 2024-05-27T09:59:17.658 INFO:teuthology.orchestra.run.smithi042.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-27T09:59:17.661 INFO:teuthology.orchestra.run.smithi143.stdout:(26/159): rbd-mirror-19.0.0-3868.ga6507388.el9. 2.4 MB/s | 3.0 MB 00:01 2024-05-27T09:59:17.700 INFO:teuthology.orchestra.run.smithi042.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 531 kB/s | 22 kB 00:00 2024-05-27T09:59:17.750 INFO:teuthology.orchestra.run.smithi042.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-05-27T09:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 802 kB/s | 33 kB 00:00 2024-05-27T09:59:17.861 INFO:teuthology.orchestra.run.smithi143.stdout:(27/159): rbd-nbd-19.0.0-3868.ga6507388.el9.x86 854 kB/s | 170 kB 00:00 2024-05-27T09:59:17.909 INFO:teuthology.orchestra.run.smithi042.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-05-27T09:59:17.942 INFO:teuthology.orchestra.run.smithi042.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 565 kB/s | 18 kB 00:00 2024-05-27T09:59:17.961 INFO:teuthology.orchestra.run.smithi143.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 344 kB/s | 34 kB 00:00 2024-05-27T09:59:17.976 INFO:teuthology.orchestra.run.smithi042.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 342 kB/s | 11 kB 00:00 2024-05-27T09:59:18.035 INFO:teuthology.orchestra.run.smithi042.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-27T09:59:18.068 INFO:teuthology.orchestra.run.smithi042.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.4 MB/s | 45 kB 00:00 2024-05-27T09:59:18.078 INFO:teuthology.orchestra.run.smithi143.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3868.ga650738 1.2 MB/s | 143 kB 00:00 2024-05-27T09:59:18.127 INFO:teuthology.orchestra.run.smithi042.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-27T09:59:18.159 INFO:teuthology.orchestra.run.smithi104.stdout:(27/159): rbd-nbd-19.0.0-3868.ga6507388.el9.x86 305 kB/s | 170 kB 00:00 2024-05-27T09:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-05-27T09:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 622 kB/s | 25 kB 00:00 2024-05-27T09:59:18.270 INFO:teuthology.orchestra.run.smithi042.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-27T09:59:18.276 INFO:teuthology.orchestra.run.smithi104.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 295 kB/s | 34 kB 00:00 2024-05-27T09:59:18.320 INFO:teuthology.orchestra.run.smithi042.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.8 MB/s | 292 kB 00:00 2024-05-27T09:59:18.387 INFO:teuthology.orchestra.run.smithi042.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-27T09:59:18.429 INFO:teuthology.orchestra.run.smithi042.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 861 kB/s | 35 kB 00:00 2024-05-27T09:59:18.459 INFO:teuthology.orchestra.run.smithi104.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3868.ga650738 783 kB/s | 143 kB 00:00 2024-05-27T09:59:18.579 INFO:teuthology.orchestra.run.smithi042.stdout:(65/159): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-05-27T09:59:18.746 INFO:teuthology.orchestra.run.smithi042.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-05-27T09:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-27T09:59:18.943 INFO:teuthology.orchestra.run.smithi104.stdout:(30/159): ceph-osd-19.0.0-3868.ga6507388.el9.x8 1.2 MB/s | 18 MB 00:14 2024-05-27T09:59:18.972 INFO:teuthology.orchestra.run.smithi042.stdout:(68/159): ceph-mgr-diskprediction-local-19.0.0- 1.5 MB/s | 7.4 MB 00:05 2024-05-27T09:59:19.245 INFO:teuthology.orchestra.run.smithi143.stdout:(30/159): ceph-osd-19.0.0-3868.ga6507388.el9.x8 1.5 MB/s | 18 MB 00:11 2024-05-27T09:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout:(69/159): ceph-test-19.0.0-3868.ga6507388.el9.x 3.1 MB/s | 43 MB 00:14 2024-05-27T09:59:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 1.3 MB/s | 911 kB 00:00 2024-05-27T09:59:19.581 INFO:teuthology.orchestra.run.smithi042.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-27T09:59:19.599 INFO:teuthology.orchestra.run.smithi042.stdout:(72/159): postgresql-private-libs-13.14-1.el9.x 213 kB/s | 133 kB 00:00 2024-05-27T09:59:19.616 INFO:teuthology.orchestra.run.smithi042.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-05-27T09:59:19.658 INFO:teuthology.orchestra.run.smithi042.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-27T09:59:19.709 INFO:teuthology.orchestra.run.smithi042.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-05-27T09:59:19.758 INFO:teuthology.orchestra.run.smithi042.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T09:59:19.817 INFO:teuthology.orchestra.run.smithi042.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 446 kB/s | 26 kB 00:00 2024-05-27T09:59:19.859 INFO:teuthology.orchestra.run.smithi042.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 469 kB/s | 19 kB 00:00 2024-05-27T09:59:19.912 INFO:teuthology.orchestra.run.smithi042.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-27T09:59:19.951 INFO:teuthology.orchestra.run.smithi042.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 852 kB/s | 35 kB 00:00 2024-05-27T09:59:20.077 INFO:teuthology.orchestra.run.smithi042.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 9.5 MB/s | 5.8 MB 00:00 2024-05-27T09:59:20.227 INFO:teuthology.orchestra.run.smithi042.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.5 MB/s | 6.0 MB 00:00 2024-05-27T09:59:20.260 INFO:teuthology.orchestra.run.smithi042.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 250 kB 00:00 2024-05-27T09:59:20.295 INFO:teuthology.orchestra.run.smithi042.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-27T09:59:20.319 INFO:teuthology.orchestra.run.smithi042.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-27T09:59:20.336 INFO:teuthology.orchestra.run.smithi042.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-27T09:59:20.345 INFO:teuthology.orchestra.run.smithi143.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3868.ga6507 1.6 MB/s | 3.5 MB 00:02 2024-05-27T09:59:20.370 INFO:teuthology.orchestra.run.smithi042.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-27T09:59:20.403 INFO:teuthology.orchestra.run.smithi042.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-05-27T09:59:20.437 INFO:teuthology.orchestra.run.smithi042.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-27T09:59:20.462 INFO:teuthology.orchestra.run.smithi042.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 878 kB/s | 51 kB 00:00 2024-05-27T09:59:20.529 INFO:teuthology.orchestra.run.smithi042.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 588 kB/s | 54 kB 00:00 2024-05-27T09:59:20.571 INFO:teuthology.orchestra.run.smithi042.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T09:59:20.606 INFO:teuthology.orchestra.run.smithi143.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga6 1.0 MB/s | 274 kB 00:00 2024-05-27T09:59:20.613 INFO:teuthology.orchestra.run.smithi042.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T09:59:20.664 INFO:teuthology.orchestra.run.smithi042.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-27T09:59:20.705 INFO:teuthology.orchestra.run.smithi143.stdout:(33/159): ceph-mgr-rook-19.0.0-3868.ga6507388.e 496 kB/s | 49 kB 00:00 2024-05-27T09:59:20.706 INFO:teuthology.orchestra.run.smithi042.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 880 kB/s | 36 kB 00:00 2024-05-27T09:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout:(96/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-05-27T09:59:20.822 INFO:teuthology.orchestra.run.smithi143.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3868.ga 142 kB/s | 16 kB 00:00 2024-05-27T09:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 408 kB/s | 47 kB 00:00 2024-05-27T09:59:20.857 INFO:teuthology.orchestra.run.smithi042.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-27T09:59:20.899 INFO:teuthology.orchestra.run.smithi042.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 775 kB/s | 58 kB 00:00 2024-05-27T09:59:20.924 INFO:teuthology.orchestra.run.smithi042.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-05-27T09:59:20.949 INFO:teuthology.orchestra.run.smithi042.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-27T09:59:20.975 INFO:teuthology.orchestra.run.smithi042.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-27T09:59:21.000 INFO:teuthology.orchestra.run.smithi042.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 452 kB/s | 22 kB 00:00 2024-05-27T09:59:21.067 INFO:teuthology.orchestra.run.smithi042.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 9.3 MB/s | 863 kB 00:00 2024-05-27T09:59:21.122 INFO:teuthology.orchestra.run.smithi143.stdout:(35/159): ceph-volume-19.0.0-3868.ga6507388.el9 913 kB/s | 273 kB 00:00 2024-05-27T09:59:21.442 INFO:teuthology.orchestra.run.smithi042.stdout:(105/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-05-27T09:59:21.619 INFO:teuthology.orchestra.run.smithi042.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 113 kB/s | 19 kB 00:00 2024-05-27T09:59:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 843 kB/s | 548 kB 00:00 2024-05-27T09:59:21.678 INFO:teuthology.orchestra.run.smithi042.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 576 kB/s | 309 kB 00:00 2024-05-27T09:59:21.780 INFO:teuthology.orchestra.run.smithi143.stdout:(36/159): cephadm-19.0.0-3868.ga6507388.el9.noa 1.1 MB/s | 764 kB 00:00 2024-05-27T09:59:21.826 INFO:teuthology.orchestra.run.smithi104.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3868.ga6507 1.1 MB/s | 3.5 MB 00:03 2024-05-27T09:59:21.885 INFO:teuthology.orchestra.run.smithi042.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 127 kB/s | 26 kB 00:00 2024-05-27T09:59:21.906 INFO:teuthology.orchestra.run.smithi143.stdout:(37/159): cephfs-top-19.0.0-3868.ga6507388.el9. 147 kB/s | 18 kB 00:00 2024-05-27T09:59:21.910 INFO:teuthology.orchestra.run.smithi042.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 211 kB/s | 49 kB 00:00 2024-05-27T09:59:22.006 INFO:teuthology.orchestra.run.smithi143.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 812 kB/s | 80 kB 00:00 2024-05-27T09:59:22.111 INFO:teuthology.orchestra.run.smithi042.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 300 kB/s | 67 kB 00:00 2024-05-27T09:59:22.131 INFO:teuthology.orchestra.run.smithi143.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-27T09:59:22.160 INFO:teuthology.orchestra.run.smithi104.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga6 826 kB/s | 274 kB 00:00 2024-05-27T09:59:22.173 INFO:teuthology.orchestra.run.smithi143.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 195 kB 00:00 2024-05-27T09:59:22.216 INFO:teuthology.orchestra.run.smithi143.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-05-27T09:59:22.243 INFO:teuthology.orchestra.run.smithi104.stdout:(33/159): ceph-mgr-rook-19.0.0-3868.ga6507388.e 599 kB/s | 49 kB 00:00 2024-05-27T09:59:22.249 INFO:teuthology.orchestra.run.smithi042.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 450 kB/s | 151 kB 00:00 2024-05-27T09:59:22.322 INFO:teuthology.orchestra.run.smithi143.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-27T09:59:22.327 INFO:teuthology.orchestra.run.smithi104.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3868.ga 199 kB/s | 16 kB 00:00 2024-05-27T09:59:22.461 INFO:teuthology.orchestra.run.smithi042.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-05-27T09:59:22.512 INFO:teuthology.orchestra.run.smithi042.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.1 MB/s | 839 kB 00:00 2024-05-27T09:59:22.525 INFO:teuthology.orchestra.run.smithi143.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 529 kB/s | 106 kB 00:00 2024-05-27T09:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-05-27T09:59:22.567 INFO:teuthology.orchestra.run.smithi143.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-05-27T09:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 406 kB/s | 43 kB 00:00 2024-05-27T09:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 391 kB/s | 32 kB 00:00 2024-05-27T09:59:22.609 INFO:teuthology.orchestra.run.smithi143.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-05-27T09:59:22.651 INFO:teuthology.orchestra.run.smithi143.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-05-27T09:59:22.652 INFO:teuthology.orchestra.run.smithi104.stdout:(35/159): ceph-volume-19.0.0-3868.ga6507388.el9 842 kB/s | 273 kB 00:00 2024-05-27T09:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 170 kB/s | 15 kB 00:00 2024-05-27T09:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-27T09:59:22.730 INFO:teuthology.orchestra.run.smithi042.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-27T09:59:22.764 INFO:teuthology.orchestra.run.smithi042.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-27T09:59:22.797 INFO:teuthology.orchestra.run.smithi042.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-27T09:59:22.868 INFO:teuthology.orchestra.run.smithi143.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-05-27T09:59:23.014 INFO:teuthology.orchestra.run.smithi042.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-05-27T09:59:23.048 INFO:teuthology.orchestra.run.smithi042.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 515 kB/s | 144 kB 00:00 2024-05-27T09:59:23.068 INFO:teuthology.orchestra.run.smithi143.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-05-27T09:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 39 kB/s | 11 kB 00:00 2024-05-27T09:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 271 kB/s | 18 kB 00:00 2024-05-27T09:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 256 kB/s | 23 kB 00:00 2024-05-27T09:59:23.190 INFO:teuthology.orchestra.run.smithi042.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 169 kB/s | 19 kB 00:00 2024-05-27T09:59:23.216 INFO:teuthology.orchestra.run.smithi042.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-05-27T09:59:23.218 INFO:teuthology.orchestra.run.smithi143.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 722 kB/s | 107 kB 00:00 2024-05-27T09:59:23.324 INFO:teuthology.orchestra.run.smithi042.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-05-27T09:59:23.352 INFO:teuthology.orchestra.run.smithi143.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 164 kB/s | 22 kB 00:00 2024-05-27T09:59:23.375 INFO:teuthology.orchestra.run.smithi042.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 253 kB/s | 46 kB 00:00 2024-05-27T09:59:23.408 INFO:teuthology.orchestra.run.smithi042.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 412 kB/s | 79 kB 00:00 2024-05-27T09:59:23.433 INFO:teuthology.orchestra.run.smithi042.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 539 kB/s | 58 kB 00:00 2024-05-27T09:59:23.435 INFO:teuthology.orchestra.run.smithi104.stdout:(36/159): cephadm-19.0.0-3868.ga6507388.el9.noa 976 kB/s | 764 kB 00:00 2024-05-27T09:59:23.484 INFO:teuthology.orchestra.run.smithi042.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-05-27T09:59:23.509 INFO:teuthology.orchestra.run.smithi042.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 166 kB/s | 16 kB 00:00 2024-05-27T09:59:23.527 INFO:teuthology.orchestra.run.smithi143.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-05-27T09:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 900 kB/s | 90 kB 00:00 2024-05-27T09:59:23.536 INFO:teuthology.orchestra.run.smithi104.stdout:(37/159): cephfs-top-19.0.0-3868.ga6507388.el9. 184 kB/s | 18 kB 00:00 2024-05-27T09:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 435 kB/s | 28 kB 00:00 2024-05-27T09:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 377 kB/s | 31 kB 00:00 2024-05-27T09:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-27T09:59:23.644 INFO:teuthology.orchestra.run.smithi104.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 746 kB/s | 80 kB 00:00 2024-05-27T09:59:23.652 INFO:teuthology.orchestra.run.smithi042.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-27T09:59:23.669 INFO:teuthology.orchestra.run.smithi042.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 494 kB/s | 36 kB 00:00 2024-05-27T09:59:23.669 INFO:teuthology.orchestra.run.smithi143.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 235 kB/s | 33 kB 00:00 2024-05-27T09:59:23.686 INFO:teuthology.orchestra.run.smithi042.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 874 kB/s | 51 kB 00:00 2024-05-27T09:59:23.753 INFO:teuthology.orchestra.run.smithi042.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-05-27T09:59:23.762 INFO:teuthology.orchestra.run.smithi104.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-05-27T09:59:23.778 INFO:teuthology.orchestra.run.smithi042.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 833 kB/s | 90 kB 00:00 2024-05-27T09:59:23.811 INFO:teuthology.orchestra.run.smithi042.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-27T09:59:23.812 INFO:teuthology.orchestra.run.smithi104.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-05-27T09:59:23.837 INFO:teuthology.orchestra.run.smithi042.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 257 kB/s | 20 kB 00:00 2024-05-27T09:59:23.862 INFO:teuthology.orchestra.run.smithi104.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-27T09:59:23.879 INFO:teuthology.orchestra.run.smithi042.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-27T09:59:23.928 INFO:teuthology.orchestra.run.smithi143.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-27T09:59:23.946 INFO:teuthology.orchestra.run.smithi104.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-27T09:59:23.988 INFO:teuthology.orchestra.run.smithi104.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.6 MB/s | 106 kB 00:00 2024-05-27T09:59:24.021 INFO:teuthology.orchestra.run.smithi042.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-05-27T09:59:24.053 INFO:teuthology.orchestra.run.smithi143.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 18 kB 00:00 2024-05-27T09:59:24.170 INFO:teuthology.orchestra.run.smithi143.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 97 kB/s | 11 kB 00:00 2024-05-27T09:59:24.329 INFO:teuthology.orchestra.run.smithi143.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-27T09:59:24.404 INFO:teuthology.orchestra.run.smithi042.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-05-27T09:59:24.455 INFO:teuthology.orchestra.run.smithi104.stdout:(44/159): ceph-test-19.0.0-3868.ga6507388.el9.x 2.4 MB/s | 43 MB 00:17 2024-05-27T09:59:24.471 INFO:teuthology.orchestra.run.smithi143.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 323 kB/s | 45 kB 00:00 2024-05-27T09:59:24.480 INFO:teuthology.orchestra.run.smithi104.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 275 kB/s | 135 kB 00:00 2024-05-27T09:59:24.514 INFO:teuthology.orchestra.run.smithi104.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-27T09:59:24.630 INFO:teuthology.orchestra.run.smithi042.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 8.0 MB/s | 1.8 MB 00:00 2024-05-27T09:59:24.630 INFO:teuthology.orchestra.run.smithi104.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-05-27T09:59:24.654 INFO:teuthology.orchestra.run.smithi143.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.6 MB/s | 662 kB 00:00 2024-05-27T09:59:24.672 INFO:teuthology.orchestra.run.smithi104.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-05-27T09:59:24.688 INFO:teuthology.orchestra.run.smithi042.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 718 kB/s | 41 kB 00:00 2024-05-27T09:59:24.714 INFO:teuthology.orchestra.run.smithi104.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-05-27T09:59:24.747 INFO:teuthology.orchestra.run.smithi042.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-27T09:59:24.764 INFO:teuthology.orchestra.run.smithi104.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 439 kB/s | 22 kB 00:00 2024-05-27T09:59:24.814 INFO:teuthology.orchestra.run.smithi042.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 556 kB/s | 37 kB 00:00 2024-05-27T09:59:24.823 INFO:teuthology.orchestra.run.smithi104.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-05-27T09:59:24.830 INFO:teuthology.orchestra.run.smithi143.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 3.3 MB/s | 582 kB 00:00 2024-05-27T09:59:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-05-27T09:59:24.873 INFO:teuthology.orchestra.run.smithi104.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-05-27T09:59:24.899 INFO:teuthology.orchestra.run.smithi104.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 439 kB/s | 33 kB 00:00 2024-05-27T09:59:24.940 INFO:teuthology.orchestra.run.smithi042.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.9 MB/s | 260 kB 00:00 2024-05-27T09:59:24.941 INFO:teuthology.orchestra.run.smithi104.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 447 kB/s | 18 kB 00:00 2024-05-27T09:59:24.963 INFO:teuthology.orchestra.run.smithi143.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 193 kB/s | 25 kB 00:00 2024-05-27T09:59:24.983 INFO:teuthology.orchestra.run.smithi104.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 272 kB/s | 11 kB 00:00 2024-05-27T09:59:25.066 INFO:teuthology.orchestra.run.smithi104.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-05-27T09:59:25.091 INFO:teuthology.orchestra.run.smithi104.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-05-27T09:59:25.105 INFO:teuthology.orchestra.run.smithi143.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 339 kB/s | 47 kB 00:00 2024-05-27T09:59:25.107 INFO:teuthology.orchestra.run.smithi042.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-27T09:59:25.117 INFO:teuthology.orchestra.run.smithi104.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 45 kB 00:00 2024-05-27T09:59:25.159 INFO:teuthology.orchestra.run.smithi104.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-27T09:59:25.202 INFO:teuthology.orchestra.run.smithi104.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 582 kB 00:00 2024-05-27T09:59:25.226 INFO:teuthology.orchestra.run.smithi104.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-05-27T09:59:25.251 INFO:teuthology.orchestra.run.smithi104.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 959 kB/s | 47 kB 00:00 2024-05-27T09:59:25.272 INFO:teuthology.orchestra.run.smithi143.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-05-27T09:59:25.293 INFO:teuthology.orchestra.run.smithi104.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-27T09:59:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-05-27T09:59:25.418 INFO:teuthology.orchestra.run.smithi104.stdout:(64/159): ceph-mgr-diskprediction-local-19.0.0- 1.1 MB/s | 7.4 MB 00:06 2024-05-27T09:59:25.431 INFO:teuthology.orchestra.run.smithi143.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-27T09:59:25.435 INFO:teuthology.orchestra.run.smithi104.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 250 kB/s | 35 kB 00:00 2024-05-27T09:59:25.469 INFO:teuthology.orchestra.run.smithi104.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 870 kB/s | 188 kB 00:00 2024-05-27T09:59:25.569 INFO:teuthology.orchestra.run.smithi104.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-27T09:59:25.615 INFO:teuthology.orchestra.run.smithi143.stdout:(64/159): ceph-mgr-diskprediction-local-19.0.0- 1.2 MB/s | 7.4 MB 00:06 2024-05-27T09:59:25.636 INFO:teuthology.orchestra.run.smithi104.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-05-27T09:59:25.674 INFO:teuthology.orchestra.run.smithi042.stdout:(158/159): librbd1-19.0.0-3868.ga6507388.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-05-27T09:59:25.728 INFO:teuthology.orchestra.run.smithi104.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-05-27T09:59:25.740 INFO:teuthology.orchestra.run.smithi143.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 115 kB/s | 35 kB 00:00 2024-05-27T09:59:25.762 INFO:teuthology.orchestra.run.smithi104.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-27T09:59:25.807 INFO:teuthology.orchestra.run.smithi042.stdout:(159/159): librados2-19.0.0-3868.ga6507388.el9. 1.7 MB/s | 3.3 MB 00:01 2024-05-27T09:59:25.816 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:59:25.817 INFO:teuthology.orchestra.run.smithi042.stdout:Total 8.6 MB/s | 219 MB 00:25 2024-05-27T09:59:25.845 INFO:teuthology.orchestra.run.smithi104.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-27T09:59:25.921 INFO:teuthology.orchestra.run.smithi104.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-05-27T09:59:26.021 INFO:teuthology.orchestra.run.smithi104.stdout:(73/159): openblas-openmp-0.3.21-2.el9.x86_64.r 7.9 MB/s | 4.7 MB 00:00 2024-05-27T09:59:26.024 INFO:teuthology.orchestra.run.smithi143.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-27T09:59:26.063 INFO:teuthology.orchestra.run.smithi104.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-27T09:59:26.099 INFO:teuthology.orchestra.run.smithi143.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-05-27T09:59:26.113 INFO:teuthology.orchestra.run.smithi104.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-05-27T09:59:26.163 INFO:teuthology.orchestra.run.smithi104.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T09:59:26.280 INFO:teuthology.orchestra.run.smithi104.stdout:(77/159): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-05-27T09:59:26.306 INFO:teuthology.orchestra.run.smithi104.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 182 kB/s | 26 kB 00:00 2024-05-27T09:59:26.325 INFO:teuthology.orchestra.run.smithi143.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 911 kB 00:00 2024-05-27T09:59:26.331 INFO:teuthology.orchestra.run.smithi104.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 388 kB/s | 19 kB 00:00 2024-05-27T09:59:26.356 INFO:teuthology.orchestra.run.smithi104.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-27T09:59:26.382 INFO:teuthology.orchestra.run.smithi104.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 702 kB/s | 35 kB 00:00 2024-05-27T09:59:26.432 INFO:teuthology.orchestra.run.smithi104.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-27T09:59:26.433 INFO:teuthology.orchestra.run.smithi143.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-05-27T09:59:26.499 INFO:teuthology.orchestra.run.smithi104.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-27T09:59:26.540 INFO:teuthology.orchestra.run.smithi104.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-27T09:59:26.550 INFO:teuthology.orchestra.run.smithi143.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 594 kB/s | 133 kB 00:00 2024-05-27T09:59:26.649 INFO:teuthology.orchestra.run.smithi104.stdout:(85/159): python3-babel-2.9.1-2.el9.noarch.rpm 8.2 MB/s | 6.0 MB 00:00 2024-05-27T09:59:26.675 INFO:teuthology.orchestra.run.smithi104.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 317 kB/s | 42 kB 00:00 2024-05-27T09:59:26.708 INFO:teuthology.orchestra.run.smithi104.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-27T09:59:26.742 INFO:teuthology.orchestra.run.smithi104.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-27T09:59:26.742 INFO:teuthology.orchestra.run.smithi143.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-05-27T09:59:26.775 INFO:teuthology.orchestra.run.smithi104.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-27T09:59:26.809 INFO:teuthology.orchestra.run.smithi104.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 772 kB/s | 51 kB 00:00 2024-05-27T09:59:26.834 INFO:teuthology.orchestra.run.smithi104.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 930 kB/s | 54 kB 00:00 2024-05-27T09:59:26.859 INFO:teuthology.orchestra.run.smithi143.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-05-27T09:59:26.876 INFO:teuthology.orchestra.run.smithi104.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T09:59:26.918 INFO:teuthology.orchestra.run.smithi104.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T09:59:26.960 INFO:teuthology.orchestra.run.smithi143.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 404 kB/s | 87 kB 00:00 2024-05-27T09:59:27.018 INFO:teuthology.orchestra.run.smithi104.stdout:(94/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.7 MB/s | 5.1 MB 00:00 2024-05-27T09:59:27.077 INFO:teuthology.orchestra.run.smithi104.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-05-27T09:59:27.085 INFO:teuthology.orchestra.run.smithi143.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 492 kB/s | 61 kB 00:00 2024-05-27T09:59:27.111 INFO:teuthology.orchestra.run.smithi104.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-05-27T09:59:27.136 INFO:teuthology.orchestra.run.smithi104.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 821 kB/s | 47 kB 00:00 2024-05-27T09:59:27.252 INFO:teuthology.orchestra.run.smithi143.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-27T09:59:27.286 INFO:teuthology.orchestra.run.smithi143.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-05-27T09:59:27.370 INFO:teuthology.orchestra.run.smithi104.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-05-27T09:59:27.395 INFO:teuthology.orchestra.run.smithi104.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 224 kB/s | 58 kB 00:00 2024-05-27T09:59:27.421 INFO:teuthology.orchestra.run.smithi104.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-27T09:59:27.446 INFO:teuthology.orchestra.run.smithi104.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-27T09:59:27.472 INFO:teuthology.orchestra.run.smithi104.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-05-27T09:59:27.503 INFO:teuthology.orchestra.run.smithi143.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 996 kB/s | 248 kB 00:00 2024-05-27T09:59:27.528 INFO:teuthology.orchestra.run.smithi143.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 134 kB/s | 26 kB 00:00 2024-05-27T09:59:27.703 INFO:teuthology.orchestra.run.smithi143.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 96 kB/s | 19 kB 00:00 2024-05-27T09:59:27.803 INFO:teuthology.orchestra.run.smithi143.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 840 kB/s | 172 kB 00:00 2024-05-27T09:59:27.838 INFO:teuthology.orchestra.run.smithi104.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-05-27T09:59:27.855 INFO:teuthology.orchestra.run.smithi104.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 55 kB/s | 22 kB 00:00 2024-05-27T09:59:27.896 INFO:teuthology.orchestra.run.smithi143.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 182 kB/s | 35 kB 00:00 2024-05-27T09:59:28.013 INFO:teuthology.orchestra.run.smithi104.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-05-27T09:59:28.055 INFO:teuthology.orchestra.run.smithi104.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 477 kB/s | 19 kB 00:00 2024-05-27T09:59:28.062 INFO:teuthology.orchestra.run.smithi143.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-05-27T09:59:28.089 INFO:teuthology.orchestra.run.smithi104.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-05-27T09:59:28.139 INFO:teuthology.orchestra.run.smithi104.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-05-27T09:59:28.223 INFO:teuthology.orchestra.run.smithi104.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-05-27T09:59:28.238 INFO:teuthology.orchestra.run.smithi143.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-27T09:59:28.272 INFO:teuthology.orchestra.run.smithi143.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-05-27T09:59:28.273 INFO:teuthology.orchestra.run.smithi104.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-27T09:59:28.323 INFO:teuthology.orchestra.run.smithi104.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-27T09:59:28.399 INFO:teuthology.orchestra.run.smithi104.stdout:(112/159): protobuf-compiler-3.14.0-13.el9.x86_ 932 kB/s | 863 kB 00:00 2024-05-27T09:59:28.447 INFO:teuthology.orchestra.run.smithi143.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 371 kB/s | 77 kB 00:00 2024-05-27T09:59:28.472 INFO:teuthology.orchestra.run.smithi143.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 290 kB/s | 42 kB 00:00 2024-05-27T09:59:28.474 INFO:teuthology.orchestra.run.smithi104.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-05-27T09:59:28.524 INFO:teuthology.orchestra.run.smithi104.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 896 kB/s | 43 kB 00:00 2024-05-27T09:59:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 788 kB/s | 32 kB 00:00 2024-05-27T09:59:28.608 INFO:teuthology.orchestra.run.smithi104.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 367 kB/s | 15 kB 00:00 2024-05-27T09:59:28.659 INFO:teuthology.orchestra.run.smithi104.stdout:(117/159): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-27T09:59:28.709 INFO:teuthology.orchestra.run.smithi104.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-27T09:59:28.826 INFO:teuthology.orchestra.run.smithi104.stdout:(119/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-05-27T09:59:28.859 INFO:teuthology.orchestra.run.smithi104.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-05-27T09:59:28.893 INFO:teuthology.orchestra.run.smithi104.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-05-27T09:59:28.926 INFO:teuthology.orchestra.run.smithi104.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-27T09:59:28.960 INFO:teuthology.orchestra.run.smithi104.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-27T09:59:28.964 INFO:teuthology.orchestra.run.smithi143.stdout:(87/159): ceph-test-19.0.0-3868.ga6507388.el9.x 2.2 MB/s | 43 MB 00:20 2024-05-27T09:59:28.985 INFO:teuthology.orchestra.run.smithi104.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-05-27T09:59:29.011 INFO:teuthology.orchestra.run.smithi104.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 357 kB/s | 18 kB 00:00 2024-05-27T09:59:29.036 INFO:teuthology.orchestra.run.smithi104.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-05-27T09:59:29.070 INFO:teuthology.orchestra.run.smithi104.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 337 kB/s | 19 kB 00:00 2024-05-27T09:59:29.095 INFO:teuthology.orchestra.run.smithi104.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 339 kB/s | 20 kB 00:00 2024-05-27T09:59:29.131 INFO:teuthology.orchestra.run.smithi143.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 394 kB/s | 269 kB 00:00 2024-05-27T09:59:29.137 INFO:teuthology.orchestra.run.smithi104.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-27T09:59:29.192 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-05-27T09:59:29.204 INFO:teuthology.orchestra.run.smithi104.stdout:(130/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-05-27T09:59:29.229 INFO:teuthology.orchestra.run.smithi104.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 865 kB/s | 79 kB 00:00 2024-05-27T09:59:29.231 INFO:teuthology.orchestra.run.smithi143.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 228 kB/s | 159 kB 00:00 2024-05-27T09:59:29.246 INFO:teuthology.orchestra.run.smithi104.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-27T09:59:29.288 INFO:teuthology.orchestra.run.smithi104.stdout:(133/159): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-05-27T09:59:29.305 INFO:teuthology.orchestra.run.smithi104.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-05-27T09:59:29.332 INFO:teuthology.orchestra.run.smithi143.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 258 kB/s | 51 kB 00:00 2024-05-27T09:59:29.360 INFO:teuthology.orchestra.run.smithi104.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-27T09:59:29.365 INFO:teuthology.orchestra.run.smithi143.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 851 kB/s | 279 kB 00:00 2024-05-27T09:59:29.389 INFO:teuthology.orchestra.run.smithi104.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-05-27T09:59:29.414 INFO:teuthology.orchestra.run.smithi104.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 586 kB/s | 31 kB 00:00 2024-05-27T09:59:29.439 INFO:teuthology.orchestra.run.smithi104.stdout:(138/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 599 kB/s | 90 kB 00:00 2024-05-27T09:59:29.449 INFO:teuthology.orchestra.run.smithi143.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 249 kB/s | 54 kB 00:00 2024-05-27T09:59:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-05-27T09:59:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-05-27T09:59:29.464 INFO:teuthology.orchestra.run.smithi104.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-27T09:59:29.490 INFO:teuthology.orchestra.run.smithi104.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 718 kB/s | 36 kB 00:00 2024-05-27T09:59:29.515 INFO:teuthology.orchestra.run.smithi104.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-27T09:59:29.540 INFO:teuthology.orchestra.run.smithi104.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-05-27T09:59:29.565 INFO:teuthology.orchestra.run.smithi104.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-05-27T09:59:29.599 INFO:teuthology.orchestra.run.smithi104.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-27T09:59:29.608 INFO:teuthology.orchestra.run.smithi143.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 226 kB/s | 42 kB 00:00 2024-05-27T09:59:29.624 INFO:teuthology.orchestra.run.smithi104.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-05-27T09:59:29.633 INFO:teuthology.orchestra.run.smithi143.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 319 kB/s | 44 kB 00:00 2024-05-27T09:59:29.658 INFO:teuthology.orchestra.run.smithi104.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-27T09:59:29.692 INFO:teuthology.orchestra.run.smithi104.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-05-27T09:59:29.775 INFO:teuthology.orchestra.run.smithi143.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 350 kB/s | 36 kB 00:00 2024-05-27T09:59:29.800 INFO:teuthology.orchestra.run.smithi143.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-05-27T09:59:29.934 INFO:teuthology.orchestra.run.smithi143.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 300 kB/s | 47 kB 00:00 2024-05-27T09:59:30.000 INFO:teuthology.orchestra.run.smithi104.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-05-27T09:59:30.009 INFO:teuthology.orchestra.run.smithi143.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 305 kB 00:00 2024-05-27T09:59:30.076 INFO:teuthology.orchestra.run.smithi143.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 408 kB/s | 58 kB 00:00 2024-05-27T09:59:30.152 INFO:teuthology.orchestra.run.smithi143.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 668 kB/s | 94 kB 00:00 2024-05-27T09:59:30.177 INFO:teuthology.orchestra.run.smithi143.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 646 kB/s | 64 kB 00:00 2024-05-27T09:59:30.200 INFO:teuthology.orchestra.run.smithi104.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-05-27T09:59:30.301 INFO:teuthology.orchestra.run.smithi104.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-05-27T09:59:30.343 INFO:teuthology.orchestra.run.smithi104.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-27T09:59:30.393 INFO:teuthology.orchestra.run.smithi104.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-27T09:59:30.411 INFO:teuthology.orchestra.run.smithi143.stdout:(102/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-05-27T09:59:30.435 INFO:teuthology.orchestra.run.smithi104.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 899 kB/s | 37 kB 00:00 2024-05-27T09:59:30.444 INFO:teuthology.orchestra.run.smithi143.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 153 kB/s | 45 kB 00:00 2024-05-27T09:59:30.470 INFO:teuthology.orchestra.run.smithi143.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 87 kB/s | 22 kB 00:00 2024-05-27T09:59:30.543 INFO:teuthology.orchestra.run.smithi104.stdout:(154/159): librbd1-19.0.0-3868.ga6507388.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-05-27T09:59:30.577 INFO:teuthology.orchestra.run.smithi104.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.4 MB/s | 207 kB 00:00 2024-05-27T09:59:30.669 INFO:teuthology.orchestra.run.smithi104.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-05-27T09:59:30.694 INFO:teuthology.orchestra.run.smithi104.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.7 MB/s | 260 kB 00:00 2024-05-27T09:59:30.762 INFO:teuthology.orchestra.run.smithi143.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-05-27T09:59:30.786 INFO:teuthology.orchestra.run.smithi104.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-05-27T09:59:30.887 INFO:teuthology.orchestra.run.smithi143.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 157 kB/s | 19 kB 00:00 2024-05-27T09:59:30.920 INFO:teuthology.orchestra.run.smithi143.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-05-27T09:59:30.954 INFO:teuthology.orchestra.run.smithi143.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 639 kB/s | 309 kB 00:00 2024-05-27T09:59:30.996 INFO:teuthology.orchestra.run.smithi143.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 348 kB/s | 26 kB 00:00 2024-05-27T09:59:31.080 INFO:teuthology.orchestra.run.smithi143.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 402 kB/s | 49 kB 00:00 2024-05-27T09:59:31.178 INFO:teuthology.orchestra.run.smithi104.stdout:(159/159): librados2-19.0.0-3868.ga6507388.el9. 2.2 MB/s | 3.3 MB 00:01 2024-05-27T09:59:31.184 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:59:31.185 INFO:teuthology.orchestra.run.smithi104.stdout:Total 7.0 MB/s | 219 MB 00:31 2024-05-27T09:59:31.196 INFO:teuthology.orchestra.run.smithi143.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-27T09:59:31.222 INFO:teuthology.orchestra.run.smithi143.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 308 kB/s | 67 kB 00:00 2024-05-27T09:59:31.256 INFO:teuthology.orchestra.run.smithi143.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 929 kB/s | 151 kB 00:00 2024-05-27T09:59:31.323 INFO:teuthology.orchestra.run.smithi143.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.4 MB/s | 839 kB 00:00 2024-05-27T09:59:31.365 INFO:teuthology.orchestra.run.smithi143.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-05-27T09:59:31.390 INFO:teuthology.orchestra.run.smithi143.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-05-27T09:59:31.424 INFO:teuthology.orchestra.run.smithi143.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 340 kB/s | 32 kB 00:00 2024-05-27T09:59:31.485 INFO:teuthology.orchestra.run.smithi143.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 160 kB/s | 15 kB 00:00 2024-05-27T09:59:31.532 INFO:teuthology.orchestra.run.smithi143.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-05-27T09:59:31.557 INFO:teuthology.orchestra.run.smithi143.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-27T09:59:31.591 INFO:teuthology.orchestra.run.smithi143.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-27T09:59:31.633 INFO:teuthology.orchestra.run.smithi143.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.8 MB/s | 206 kB 00:00 2024-05-27T09:59:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-27T09:59:31.742 INFO:teuthology.orchestra.run.smithi143.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 964 kB/s | 144 kB 00:00 2024-05-27T09:59:31.767 INFO:teuthology.orchestra.run.smithi143.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 87 kB/s | 11 kB 00:00 2024-05-27T09:59:31.809 INFO:teuthology.orchestra.run.smithi143.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 269 kB/s | 18 kB 00:00 2024-05-27T09:59:31.835 INFO:teuthology.orchestra.run.smithi143.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 255 kB/s | 23 kB 00:00 2024-05-27T09:59:31.860 INFO:teuthology.orchestra.run.smithi143.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-05-27T09:59:31.902 INFO:teuthology.orchestra.run.smithi143.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 300 kB/s | 20 kB 00:00 2024-05-27T09:59:31.952 INFO:teuthology.orchestra.run.smithi143.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-05-27T09:59:31.977 INFO:teuthology.orchestra.run.smithi143.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-05-27T09:59:32.003 INFO:teuthology.orchestra.run.smithi143.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 877 kB/s | 79 kB 00:00 2024-05-27T09:59:32.044 INFO:teuthology.orchestra.run.smithi143.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 894 kB/s | 58 kB 00:00 2024-05-27T09:59:32.078 INFO:teuthology.orchestra.run.smithi143.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-27T09:59:32.103 INFO:teuthology.orchestra.run.smithi143.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-05-27T09:59:32.129 INFO:teuthology.orchestra.run.smithi143.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-27T09:59:32.171 INFO:teuthology.orchestra.run.smithi143.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 431 kB/s | 28 kB 00:00 2024-05-27T09:59:32.196 INFO:teuthology.orchestra.run.smithi143.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 341 kB/s | 31 kB 00:00 2024-05-27T09:59:32.221 INFO:teuthology.orchestra.run.smithi143.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-27T09:59:32.231 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-05-27T09:59:32.231 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-05-27T09:59:32.246 INFO:teuthology.orchestra.run.smithi143.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 816 kB/s | 59 kB 00:00 2024-05-27T09:59:32.288 INFO:teuthology.orchestra.run.smithi143.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 547 kB/s | 36 kB 00:00 2024-05-27T09:59:32.314 INFO:teuthology.orchestra.run.smithi143.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 564 kB/s | 51 kB 00:00 2024-05-27T09:59:32.347 INFO:teuthology.orchestra.run.smithi143.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-27T09:59:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-05-27T09:59:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-05-27T09:59:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-05-27T09:59:32.499 INFO:teuthology.orchestra.run.smithi143.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-27T09:59:32.644 INFO:teuthology.orchestra.run.smithi143.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 6.7 MB/s | 1.6 MB 00:00 2024-05-27T09:59:32.866 INFO:teuthology.orchestra.run.smithi143.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 10 MB/s | 2.3 MB 00:00 2024-05-27T09:59:32.958 INFO:teuthology.orchestra.run.smithi143.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-05-27T09:59:33.000 INFO:teuthology.orchestra.run.smithi143.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-27T09:59:33.050 INFO:teuthology.orchestra.run.smithi143.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-27T09:59:33.092 INFO:teuthology.orchestra.run.smithi143.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 900 kB/s | 37 kB 00:00 2024-05-27T09:59:33.134 INFO:teuthology.orchestra.run.smithi143.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 5.0 MB/s | 207 kB 00:00 2024-05-27T09:59:33.186 INFO:teuthology.orchestra.run.smithi143.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.1 MB/s | 260 kB 00:00 2024-05-27T09:59:33.268 INFO:teuthology.orchestra.run.smithi143.stdout:(156/159): librbd1-19.0.0-3868.ga6507388.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-05-27T09:59:33.335 INFO:teuthology.orchestra.run.smithi143.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-27T09:59:33.494 INFO:teuthology.orchestra.run.smithi143.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-27T09:59:33.569 INFO:teuthology.orchestra.run.smithi143.stdout:(159/159): librados2-19.0.0-3868.ga6507388.el9. 3.0 MB/s | 3.3 MB 00:01 2024-05-27T09:59:33.580 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:59:33.580 INFO:teuthology.orchestra.run.smithi143.stdout:Total 7.5 MB/s | 219 MB 00:29 2024-05-27T09:59:34.371 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-05-27T09:59:34.621 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-05-27T09:59:34.621 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-05-27T09:59:36.712 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-05-27T09:59:36.963 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-05-27T09:59:36.963 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-05-27T09:59:37.274 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-05-27T09:59:37.274 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-05-27T09:59:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-05-27T09:59:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T09:59:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T09:59:39.619 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-05-27T09:59:39.621 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-05-27T09:59:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T09:59:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T09:59:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T09:59:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T09:59:40.387 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T09:59:40.526 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:59:40.716 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:59:40.744 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:59:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:59:41.046 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T09:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T09:59:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T09:59:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:59:41.726 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:59:41.760 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:59:41.838 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:59:41.894 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:59:41.973 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:59:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:59:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:59:43.032 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:59:43.232 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:43.694 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:43.841 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-05-27T09:59:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T09:59:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T09:59:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T09:59:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T09:59:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T09:59:44.117 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T09:59:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T09:59:44.170 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T09:59:44.202 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T09:59:44.210 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T09:59:44.287 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T09:59:44.295 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T09:59:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T09:59:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T09:59:44.540 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T09:59:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T09:59:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T09:59:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T09:59:44.748 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T09:59:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T09:59:44.850 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:59:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 33/170 2024-05-27T09:59:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:59:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 35/170 2024-05-27T09:59:45.057 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T09:59:45.084 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:59:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T09:59:45.218 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:59:45.250 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:59:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T09:59:45.334 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T09:59:45.482 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T09:59:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T09:59:45.677 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T09:59:45.739 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T09:59:45.764 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:59:45.909 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:59:45.940 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:59:46.025 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:59:46.083 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:59:46.118 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-05-27T09:59:46.149 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:59:46.209 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T09:59:46.279 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T09:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T09:59:46.379 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T09:59:46.408 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T09:59:46.428 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T09:59:46.461 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T09:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T09:59:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T09:59:46.565 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T09:59:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T09:59:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 46/170 2024-05-27T09:59:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T09:59:46.709 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T09:59:46.964 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:59:46.974 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T09:59:47.008 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T09:59:47.052 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:59:47.085 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:59:47.150 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T09:59:47.210 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:47.240 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T09:59:47.278 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:59:47.311 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:59:47.326 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:47.338 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T09:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T09:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T09:59:47.442 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:59:47.486 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:59:47.490 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T09:59:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T09:59:47.579 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T09:59:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T09:59:47.655 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:47.659 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T09:59:47.857 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T09:59:47.867 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T09:59:47.889 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T09:59:47.917 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T09:59:47.927 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T09:59:47.942 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T09:59:47.974 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T09:59:47.998 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T09:59:48.000 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:59:48.057 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T09:59:48.101 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T09:59:48.144 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:59:48.145 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T09:59:48.164 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T09:59:48.168 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:59:48.192 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T09:59:48.215 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T09:59:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T09:59:48.254 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:59:48.254 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T09:59:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T09:59:48.286 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:59:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T09:59:48.345 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:59:48.370 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T09:59:48.442 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T09:59:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T09:59:48.527 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T09:59:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T09:59:48.634 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T09:59:48.634 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:59:48.667 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 33/170 2024-05-27T09:59:48.716 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:59:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T09:59:48.749 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 35/170 2024-05-27T09:59:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T09:59:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T09:59:48.862 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T09:59:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T09:59:48.920 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T09:59:48.924 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:59:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T09:59:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T09:59:49.028 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T09:59:49.043 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T09:59:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T09:59:49.102 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:59:49.132 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T09:59:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:59:49.204 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T09:59:49.207 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T09:59:49.253 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T09:59:49.264 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T09:59:49.359 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T09:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T09:59:49.505 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T09:59:49.594 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T09:59:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T09:59:49.821 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:59:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T09:59:49.949 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T09:59:50.015 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T09:59:50.052 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T09:59:50.053 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T09:59:50.099 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T09:59:50.106 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T09:59:50.145 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T09:59:50.202 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T09:59:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T09:59:50.247 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 46/170 2024-05-27T09:59:50.257 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T09:59:50.321 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T09:59:50.321 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T09:59:50.371 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T09:59:50.410 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T09:59:50.514 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T09:59:50.539 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T09:59:50.583 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T09:59:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T09:59:50.620 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T09:59:50.675 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T09:59:50.772 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T09:59:50.784 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T09:59:50.826 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 33/170 2024-05-27T09:59:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T09:59:50.847 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T09:59:50.873 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:59:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T09:59:50.908 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 35/170 2024-05-27T09:59:50.923 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T09:59:50.938 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T09:59:50.977 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T09:59:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T09:59:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T09:59:51.052 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T09:59:51.068 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T09:59:51.086 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T09:59:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T09:59:51.118 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T09:59:51.174 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T09:59:51.177 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T09:59:51.178 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T09:59:51.234 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T09:59:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T09:59:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T09:59:51.350 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T09:59:51.420 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T09:59:51.442 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T09:59:51.492 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T09:59:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T09:59:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T09:59:51.614 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T09:59:51.671 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T09:59:51.716 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T09:59:51.737 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T09:59:51.766 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T09:59:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T09:59:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T09:59:52.048 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T09:59:52.125 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T09:59:52.223 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T09:59:52.234 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T09:59:52.270 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T09:59:52.279 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T09:59:52.309 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T09:59:52.318 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T09:59:52.363 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T09:59:52.373 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T09:59:52.419 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 46/170 2024-05-27T09:59:52.449 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T09:59:52.494 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T09:59:52.503 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T09:59:52.551 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T09:59:52.597 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T09:59:52.632 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T09:59:52.639 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T09:59:52.705 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T09:59:52.715 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T09:59:52.753 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T09:59:52.790 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T09:59:52.793 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T09:59:52.893 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T09:59:52.906 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T09:59:52.949 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T09:59:52.982 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T09:59:53.001 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T09:59:53.062 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T09:59:53.070 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T09:59:53.111 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T09:59:53.148 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T09:59:53.210 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T09:59:53.214 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T09:59:53.257 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T09:59:53.281 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T09:59:53.317 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T09:59:53.377 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T09:59:53.574 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T09:59:53.626 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T09:59:53.678 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T09:59:53.805 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T09:59:53.850 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T09:59:53.913 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T09:59:53.949 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T09:59:53.982 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T09:59:54.164 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T09:59:54.218 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T09:59:54.281 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T09:59:54.314 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T09:59:54.343 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T09:59:54.387 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T09:59:54.430 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T09:59:54.436 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T09:59:54.471 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T09:59:54.497 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T09:59:54.530 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T09:59:54.550 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T09:59:54.551 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T09:59:54.608 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T09:59:54.650 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T09:59:54.650 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T09:59:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T09:59:54.694 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T09:59:54.708 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T09:59:54.715 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T09:59:54.745 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T09:59:54.750 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T09:59:54.819 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T09:59:54.867 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T09:59:54.951 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T09:59:54.997 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T09:59:55.010 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T09:59:55.072 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T09:59:55.095 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T09:59:55.116 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T09:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T09:59:55.185 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T09:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:59:55.333 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T09:59:55.367 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:59:55.396 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T09:59:55.406 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 104/170 2024-05-27T09:59:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T09:59:55.898 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T09:59:56.034 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T09:59:56.071 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T09:59:56.119 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:59:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:59:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 108/170 2024-05-27T09:59:56.357 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T09:59:56.412 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T09:59:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T09:59:56.586 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T09:59:56.728 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T09:59:57.106 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T09:59:57.244 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T09:59:57.299 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T09:59:57.562 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T09:59:57.595 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T09:59:57.630 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T09:59:57.726 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T09:59:57.736 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T09:59:57.771 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T09:59:57.806 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T09:59:58.182 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T09:59:58.223 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:59:58.344 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:59:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 104/170 2024-05-27T09:59:58.641 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T09:59:58.781 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T09:59:58.842 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T09:59:59.121 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:59:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:59:59.457 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T09:59:59.501 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:59:59.602 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:59:59.628 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 108/170 2024-05-27T09:59:59.846 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:59:59.887 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T10:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T10:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T10:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T10:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:00.129 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T10:00:00.402 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T10:00:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T10:00:00.533 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T10:00:00.569 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T10:00:00.939 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T10:00:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T10:00:01.082 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T10:00:01.118 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 104/170 2024-05-27T10:00:01.172 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T10:00:01.753 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T10:00:01.797 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T10:00:01.897 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T10:00:01.924 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 108/170 2024-05-27T10:00:02.264 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T10:00:02.338 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T10:00:03.002 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T10:00:03.044 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T10:00:03.243 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T10:00:03.244 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T10:00:03.244 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T10:00:03.244 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:03.287 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T10:00:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T10:00:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T10:00:05.231 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T10:00:05.273 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T10:00:05.439 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T10:00:05.439 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T10:00:05.439 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T10:00:05.439 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:05.474 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T10:00:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-05-27T10:00:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T10:00:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T10:00:13.856 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:13.856 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T10:00:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T10:00:15.773 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T10:00:15.773 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:15.773 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T10:00:15.773 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T10:00:15.773 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T10:00:15.773 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:16.048 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T10:00:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T10:00:16.139 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:16.139 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T10:00:16.139 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T10:00:16.139 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T10:00:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:16.449 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-05-27T10:00:16.450 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T10:00:16.677 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T10:00:16.729 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T10:00:16.729 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:16.729 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T10:00:16.729 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:16.773 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T10:00:16.773 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:16.773 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T10:00:16.774 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T10:00:16.774 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T10:00:16.774 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:16.910 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T10:00:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T10:00:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T10:00:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T10:00:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T10:00:17.518 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T10:00:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T10:00:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T10:00:17.897 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T10:00:17.936 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T10:00:17.969 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T10:00:18.471 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T10:00:18.553 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T10:00:18.553 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:18.553 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T10:00:18.553 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T10:00:18.553 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T10:00:18.554 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T10:00:18.722 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T10:00:18.722 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-05-27T10:00:18.722 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-05-27T10:00:18.723 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-05-27T10:00:18.723 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-05-27T10:00:18.723 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-05-27T10:00:18.723 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-05-27T10:00:18.723 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-05-27T10:00:18.723 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T10:00:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 128/170 2024-05-27T10:00:18.829 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T10:00:18.914 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T10:00:18.915 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:18.915 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T10:00:18.915 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T10:00:18.915 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T10:00:18.915 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 129/170 2024-05-27T10:00:18.946 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T10:00:18.990 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T10:00:18.991 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:18.991 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T10:00:18.991 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T10:00:19.430 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T10:00:19.512 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T10:00:19.512 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:19.512 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T10:00:19.512 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T10:00:19.512 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T10:00:19.512 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:19.647 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T10:00:19.708 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T10:00:19.801 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T10:00:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T10:00:20.109 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T10:00:20.196 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T10:00:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T10:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T10:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T10:00:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T10:00:20.564 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T10:00:20.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T10:00:20.637 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T10:00:20.715 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T10:00:20.814 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T10:00:20.814 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:20.814 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T10:00:20.814 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T10:00:20.814 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T10:00:20.815 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:21.065 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T10:00:21.129 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T10:00:21.129 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:21.129 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T10:00:21.129 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T10:00:21.129 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T10:00:21.130 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:21.277 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T10:00:21.324 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T10:00:21.379 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 128/170 2024-05-27T10:00:21.488 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 129/170 2024-05-27T10:00:21.517 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T10:00:21.608 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T10:00:21.632 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T10:00:21.635 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T10:00:21.724 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T10:00:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T10:00:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T10:00:21.725 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T10:00:21.725 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T10:00:21.883 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T10:00:21.948 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T10:00:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 133/170 2024-05-27T10:00:22.027 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T10:00:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T10:00:22.113 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T10:00:22.113 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:22.113 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T10:00:22.113 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T10:00:22.113 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T10:00:22.113 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:22.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T10:00:22.231 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T10:00:22.270 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T10:00:22.337 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T10:00:22.389 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 136/170 2024-05-27T10:00:22.427 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T10:00:22.469 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 137/170 2024-05-27T10:00:22.539 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T10:00:22.704 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T10:00:22.769 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T10:00:22.829 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T10:00:22.835 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T10:00:22.867 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T10:00:22.885 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T10:00:23.419 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T10:00:23.456 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T10:00:23.509 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 128/170 2024-05-27T10:00:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T10:00:23.619 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 129/170 2024-05-27T10:00:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T10:00:23.659 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T10:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T10:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T10:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T10:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T10:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T10:00:23.838 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T10:00:23.839 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:23.839 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T10:00:23.839 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T10:00:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T10:00:23.977 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:23.977 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T10:00:23.977 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T10:00:23.977 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T10:00:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:23.979 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T10:00:24.027 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T10:00:24.184 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T10:00:24.363 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 133/170 2024-05-27T10:00:24.404 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T10:00:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T10:00:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T10:00:24.484 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:24.484 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T10:00:24.485 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T10:00:24.486 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T10:00:24.486 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:24.546 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T10:00:24.584 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T10:00:24.726 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 136/170 2024-05-27T10:00:24.793 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 137/170 2024-05-27T10:00:24.974 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T10:00:25.027 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T10:00:25.828 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T10:00:25.857 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T10:00:25.929 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T10:00:25.929 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:25.929 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T10:00:25.930 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T10:00:25.930 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T10:00:25.930 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:25.983 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T10:00:26.067 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T10:00:26.068 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:26.068 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T10:00:26.068 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:26.120 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T10:00:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T10:00:26.152 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T10:00:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T10:00:26.221 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:26.221 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T10:00:26.221 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T10:00:26.221 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T10:00:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:26.309 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T10:00:26.489 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 133/170 2024-05-27T10:00:26.521 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T10:00:26.586 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T10:00:26.586 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:26.586 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T10:00:26.586 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T10:00:26.586 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T10:00:26.587 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:26.634 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T10:00:26.672 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T10:00:26.709 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T10:00:26.711 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T10:00:26.711 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:26.711 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T10:00:26.711 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T10:00:26.712 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T10:00:26.712 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:26.839 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 136/170 2024-05-27T10:00:26.917 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 137/170 2024-05-27T10:00:27.990 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T10:00:28.024 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T10:00:28.092 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T10:00:28.092 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:28.092 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T10:00:28.092 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T10:00:28.092 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T10:00:28.092 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:28.142 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T10:00:28.237 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T10:00:28.237 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:28.237 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T10:00:28.238 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:28.309 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T10:00:28.413 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T10:00:28.413 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:28.413 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T10:00:28.413 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T10:00:28.413 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T10:00:28.413 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:28.827 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T10:00:28.899 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T10:00:28.899 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T10:00:28.899 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T10:00:28.899 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T10:00:28.899 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T10:00:28.900 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:29.521 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 143/170 2024-05-27T10:00:29.588 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 144/170 2024-05-27T10:00:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 145/170 2024-05-27T10:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T10:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T10:00:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 148/170 2024-05-27T10:00:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T10:00:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 150/170 2024-05-27T10:00:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T10:00:30.286 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T10:00:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T10:00:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:31.389 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 143/170 2024-05-27T10:00:31.442 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 144/170 2024-05-27T10:00:31.446 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:31.522 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 145/170 2024-05-27T10:00:31.564 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T10:00:31.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T10:00:31.899 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 148/170 2024-05-27T10:00:31.950 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T10:00:31.974 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 150/170 2024-05-27T10:00:32.006 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T10:00:32.051 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T10:00:32.107 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T10:00:32.960 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 155/170 2024-05-27T10:00:33.012 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:33.079 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T10:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T10:00:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T10:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T10:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T10:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T10:00:33.557 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 143/170 2024-05-27T10:00:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T10:00:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T10:00:33.612 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 144/170 2024-05-27T10:00:33.665 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 145/170 2024-05-27T10:00:33.693 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T10:00:33.698 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T10:00:33.736 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T10:00:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T10:00:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T10:00:33.876 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T10:00:33.946 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T10:00:33.981 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:33.981 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:34.011 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 148/170 2024-05-27T10:00:34.071 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T10:00:34.102 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 150/170 2024-05-27T10:00:34.143 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T10:00:34.176 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T10:00:34.227 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T10:00:34.345 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:34.345 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T10:00:34.405 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 155/170 2024-05-27T10:00:34.490 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T10:00:34.501 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T10:00:34.534 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T10:00:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T10:00:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T10:00:34.685 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T10:00:34.728 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T10:00:34.761 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T10:00:34.795 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T10:00:34.835 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T10:00:34.836 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T10:00:34.942 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T10:00:34.943 INFO:teuthology.orchestra.run.smithi042.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T10:00:34.943 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:34.979 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T10:00:35.024 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T10:00:35.024 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T10:00:35.169 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T10:00:35.192 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:35.208 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T10:00:35.241 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:35.242 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:35.242 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:35.295 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T10:00:35.579 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:35.579 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T10:00:35.734 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T10:00:35.778 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T10:00:35.897 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T10:00:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T10:00:36.156 INFO:teuthology.orchestra.run.smithi104.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T10:00:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:36.612 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 155/170 2024-05-27T10:00:36.723 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T10:00:36.781 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T10:00:36.906 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T10:00:36.942 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T10:00:36.975 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T10:00:37.009 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T10:00:37.049 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T10:00:37.049 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T10:00:37.177 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T10:00:37.247 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T10:00:37.247 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T10:00:37.384 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T10:00:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T10:00:37.455 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:37.455 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:37.784 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T10:00:37.784 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T10:00:37.908 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T10:00:37.942 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T10:00:38.070 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T10:00:38.352 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T10:00:38.352 INFO:teuthology.orchestra.run.smithi143.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T10:00:38.352 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:41.062 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 1/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 3/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 4/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 5/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 6/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 7/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 11/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 12/170 2024-05-27T10:00:41.063 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 13/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 15/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 16/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 17/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 18/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 19/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 20/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 21/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 22/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 23/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 24/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 25/170 2024-05-27T10:00:41.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 26/170 2024-05-27T10:00:41.065 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 27/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 28/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 29/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 30/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 31/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 32/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 33/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 35/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 36/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 37/170 2024-05-27T10:00:41.078 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 38/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 39/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T10:00:41.079 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T10:00:41.080 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T10:00:41.081 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T10:00:41.082 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T10:00:41.083 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T10:00:41.084 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T10:00:41.085 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T10:00:41.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 149/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 151/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T10:00:41.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T10:00:41.088 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T10:00:41.881 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 1/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 3/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 4/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 5/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 6/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 7/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 11/170 2024-05-27T10:00:41.882 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 12/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 13/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 15/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 16/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 17/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 18/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 19/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 20/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 21/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 22/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 23/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 24/170 2024-05-27T10:00:41.883 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 25/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 26/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 27/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 28/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 29/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 30/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 31/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 32/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 33/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 35/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 36/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 37/170 2024-05-27T10:00:41.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 38/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 39/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T10:00:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T10:00:41.886 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T10:00:41.887 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T10:00:41.888 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T10:00:41.889 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T10:00:41.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T10:00:41.891 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T10:00:41.892 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 149/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 151/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T10:00:41.893 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T10:00:41.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T10:00:41.895 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T10:00:41.895 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T10:00:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T10:00:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T10:00:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T10:00:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T10:00:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T10:00:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T10:00:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T10:00:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T10:00:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T10:00:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:00:43.563 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T10:00:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T10:00:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T10:00:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T10:00:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T10:00:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T10:00:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T10:00:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T10:00:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T10:00:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T10:00:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T10:00:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T10:00:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-27T10:00:44.166 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-05-27T10:00:44.374 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 1/170 2024-05-27T10:00:44.374 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2/170 2024-05-27T10:00:44.374 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 3/170 2024-05-27T10:00:44.374 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 4/170 2024-05-27T10:00:44.374 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 5/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 6/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 7/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 11/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 12/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 13/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T10:00:44.375 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 15/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 16/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 17/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 18/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 19/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 20/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 21/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 22/170 2024-05-27T10:00:44.376 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 23/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 24/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 25/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 26/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 27/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 28/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 29/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 30/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 31/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 32/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 33/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 35/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 36/170 2024-05-27T10:00:44.377 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 37/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 38/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 39/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T10:00:44.378 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T10:00:44.379 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T10:00:44.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T10:00:44.381 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T10:00:44.382 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T10:00:44.383 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T10:00:44.384 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T10:00:44.385 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 149/170 2024-05-27T10:00:44.386 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 151/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T10:00:44.387 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T10:00:44.388 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T10:00:44.388 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T10:00:44.388 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T10:00:44.388 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T10:00:44.388 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T10:00:44.471 DEBUG:teuthology.parallel:result is None 2024-05-27T10:00:44.808 DEBUG:teuthology.parallel:result is None 2024-05-27T10:00:46.688 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T10:00:46.688 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:46.688 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-05-27T10:00:46.688 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-05-27T10:00:46.689 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.690 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.691 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T10:00:46.692 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T10:00:46.693 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T10:00:46.694 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T10:00:46.695 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T10:00:46.696 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T10:00:46.697 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T10:00:46.698 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T10:00:46.699 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T10:00:46.700 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T10:00:46.701 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:00:46.702 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-05-27T10:00:47.354 DEBUG:teuthology.parallel:result is None 2024-05-27T10:00:47.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:00:47.581 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T10:00:47.639 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-3868.ga6507388.el9 2024-05-27T10:00:47.639 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga6507388.el9 2024-05-27T10:00:47.640 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga6507388 is installed. 2024-05-27T10:00:47.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:00:47.780 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T10:00:47.838 INFO:teuthology.orchestra.run.smithi104.stdout:19.0.0-3868.ga6507388.el9 2024-05-27T10:00:47.838 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga6507388.el9 2024-05-27T10:00:47.838 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga6507388 is installed. 2024-05-27T10:00:47.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:00:48.065 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T10:00:48.119 INFO:teuthology.orchestra.run.smithi143.stdout:19.0.0-3868.ga6507388.el9 2024-05-27T10:00:48.120 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga6507388.el9 2024-05-27T10:00:48.120 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga6507388 is installed. 2024-05-27T10:00:48.122 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T10:00:48.122 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:00:48.122 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T10:00:48.202 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:00:48.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T10:00:48.288 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:00:48.288 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T10:00:48.356 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T10:00:48.356 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:00:48.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T10:00:48.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T10:00:48.543 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:00:48.543 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T10:00:48.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T10:00:48.762 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:00:48.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T10:00:48.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T10:00:48.947 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T10:00:48.948 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:00:48.948 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T10:00:49.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T10:00:49.168 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:00:49.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T10:00:49.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T10:00:49.386 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:00:49.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T10:00:49.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T10:00:49.550 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T10:00:49.551 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:00:49.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T10:00:49.646 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T10:00:49.738 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:00:49.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T10:00:49.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T10:00:49.900 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:00:49.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T10:00:49.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T10:00:50.069 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-27T10:00:50.175 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'subvols': {'create': 2}} 2024-05-27T10:00:50.175 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:00:50.176 INFO:tasks.cephadm:Cluster fsid is ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:00:50.176 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-27T10:00:50.176 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.42', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.143'} 2024-05-27T10:00:50.176 INFO:tasks.cephadm:First mon is mon.a on smithi042 2024-05-27T10:00:50.177 INFO:tasks.cephadm:First mgr is x 2024-05-27T10:00:50.177 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-27T10:00:50.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-05-27T10:00:50.273 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-05-27T10:00:50.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname $(hostname -s) 2024-05-27T10:00:50.441 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-27T10:00:50.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:00:50.659 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79602/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3868-ga6507388', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3868.ga6507388'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 12:50:55.456707', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.105740-debug', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79604/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3868-ga6507388', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3868.ga6507388'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 12:34:17.608165', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.105740-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-27T10:00:50.795 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240521.105740-debug, sha1 a6507388dd5057528934822c0163b0c347ef1d5d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:00:50.796 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/x86_64/flavors/default/cephadm 2024-05-27T10:00:50.796 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/x86_64/flavors/default/cephadm 2024-05-27T10:00:50.797 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:51.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r--. 1 ubuntu ubuntu 782785 May 27 10:00 /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:51.329 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:51.841 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 782785 May 27 10:00 /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:51.842 DEBUG:teuthology.orchestra.run.smithi143:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:52.529 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r--. 1 ubuntu ubuntu 782785 May 27 10:00 /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:52.529 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:52.594 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:52.654 DEBUG:teuthology.orchestra.run.smithi143:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T10:00:52.749 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d on all hosts... 2024-05-27T10:00:52.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d pull 2024-05-27T10:00:52.756 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d pull 2024-05-27T10:00:52.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d pull 2024-05-27T10:00:53.946 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d... 2024-05-27T10:00:53.969 INFO:teuthology.orchestra.run.smithi143.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d... 2024-05-27T10:00:54.189 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d... 2024-05-27T10:01:47.896 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:01:47.897 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)", 2024-05-27T10:01:47.897 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76", 2024-05-27T10:01:47.897 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2024-05-27T10:01:47.897 INFO:teuthology.orchestra.run.smithi042.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6" 2024-05-27T10:01:47.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:01:47.897 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)", 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: "image_id": "850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76", 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: "repo_digests": [ 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6" 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-05-27T10:01:59.689 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-27T10:02:06.970 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)", 2024-05-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76", 2024-05-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-05-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6" 2024-05-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-05-27T10:02:06.971 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-27T10:02:07.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-05-27T10:02:07.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-05-27T10:02:07.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph 2024-05-27T10:02:07.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-05-27T10:02:07.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-05-27T10:02:07.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /etc/ceph 2024-05-27T10:02:07.584 INFO:tasks.cephadm:Writing seed config... 2024-05-27T10:02:07.585 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-27T10:02:07.585 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-27T10:02:07.585 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-27T10:02:07.585 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-27T10:02:07.585 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-27T10:02:07.585 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-27T10:02:07.586 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-27T10:02:07.587 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-27T10:02:07.588 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-27T10:02:07.589 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-27T10:02:07.589 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:02:07.589 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-27T10:02:07.652 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ff1a20ea-1c0f-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-27T10:02:07.652 DEBUG:teuthology.orchestra.run.smithi042:mon.a> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service 2024-05-27T10:02:07.700 DEBUG:teuthology.orchestra.run.smithi042:mgr.x> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x.service 2024-05-27T10:02:07.747 INFO:tasks.cephadm:Bootstrapping... 2024-05-27T10:02:07.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d -v bootstrap --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:02:08.028 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-05-27T10:02:08.028 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d', '-v', 'bootstrap', '--fsid', 'ff1a20ea-1c0f-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2024-05-27T10:02:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-27T10:02:08.029 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2024-05-27T10:02:08.094 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.0.2 2024-05-27T10:02:08.095 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2024-05-27T10:02:08.095 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2024-05-27T10:02:08.114 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T10:02:08.114 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T10:02:08.132 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T10:02:08.132 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-05-27T10:02:08.151 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-05-27T10:02:08.168 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-05-27T10:02:08.169 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-05-27T10:02:08.169 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2024-05-27T10:02:08.223 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.0.2 2024-05-27T10:02:08.224 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-27T10:02:08.224 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2024-05-27T10:02:08.224 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2024-05-27T10:02:08.240 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T10:02:08.241 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T10:02:08.262 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T10:02:08.262 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-05-27T10:02:08.280 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-05-27T10:02:08.299 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-05-27T10:02:08.300 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-05-27T10:02:08.300 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2024-05-27T10:02:08.300 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:08.300 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 139922348326432 on /run/cephadm/ff1a20ea-1c0f-11ef-bc9b-c7b262605968.lock 2024-05-27T10:02:08.300 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 139922348326432 acquired on /run/cephadm/ff1a20ea-1c0f-11ef-bc9b-c7b262605968.lock 2024-05-27T10:02:08.301 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2024-05-27T10:02:08.302 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2024-05-27T10:02:08.303 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-05-27T10:02:08.315 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-05-27T10:02:08.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-05-27T10:02:08.326 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-27T10:02:08.326 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-27T10:02:08.327 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-27T10:02:08.336 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-27T10:02:08.336 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-27T10:02:08.337 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T10:02:08.337 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-27T10:02:08.337 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-05-27T10:02:08.337 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T10:02:08.340 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-05-27T10:02:08.340 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-05-27T10:02:08.340 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-27T10:02:08.342 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-27T10:02:08.344 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d... 2024-05-27T10:02:15.744 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 2024-05-27T10:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d... 2024-05-27T10:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2024-05-27T10:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:30c527fbaaeaa4fed4aa057eca2023a2923eff3c212014d5d990bfbe5e3b29dc 2024-05-27T10:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-27T10:02:15.745 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 2024-05-27T10:02:15.746 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-27T10:02:17.549 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev) 2024-05-27T10:02:17.550 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev) 2024-05-27T10:02:17.550 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2024-05-27T10:02:18.804 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2024-05-27T10:02:18.804 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2024-05-27T10:02:19.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQArWlRmydtKFhAAnVxJWRsWnMhCEEJP1osc4Q== 2024-05-27T10:02:21.217 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQAsWlRmKsC5HxAAOxFaYVaSrUQtdpA/dODtvw== 2024-05-27T10:02:22.598 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQAtWlRmicjnNhAAoCcWr6ORA4kaR9I5UYGa8A== 2024-05-27T10:02:22.600 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2024-05-27T10:02:23.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T10:02:23.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for a [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:02:23.965 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2024-05-27T10:02:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.a on 2024-05-27T10:02:25.727 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-27T10:02:25.979 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-27T10:02:26.281 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968.target. 2024-05-27T10:02:26.281 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968.target. 2024-05-27T10:02:26.688 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a 2024-05-27T10:02:26.688 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-05-27T10:02:27.001 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968.target.wants/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@.service. 2024-05-27T10:02:27.861 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T10:02:27.862 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-05-27T10:02:27.880 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T10:02:27.880 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-05-27T10:02:27.880 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-05-27T10:02:27.880 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T10:02:27.881 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2024-05-27T10:02:27.881 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2024-05-27T10:02:28.025 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:27 smithi042 ceph-mon[31298]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: monmap epoch 1 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: last_changed 2024-05-27T10:02:23.315287+0000 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: min_mon_release 19 (squid) 2024-05-27T10:02:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: election_strategy: 1 2024-05-27T10:02:29.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:02:29.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: fsmap 2024-05-27T10:02:29.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T10:02:29.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: mgrmap e1: no daemons active 2024-05-27T10:02:29.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:28 smithi042 ceph-mon[31298]: from='client.? 172.21.15.42:0/386994379' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-27T10:02:29.452 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.755935s) 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-27T10:02:29.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.551+0000 7fc832800700 1 Processor -- start 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.551+0000 7fc832800700 1 -- start start 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.552+0000 7fc832800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c10bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.552+0000 7fc832800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c10c540 con 0x7fc82c10bb40 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.553+0000 7fc831400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c10bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:29.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.553+0000 7fc831400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c10bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58670/0 (socket says 172.21.15.42:58670) 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.553+0000 7fc831400700 1 -- 172.21.15.42:0/238417689 learned_addr learned my addr 172.21.15.42:0/238417689 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.554+0000 7fc831400700 1 -- 172.21.15.42:0/238417689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c10cd10 con 0x7fc82c10bb40 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.554+0000 7fc831400700 1 --2- 172.21.15.42:0/238417689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c10bf50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc8200098a0 tx=0x7fc820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb9f8924df85422 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.554+0000 7fc82be00700 1 -- 172.21.15.42:0/238417689 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc820004160 con 0x7fc82c10bb40 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.555+0000 7fc82be00700 1 -- 172.21.15.42:0/238417689 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820004300 con 0x7fc82c10bb40 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.555+0000 7fc82be00700 1 -- 172.21.15.42:0/238417689 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8200036a0 con 0x7fc82c10bb40 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.555+0000 7fc832800700 1 -- 172.21.15.42:0/238417689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 msgr2=0x7fc82c10bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.555+0000 7fc832800700 1 --2- 172.21.15.42:0/238417689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c10bf50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc8200098a0 tx=0x7fc820009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.556+0000 7fc832800700 1 -- 172.21.15.42:0/238417689 shutdown_connections 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.556+0000 7fc832800700 1 --2- 172.21.15.42:0/238417689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c10bf50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.556+0000 7fc832800700 1 -- 172.21.15.42:0/238417689 >> 172.21.15.42:0/238417689 conn(0x7fc82c07f030 msgr2=0x7fc82c07f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:29.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.556+0000 7fc832800700 1 -- 172.21.15.42:0/238417689 shutdown_connections 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.556+0000 7fc832800700 1 -- 172.21.15.42:0/238417689 wait complete. 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.557+0000 7fc832800700 1 Processor -- start 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.557+0000 7fc832800700 1 -- start start 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.558+0000 7fc832800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.558+0000 7fc832800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c10d240 con 0x7fc82c10bb40 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.558+0000 7fc831400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.558+0000 7fc831400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58672/0 (socket says 172.21.15.42:58672) 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.558+0000 7fc831400700 1 -- 172.21.15.42:0/386994379 learned_addr learned my addr 172.21.15.42:0/386994379 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.559+0000 7fc831400700 1 -- 172.21.15.42:0/386994379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c19fae0 con 0x7fc82c10bb40 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.559+0000 7fc831400700 1 --2- 172.21.15.42:0/386994379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c19f5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc820000c00 tx=0x7fc820004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.559+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc820003db0 con 0x7fc82c10bb40 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.559+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820011630 con 0x7fc82c10bb40 2024-05-27T10:02:29.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.559+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc820011950 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.559+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c19fd70 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.560+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c1a2980 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.560+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc820011af0 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.560+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc82001c450 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.561+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c064e10 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.564+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc820032020 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.646+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc82c02e8e0 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.646+0000 7fc82a000700 1 -- 172.21.15.42:0/386994379 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc820004260 con 0x7fc82c10bb40 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.647+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 msgr2=0x7fc82c19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.647+0000 7fc832800700 1 --2- 172.21.15.42:0/386994379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c19f5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc820000c00 tx=0x7fc820004750 comp rx=0 tx=0).stop 2024-05-27T10:02:29.462 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.647+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 shutdown_connections 2024-05-27T10:02:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.647+0000 7fc832800700 1 --2- 172.21.15.42:0/386994379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc82c10bb40 0x7fc82c19f5a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.647+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 >> 172.21.15.42:0/386994379 conn(0x7fc82c07f030 msgr2=0x7fc82c193540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.648+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 shutdown_connections 2024-05-27T10:02:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:28.648+0000 7fc832800700 1 -- 172.21.15.42:0/386994379 wait complete. 2024-05-27T10:02:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2024-05-27T10:02:29.463 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2024-05-27T10:02:30.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:30 smithi042 ceph-mon[31298]: from='client.? 172.21.15.42:0/45725487' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T10:02:30.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:30 smithi042 ceph-mon[31298]: from='client.? 172.21.15.42:0/45725487' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-27T10:02:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T10:02:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T10:02:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-05-27T10:02:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T10:02:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.142+0000 7f843e400700 1 Processor -- start 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.143+0000 7f843e400700 1 -- start start 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.143+0000 7f843e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843810bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.143+0000 7f843e400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f843810c4c0 con 0x7f843810bac0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.144+0000 7f843d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843810bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.144+0000 7f843d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843810bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58674/0 (socket says 172.21.15.42:58674) 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.144+0000 7f843d000700 1 -- 172.21.15.42:0/4117883024 learned_addr learned my addr 172.21.15.42:0/4117883024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.144+0000 7f843d000700 1 -- 172.21.15.42:0/4117883024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843810cc90 con 0x7f843810bac0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.144+0000 7f843d000700 1 --2- 172.21.15.42:0/4117883024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843810bed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098a0 tx=0x7f842c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7a85dd41eee1b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.145+0000 7f8437400700 1 -- 172.21.15.42:0/4117883024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c004160 con 0x7f843810bac0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.145+0000 7f8437400700 1 -- 172.21.15.42:0/4117883024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c004300 con 0x7f843810bac0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.145+0000 7f8437400700 1 -- 172.21.15.42:0/4117883024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c0036a0 con 0x7f843810bac0 2024-05-27T10:02:30.937 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.145+0000 7f843e400700 1 -- 172.21.15.42:0/4117883024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 msgr2=0x7f843810bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.146+0000 7f843e400700 1 --2- 172.21.15.42:0/4117883024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843810bed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098a0 tx=0x7f842c009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.146+0000 7f843e400700 1 -- 172.21.15.42:0/4117883024 shutdown_connections 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.146+0000 7f843e400700 1 --2- 172.21.15.42:0/4117883024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843810bed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.146+0000 7f843e400700 1 -- 172.21.15.42:0/4117883024 >> 172.21.15.42:0/4117883024 conn(0x7f843807f0b0 msgr2=0x7f8438109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.146+0000 7f843e400700 1 -- 172.21.15.42:0/4117883024 shutdown_connections 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.146+0000 7f843e400700 1 -- 172.21.15.42:0/4117883024 wait complete. 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.147+0000 7f843e400700 1 Processor -- start 2024-05-27T10:02:30.938 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.147+0000 7f843e400700 1 -- start start 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.148+0000 7f843e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.148+0000 7f843e400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f843810d1c0 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.148+0000 7f843d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.148+0000 7f843d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58682/0 (socket says 172.21.15.42:58682) 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.148+0000 7f843d000700 1 -- 172.21.15.42:0/45725487 learned_addr learned my addr 172.21.15.42:0/45725487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.148+0000 7f843d000700 1 -- 172.21.15.42:0/45725487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819fb10 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f843d000700 1 --2- 172.21.15.42:0/45725487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843819f5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c003db0 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c011630 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c011950 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819fda0 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a29b0 con 0x7f843810bac0 2024-05-27T10:02:30.939 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f842c01a460 con 0x7f843810bac0 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.149+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f842c01b780 con 0x7f843810bac0 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.150+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438064e10 con 0x7f843810bac0 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.153+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f842c031020 con 0x7f843810bac0 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.231+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f843802e860 con 0x7f843810bac0 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.244+0000 7f8435600700 1 -- 172.21.15.42:0/45725487 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f842c01b9d0 con 0x7f843810bac0 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 msgr2=0x7f843819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 --2- 172.21.15.42:0/45725487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843819f5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c004750 comp rx=0 tx=0).stop 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 shutdown_connections 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 --2- 172.21.15.42:0/45725487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f843810bac0 0x7f843819f5d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 >> 172.21.15.42:0/45725487 conn(0x7f843807f0b0 msgr2=0x7f8438195cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:30.940 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 shutdown_connections 2024-05-27T10:02:30.941 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:30.246+0000 7f843e400700 1 -- 172.21.15.42:0/45725487 wait complete. 2024-05-27T10:02:30.941 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2024-05-27T10:02:32.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:31 smithi042 ceph-mon[31298]: from='client.? 172.21.15.42:0/2017852216' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:02:32.426 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.658+0000 7f3551000700 1 Processor -- start 2024-05-27T10:02:32.427 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.659+0000 7f3551000700 1 -- start start 2024-05-27T10:02:32.427 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.659+0000 7f3551000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:32.427 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.659+0000 7f3551000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f354c10a4e0 con 0x7f354c109ae0 2024-05-27T10:02:32.428 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.660+0000 7f3549e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:32.428 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.660+0000 7f3549e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58684/0 (socket says 172.21.15.42:58684) 2024-05-27T10:02:32.428 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.660+0000 7f3549e00700 1 -- 172.21.15.42:0/3380170765 learned_addr learned my addr 172.21.15.42:0/3380170765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:32.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.660+0000 7f3549e00700 1 -- 172.21.15.42:0/3380170765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c10acb0 con 0x7f354c109ae0 2024-05-27T10:02:32.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.661+0000 7f3549e00700 1 --2- 172.21.15.42:0/3380170765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c109ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f353c0098a0 tx=0x7f353c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69879224e137a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:32.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.661+0000 7f3548a00700 1 -- 172.21.15.42:0/3380170765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c004160 con 0x7f354c109ae0 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.661+0000 7f3548a00700 1 -- 172.21.15.42:0/3380170765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f353c004300 con 0x7f354c109ae0 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.661+0000 7f3548a00700 1 -- 172.21.15.42:0/3380170765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c0119e0 con 0x7f354c109ae0 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 -- 172.21.15.42:0/3380170765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 msgr2=0x7f354c109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 --2- 172.21.15.42:0/3380170765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c109ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f353c0098a0 tx=0x7f353c009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 -- 172.21.15.42:0/3380170765 shutdown_connections 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 --2- 172.21.15.42:0/3380170765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c109ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 -- 172.21.15.42:0/3380170765 >> 172.21.15.42:0/3380170765 conn(0x7f354c1051a0 msgr2=0x7f354c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 -- 172.21.15.42:0/3380170765 shutdown_connections 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.662+0000 7f3551000700 1 -- 172.21.15.42:0/3380170765 wait complete. 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3551000700 1 Processor -- start 2024-05-27T10:02:32.432 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3551000700 1 -- start start 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3551000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c07f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3551000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f354c10dee0 con 0x7f354c109ae0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3549e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c07f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3549e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c07f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58688/0 (socket says 172.21.15.42:58688) 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.664+0000 7f3549e00700 1 -- 172.21.15.42:0/2017852216 learned_addr learned my addr 172.21.15.42:0/2017852216 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.665+0000 7f3549e00700 1 -- 172.21.15.42:0/2017852216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c07f830 con 0x7f354c109ae0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.665+0000 7f3549e00700 1 --2- 172.21.15.42:0/2017852216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c07f2f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f353c009fb0 tx=0x7f353c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.665+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c0038b0 con 0x7f354c109ae0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.665+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f353c003a50 con 0x7f354c109ae0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.666+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c07da80 con 0x7f354c109ae0 2024-05-27T10:02:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.666+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c01a5c0 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.666+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c07df60 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.666+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f353c01ac00 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.666+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f353c019700 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.667+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c064e10 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.670+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f353c024a60 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.748+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f354c07e650 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.749+0000 7f3542a00700 1 -- 172.21.15.42:0/2017852216 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f353c024440 con 0x7f354c109ae0 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 msgr2=0x7f354c07f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:32.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 --2- 172.21.15.42:0/2017852216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c07f2f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f353c009fb0 tx=0x7f353c011a60 comp rx=0 tx=0).stop 2024-05-27T10:02:32.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 shutdown_connections 2024-05-27T10:02:32.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 --2- 172.21.15.42:0/2017852216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354c109ae0 0x7f354c07f2f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:32.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 >> 172.21.15.42:0/2017852216 conn(0x7f354c1051a0 msgr2=0x7f354c106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:32.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 shutdown_connections 2024-05-27T10:02:32.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:31.751+0000 7f3551000700 1 -- 172.21.15.42:0/2017852216 wait complete. 2024-05-27T10:02:32.436 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2024-05-27T10:02:32.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:32 smithi042 systemd[1]: Stopping Ceph mon.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:02:33.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31294]: 2024-05-27T10:02:32.962+0000 7fb0ebe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:02:33.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31294]: 2024-05-27T10:02:32.962+0000 7fb0ebe00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T10:02:33.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:33 smithi042 podman[31540]: 2024-05-27 10:02:33.055270974 +0000 UTC m=+0.221210720 container died 12c8a8cf1afb294afdd1688da0d9829404b664cea18f1bf66864be227b400149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:02:33.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:33 smithi042 podman[31540]: 2024-05-27 10:02:33.529362121 +0000 UTC m=+0.695301864 container cleanup 12c8a8cf1afb294afdd1688da0d9829404b664cea18f1bf66864be227b400149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:02:33.739 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:33 smithi042 bash[31540]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a 2024-05-27T10:02:34.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:33 smithi042 podman[31553]: 2024-05-27 10:02:33.738348854 +0000 UTC m=+0.661158427 container remove 12c8a8cf1afb294afdd1688da0d9829404b664cea18f1bf66864be227b400149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T10:02:34.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:34 smithi042 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-27T10:02:34.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:34 smithi042 systemd[1]: Stopped Ceph mon.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:02:34.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:34 smithi042 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service: Consumed 2.150s CPU time. 2024-05-27T10:02:34.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:34 smithi042 systemd[1]: Starting Ceph mon.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:02:35.060 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:34 smithi042 podman[31643]: 2024-05-27 10:02:34.55730491 +0000 UTC m=+0.035894740 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:02:35.060 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:34 smithi042 podman[31643]: 2024-05-27 10:02:34.697813188 +0000 UTC m=+0.176403051 container create c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T10:02:35.060 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 podman[31643]: 2024-05-27 10:02:35.059697499 +0000 UTC m=+0.538287405 container init c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T10:02:35.170 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-27T10:02:35.330 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 podman[31643]: 2024-05-27 10:02:35.071105704 +0000 UTC m=+0.549695570 container start c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 bash[31643]: c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 systemd[1]: Started Ceph mon.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev), process ceph-mon, pid 2 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: pidfile_write: ignore empty --pid-file 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: load: jerasure load: lrc 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: RocksDB version: 7.9.2 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Git sha 0 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Compile date 2024-05-21 11:39:56 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: DB SUMMARY 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: DB Session ID: OH8K7884BRAPSTJ91MLJ 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: CURRENT file: CURRENT 2024-05-27T10:02:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105118 ; 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.error_if_exists: 0 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.create_if_missing: 0 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.paranoid_checks: 1 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.env: 0x55561e6d6500 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.info_log: 0x55564ba56040 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T10:02:35.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.statistics: (nil) 2024-05-27T10:02:35.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.use_fsync: 0 2024-05-27T10:02:35.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_log_file_size: 0 2024-05-27T10:02:35.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T10:02:35.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T10:02:35.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T10:02:35.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.allow_fallocate: 1 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.use_direct_reads: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.db_log_dir: 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.wal_dir: 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T10:02:35.334 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.write_buffer_manager: 0x55564b9208c0 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T10:02:35.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.unordered_write: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.row_cache: None 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.wal_filter: None 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.two_write_queues: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.wal_compression: 0 2024-05-27T10:02:35.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.atomic_flush: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.log_readahead_size: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_background_jobs: 2 2024-05-27T10:02:35.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_background_compactions: -1 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_subcompactions: 1 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_open_files: -1 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T10:02:35.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_background_flushes: -1 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Compression algorithms supported: 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kZSTD supported: 0 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kXpressCompression supported: 0 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kZlibCompression supported: 1 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kSnappyCompression supported: 1 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kLZ4Compression supported: 1 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: kBZip2Compression supported: 0 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-27T10:02:35.339 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.merge_operator: 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_filter: None 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55564ba56200) 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_type: 0 2024-05-27T10:02:35.340 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_index_type: 0 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_shortening: 1 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: checksum: 4 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: no_block_cache: 0 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache: 0x55564b9531f0 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_options: 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: capacity : 536870912 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_shard_bits : 4 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: strict_capacity_limit : 0 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_compressed: (nil) 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: persistent_cache: (nil) 2024-05-27T10:02:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size: 4096 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size_deviation: 10 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_restart_interval: 16 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_block_restart_interval: 1 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: metadata_block_size: 4096 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: partition_filters: 0 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: use_delta_encoding: 1 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: filter_policy: bloomfilter 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: whole_key_filtering: 1 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: verify_compression: 0 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: format_version: 5 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_index_compression: 1 2024-05-27T10:02:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_align: 0 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout: max_auto_readahead_size: 262144 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout: prepopulate_block_cache: 0 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout: initial_auto_readahead_size: 8192 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression: NoCompression 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T10:02:35.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.num_levels: 7 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T10:02:35.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T10:02:35.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T10:02:35.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T10:02:35.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T10:02:35.348 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.inplace_update_support: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.bloom_locality: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.max_successive_merges: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T10:02:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.ttl: 2592000 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.enable_blob_files: false 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.min_blob_size: 0 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T10:02:35.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: af74e5a4-5d10-4cd4-b15e-597db90c2c59 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804155209835, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804155212192, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100295, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98180, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14167, "raw_average_key_size": 49, "raw_value_size": 90456, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716804155, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "af74e5a4-5d10-4cd4-b15e-597db90c2c59", "db_session_id": "OH8K7884BRAPSTJ91MLJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804155212427, "job": 1, "event": "recovery_finished"} 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T10:02:35.351 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55564b937c00 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: DB pointer 0x55565072a000 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** DB Stats ** 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-05-27T10:02:35.352 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: L0 2/0 99.80 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: Sum 2/0 99.80 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 57.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T10:02:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative compaction: 0.00 GB write, 12.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval compaction: 0.00 GB write, 12.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T10:02:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache BinnedLRUCache@0x55564b9531f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: starting mon.a rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???) e1 preinit fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).mds e1 new map 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).mds e1 print_map 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: e1 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T10:02:35.355 INFO:journalctl@ceph.mon.a.smithi042.stdout: legacy client fscid: -1 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout: No filesystems configured 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: monmap epoch 1 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: last_changed 2024-05-27T10:02:23.315287+0000 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: min_mon_release 19 (squid) 2024-05-27T10:02:35.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: election_strategy: 1 2024-05-27T10:02:35.357 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:02:35.357 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: fsmap 2024-05-27T10:02:35.357 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T10:02:35.357 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:35 smithi042 ceph-mon[31661]: mgrmap e1: no daemons active 2024-05-27T10:02:36.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:35.999+0000 7fdbbb600700 1 Processor -- start 2024-05-27T10:02:36.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:35.999+0000 7fdbbb600700 1 -- start start 2024-05-27T10:02:36.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.000+0000 7fdbbb600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb407db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:36.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.000+0000 7fdbbb600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb407e0f0 con 0x7fdbb407f6f0 2024-05-27T10:02:36.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.001+0000 7fdbb8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb407db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:36.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.001+0000 7fdbb8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb407db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38170/0 (socket says 172.21.15.42:38170) 2024-05-27T10:02:36.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.001+0000 7fdbb8c00700 1 -- 172.21.15.42:0/3599909691 learned_addr learned my addr 172.21.15.42:0/3599909691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:36.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.002+0000 7fdbb8c00700 1 -- 172.21.15.42:0/3599909691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb407e270 con 0x7fdbb407f6f0 2024-05-27T10:02:36.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.002+0000 7fdbb8c00700 1 --2- 172.21.15.42:0/3599909691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb407db20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76224645b010de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:36.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.003+0000 7fdbb3400700 1 -- 172.21.15.42:0/3599909691 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8004160 con 0x7fdbb407f6f0 2024-05-27T10:02:36.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.003+0000 7fdbb3400700 1 -- 172.21.15.42:0/3599909691 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdba8004300 con 0x7fdbb407f6f0 2024-05-27T10:02:36.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.003+0000 7fdbb3400700 1 -- 172.21.15.42:0/3599909691 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba80119e0 con 0x7fdbb407f6f0 2024-05-27T10:02:36.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 -- 172.21.15.42:0/3599909691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 msgr2=0x7fdbb407db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:36.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 --2- 172.21.15.42:0/3599909691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb407db20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:36.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 -- 172.21.15.42:0/3599909691 shutdown_connections 2024-05-27T10:02:36.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 --2- 172.21.15.42:0/3599909691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb407db20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:36.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 -- 172.21.15.42:0/3599909691 >> 172.21.15.42:0/3599909691 conn(0x7fdbb4105150 msgr2=0x7fdbb41075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:36.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 -- 172.21.15.42:0/3599909691 shutdown_connections 2024-05-27T10:02:36.804 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.004+0000 7fdbbb600700 1 -- 172.21.15.42:0/3599909691 wait complete. 2024-05-27T10:02:36.804 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.005+0000 7fdbbb600700 1 Processor -- start 2024-05-27T10:02:36.804 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.006+0000 7fdbbb600700 1 -- start start 2024-05-27T10:02:36.805 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.006+0000 7fdbbb600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:36.805 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.006+0000 7fdbbb600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4110080 con 0x7fdbb407f6f0 2024-05-27T10:02:36.805 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.006+0000 7fdbb8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:36.805 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.006+0000 7fdbb8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb419ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38184/0 (socket says 172.21.15.42:38184) 2024-05-27T10:02:36.805 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.006+0000 7fdbb8c00700 1 -- 172.21.15.42:0/3499150583 learned_addr learned my addr 172.21.15.42:0/3499150583 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:36.805 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.007+0000 7fdbb8c00700 1 -- 172.21.15.42:0/3499150583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb41a04d0 con 0x7fdbb407f6f0 2024-05-27T10:02:36.806 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.007+0000 7fdbb8c00700 1 --2- 172.21.15.42:0/3499150583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb419ff90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdba80116a0 tx=0x7fdba8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:36.806 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba80039f0 con 0x7fdbb407f6f0 2024-05-27T10:02:36.806 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdba8003b90 con 0x7fdbb407f6f0 2024-05-27T10:02:36.806 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb41a0760 con 0x7fdbb407f6f0 2024-05-27T10:02:36.807 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba801a600 con 0x7fdbb407f6f0 2024-05-27T10:02:36.807 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb41a3370 con 0x7fdbb407f6f0 2024-05-27T10:02:36.807 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdba801acb0 con 0x7fdbb407f6f0 2024-05-27T10:02:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.008+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdba8019700 con 0x7fdbb407f6f0 2024-05-27T10:02:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.009+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb4064e10 con 0x7fdbb407f6f0 2024-05-27T10:02:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.013+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdba8024a50 con 0x7fdbb407f6f0 2024-05-27T10:02:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.089+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fdbb41a3540 con 0x7fdbb407f6f0 2024-05-27T10:02:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.095+0000 7fdbb1600700 1 -- 172.21.15.42:0/3499150583 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fdba8024430 con 0x7fdbb407f6f0 2024-05-27T10:02:36.809 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 msgr2=0x7fdbb419ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:36.809 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 --2- 172.21.15.42:0/3499150583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb419ff90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdba80116a0 tx=0x7fdba8011a60 comp rx=0 tx=0).stop 2024-05-27T10:02:36.810 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 shutdown_connections 2024-05-27T10:02:36.810 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 --2- 172.21.15.42:0/3499150583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb407f6f0 0x7fdbb419ff90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:36.810 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 >> 172.21.15.42:0/3499150583 conn(0x7fdbb4105150 msgr2=0x7fdbb4105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:36.810 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 shutdown_connections 2024-05-27T10:02:36.810 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:36.096+0000 7fdbbb600700 1 -- 172.21.15.42:0/3499150583 wait complete. 2024-05-27T10:02:36.811 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-27T10:02:36.839 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:02:36.839 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2024-05-27T10:02:36.841 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-27T10:02:36.842 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-27T10:02:37.276 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T10:02:37.276 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-05-27T10:02:37.542 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968.target.wants/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@.service. 2024-05-27T10:02:37.579 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:37 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3499150583' entity='client.admin' 2024-05-27T10:02:37.844 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:37 smithi042 systemd[1]: Starting Ceph mgr.x for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:02:38.262 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:37 smithi042 podman[31874]: 2024-05-27 10:02:37.846055188 +0000 UTC m=+0.034931195 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:02:38.262 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:37 smithi042 podman[31874]: 2024-05-27 10:02:37.942734582 +0000 UTC m=+0.131610646 container create f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:02:38.262 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 podman[31874]: 2024-05-27 10:02:38.154515184 +0000 UTC m=+0.343391278 container init f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-27T10:02:38.263 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 podman[31874]: 2024-05-27 10:02:38.165002687 +0000 UTC m=+0.353878701 container start f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:02:38.263 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 bash[31874]: f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 2024-05-27T10:02:38.298 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T10:02:38.298 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-05-27T10:02:38.314 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T10:02:38.314 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-05-27T10:02:38.314 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-05-27T10:02:38.314 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T10:02:38.338 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T10:02:38.338 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-05-27T10:02:38.358 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T10:02:38.358 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-05-27T10:02:38.358 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-05-27T10:02:38.359 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-27T10:02:38.359 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2024-05-27T10:02:38.359 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2024-05-27T10:02:38.529 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 systemd[1]: Started Ceph mgr.x for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:02:38.530 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:38.510+0000 7f704bc11200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:02:39.064 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:38.783+0000 7f704bc11200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:02:39.064 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:38.934+0000 7f704bc11200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:02:39.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:39 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2955375605' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T10:02:39.757 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:39.387+0000 7f704bc11200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:02:39.757 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:39.508+0000 7f704bc11200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:02:40.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:40.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:02:40.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:02:40.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T10:02:40.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T10:02:40.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T10:02:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T10:02:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T10:02:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T10:02:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T10:02:40.047 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T10:02:27.893171+0000", 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.062+0000 7f031fe00700 1 Processor -- start 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.063+0000 7f031fe00700 1 -- start start 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.063+0000 7f031fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032007d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.063+0000 7f031fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f032007d770 con 0x7f032007ee00 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.063+0000 7f031ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032007d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.064+0000 7f031ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032007d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38214/0 (socket says 172.21.15.42:38214) 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.064+0000 7f031ea00700 1 -- 172.21.15.42:0/2314595576 learned_addr learned my addr 172.21.15.42:0/2314595576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.064+0000 7f031ea00700 1 -- 172.21.15.42:0/2314595576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032007d8f0 con 0x7f032007ee00 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.065+0000 7f031ea00700 1 --2- 172.21.15.42:0/2314595576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032007d230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0310009e80 tx=0x7f031000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da2a822cd10f05b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.065+0000 7f031e000700 1 -- 172.21.15.42:0/2314595576 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0310017070 con 0x7f032007ee00 2024-05-27T10:02:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.065+0000 7f031e000700 1 -- 172.21.15.42:0/2314595576 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f031000b8c0 con 0x7f032007ee00 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.065+0000 7f031e000700 1 -- 172.21.15.42:0/2314595576 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0310004420 con 0x7f032007ee00 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.066+0000 7f031fe00700 1 -- 172.21.15.42:0/2314595576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 msgr2=0x7f032007d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.066+0000 7f031fe00700 1 --2- 172.21.15.42:0/2314595576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032007d230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0310009e80 tx=0x7f031000b330 comp rx=0 tx=0).stop 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.066+0000 7f031fe00700 1 -- 172.21.15.42:0/2314595576 shutdown_connections 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.066+0000 7f031fe00700 1 --2- 172.21.15.42:0/2314595576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032007d230 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.066+0000 7f031fe00700 1 -- 172.21.15.42:0/2314595576 >> 172.21.15.42:0/2314595576 conn(0x7f0320104f90 msgr2=0x7f0320107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.067+0000 7f031fe00700 1 -- 172.21.15.42:0/2314595576 shutdown_connections 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.067+0000 7f031fe00700 1 -- 172.21.15.42:0/2314595576 wait complete. 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.068+0000 7f031fe00700 1 Processor -- start 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.068+0000 7f031fe00700 1 -- start start 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.069+0000 7f031fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.069+0000 7f031fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f032010fe60 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.069+0000 7f031ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.069+0000 7f031ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38228/0 (socket says 172.21.15.42:38228) 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.069+0000 7f031ea00700 1 -- 172.21.15.42:0/2955375605 learned_addr learned my addr 172.21.15.42:0/2955375605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.069+0000 7f031ea00700 1 -- 172.21.15.42:0/2955375605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03201a02d0 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.070+0000 7f031ea00700 1 --2- 172.21.15.42:0/2955375605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032019fd90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0310004000 tx=0x7f0310003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.070+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0310020020 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.070+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f031000be50 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.070+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a0560 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.070+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0310017040 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.070+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a3170 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.071+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0310023020 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.071+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f031000f050 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.071+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0320064e10 con 0x7f032007ee00 2024-05-27T10:02:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.074+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f031002a020 con 0x7f032007ee00 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.154+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f03201a3340 con 0x7f032007ee00 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.155+0000 7f0317e00700 1 -- 172.21.15.42:0/2955375605 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f031000b860 con 0x7f032007ee00 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.157+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 msgr2=0x7f032019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.157+0000 7f031fe00700 1 --2- 172.21.15.42:0/2955375605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032019fd90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0310004000 tx=0x7f0310003d00 comp rx=0 tx=0).stop 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.158+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 shutdown_connections 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.158+0000 7f031fe00700 1 --2- 172.21.15.42:0/2955375605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f032007ee00 0x7f032019fd90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.158+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 >> 172.21.15.42:0/2955375605 conn(0x7f0320104f90 msgr2=0x7f03201057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.158+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 shutdown_connections 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:39.158+0000 7f031fe00700 1 -- 172.21.15.42:0/2955375605 wait complete. 2024-05-27T10:02:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2024-05-27T10:02:41.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:41.182+0000 7f704bc11200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:02:41.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:41.421+0000 7f704bc11200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:02:42.045 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:41.644+0000 7f704bc11200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:02:42.045 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:42.044+0000 7f704bc11200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:02:42.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:42.179+0000 7f704bc11200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:02:42.698 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:42.448+0000 7f704bc11200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:02:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:42 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1111398941' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T10:02:43.094 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:42.724+0000 7f704bc11200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:02:43.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T10:02:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T10:02:43.621 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T10:02:43.622 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T10:02:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T10:02:27.893171+0000", 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.697+0000 7f52dba00700 1 Processor -- start 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.698+0000 7f52dba00700 1 -- start start 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.698+0000 7f52dba00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d407d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:43.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.698+0000 7f52dba00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52d407d770 con 0x7f52d407ee00 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.699+0000 7f52d9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.699+0000 7f52d9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d407d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38230/0 (socket says 172.21.15.42:38230) 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.699+0000 7f52d9000700 1 -- 172.21.15.42:0/2976127678 learned_addr learned my addr 172.21.15.42:0/2976127678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.699+0000 7f52d9000700 1 -- 172.21.15.42:0/2976127678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d407d8f0 con 0x7f52d407ee00 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.699+0000 7f52d9000700 1 --2- 172.21.15.42:0/2976127678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d407d230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52c40098a0 tx=0x7f52c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc59930ce8278adc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.700+0000 7f52d3e00700 1 -- 172.21.15.42:0/2976127678 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c4004160 con 0x7f52d407ee00 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.700+0000 7f52d3e00700 1 -- 172.21.15.42:0/2976127678 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f52c4004300 con 0x7f52d407ee00 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.700+0000 7f52d3e00700 1 -- 172.21.15.42:0/2976127678 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c40119e0 con 0x7f52d407ee00 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 -- 172.21.15.42:0/2976127678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 msgr2=0x7f52d407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 --2- 172.21.15.42:0/2976127678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d407d230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52c40098a0 tx=0x7f52c4009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 -- 172.21.15.42:0/2976127678 shutdown_connections 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 --2- 172.21.15.42:0/2976127678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d407d230 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:43.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 -- 172.21.15.42:0/2976127678 >> 172.21.15.42:0/2976127678 conn(0x7f52d41051c0 msgr2=0x7f52d4107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 -- 172.21.15.42:0/2976127678 shutdown_connections 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.701+0000 7f52dba00700 1 -- 172.21.15.42:0/2976127678 wait complete. 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.702+0000 7f52dba00700 1 Processor -- start 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.702+0000 7f52dba00700 1 -- start start 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.702+0000 7f52dba00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.702+0000 7f52dba00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4110030 con 0x7f52d407ee00 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.703+0000 7f52d9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.703+0000 7f52d9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d419ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38240/0 (socket says 172.21.15.42:38240) 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.703+0000 7f52d9000700 1 -- 172.21.15.42:0/1111398941 learned_addr learned my addr 172.21.15.42:0/1111398941 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.703+0000 7f52d9000700 1 -- 172.21.15.42:0/1111398941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d41a0480 con 0x7f52d407ee00 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.703+0000 7f52d9000700 1 --2- 172.21.15.42:0/1111398941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d419ff40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52c40119b0 tx=0x7f52c4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c4003af0 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f52c401a8a0 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d41a0710 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c4024830 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d41a3320 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f52c4022020 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.704+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52c402dde0 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.705+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d4064e10 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.708+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f52c401aa50 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.789+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52d41a34f0 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.789+0000 7f52d2000700 1 -- 172.21.15.42:0/1111398941 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f52c4032030 con 0x7f52d407ee00 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 msgr2=0x7f52d419ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 --2- 172.21.15.42:0/1111398941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d419ff40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52c40119b0 tx=0x7f52c4011a90 comp rx=0 tx=0).stop 2024-05-27T10:02:43.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 shutdown_connections 2024-05-27T10:02:43.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 --2- 172.21.15.42:0/1111398941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d407ee00 0x7f52d419ff40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:43.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 >> 172.21.15.42:0/1111398941 conn(0x7f52d41051c0 msgr2=0x7f52d41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:43.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 shutdown_connections 2024-05-27T10:02:43.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:42.791+0000 7f52dba00700 1 -- 172.21.15.42:0/1111398941 wait complete. 2024-05-27T10:02:43.628 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2024-05-27T10:02:43.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:43.562+0000 7f704bc11200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:02:44.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:43.868+0000 7f704bc11200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:02:44.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:44.450+0000 7f704bc11200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:02:44.844 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:44.584+0000 7f704bc11200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:02:45.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:45.064+0000 7f704bc11200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:02:45.809 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:45.540+0000 7f704bc11200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:02:45.809 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:45.680+0000 7f704bc11200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:02:46.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:46.024+0000 7f704bc11200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:02:46.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:46.215+0000 7f704bc11200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:02:46.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:46 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1039157122' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T10:02:46.774 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:46.352+0000 7f704bc11200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:02:47.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:46.772+0000 7f704bc11200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:02:47.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:46.905+0000 7f704bc11200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:02:47.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:47.335 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T10:02:47.336 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T10:02:47.336 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:47.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-27T10:02:47.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T10:02:47.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:47.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T10:02:47.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T10:02:47.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T10:02:27.893171+0000", 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.332+0000 7f22e8000700 1 Processor -- start 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.332+0000 7f22e8000700 1 -- start start 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.333+0000 7f22e8000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e0108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.333+0000 7f22e8000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22e007ce70 con 0x7f22e0108370 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.333+0000 7f22e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e0108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.333+0000 7f22e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e0108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48378/0 (socket says 172.21.15.42:48378) 2024-05-27T10:02:47.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.333+0000 7f22e5600700 1 -- 172.21.15.42:0/2791382105 learned_addr learned my addr 172.21.15.42:0/2791382105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.334+0000 7f22e5600700 1 -- 172.21.15.42:0/2791382105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e0108cc0 con 0x7f22e0108370 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.334+0000 7f22e5600700 1 --2- 172.21.15.42:0/2791382105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e0108780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22d40098a0 tx=0x7f22d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf9d6afb91584d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.335+0000 7f22dfe00700 1 -- 172.21.15.42:0/2791382105 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22d4004160 con 0x7f22e0108370 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.335+0000 7f22dfe00700 1 -- 172.21.15.42:0/2791382105 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f22d4004300 con 0x7f22e0108370 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.335+0000 7f22dfe00700 1 -- 172.21.15.42:0/2791382105 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22d40119e0 con 0x7f22e0108370 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.336+0000 7f22e8000700 1 -- 172.21.15.42:0/2791382105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 msgr2=0x7f22e0108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.336+0000 7f22e8000700 1 --2- 172.21.15.42:0/2791382105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e0108780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22d40098a0 tx=0x7f22d4009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.336+0000 7f22e8000700 1 -- 172.21.15.42:0/2791382105 shutdown_connections 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.336+0000 7f22e8000700 1 --2- 172.21.15.42:0/2791382105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e0108780 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.336+0000 7f22e8000700 1 -- 172.21.15.42:0/2791382105 >> 172.21.15.42:0/2791382105 conn(0x7f22e0104080 msgr2=0x7f22e01064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.336+0000 7f22e8000700 1 -- 172.21.15.42:0/2791382105 shutdown_connections 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.337+0000 7f22e8000700 1 -- 172.21.15.42:0/2791382105 wait complete. 2024-05-27T10:02:47.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.338+0000 7f22e8000700 1 Processor -- start 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.338+0000 7f22e8000700 1 -- start start 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.338+0000 7f22e8000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.338+0000 7f22e8000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0109ce0 con 0x7f22e0108370 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.339+0000 7f22e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.339+0000 7f22e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e019feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48394/0 (socket says 172.21.15.42:48394) 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.339+0000 7f22e5600700 1 -- 172.21.15.42:0/1039157122 learned_addr learned my addr 172.21.15.42:0/1039157122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.339+0000 7f22e5600700 1 -- 172.21.15.42:0/1039157122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e01a03f0 con 0x7f22e0108370 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.340+0000 7f22e5600700 1 --2- 172.21.15.42:0/1039157122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e019feb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f22d40116a0 tx=0x7f22d4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.340+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22d4003970 con 0x7f22e0108370 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.340+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f22d4003b10 con 0x7f22e0108370 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.340+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e01a0680 con 0x7f22e0108370 2024-05-27T10:02:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.340+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22d4019550 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.340+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e01a3290 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.341+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22d4019a80 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.341+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22d400b720 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.341+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e0064e10 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.344+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f22d401aa50 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.426+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f22e002e850 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.426+0000 7f22de000700 1 -- 172.21.15.42:0/1039157122 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f22d401a430 con 0x7f22e0108370 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 msgr2=0x7f22e019feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 --2- 172.21.15.42:0/1039157122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e019feb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f22d40116a0 tx=0x7f22d4011a60 comp rx=0 tx=0).stop 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 shutdown_connections 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 --2- 172.21.15.42:0/1039157122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e0108370 0x7f22e019feb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 >> 172.21.15.42:0/1039157122 conn(0x7f22e0104080 msgr2=0x7f22e0193e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 shutdown_connections 2024-05-27T10:02:47.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:46.428+0000 7f22e8000700 1 -- 172.21.15.42:0/1039157122 wait complete. 2024-05-27T10:02:47.352 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (3/15)... 2024-05-27T10:02:48.030 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:47 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:47.730+0000 7f704bc11200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:02:48.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:48 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:48.028+0000 7f704bc11200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: Activating manager daemon x 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: mgrmap e2: x(active, starting, since 0.00512047s) 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: Manager daemon x is now available 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T10:02:48.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:48 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' 2024-05-27T10:02:49.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:49 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' 2024-05-27T10:02:49.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:49 smithi042 ceph-mon[31661]: from='mgr.14100 172.21.15.42:0/1268660070' entity='mgr.x' 2024-05-27T10:02:49.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:49 smithi042 ceph-mon[31661]: mgrmap e3: x(active, since 1.02366s) 2024-05-27T10:02:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:02:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:02:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T10:02:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T10:02:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T10:02:51.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T10:02:51.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:51.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T10:02:51.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T10:02:51.065 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T10:02:51.066 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T10:02:27.893171+0000", 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T10:02:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.042+0000 7fd0da600700 1 Processor -- start 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.042+0000 7fd0da600700 1 -- start start 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.043+0000 7fd0da600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d410c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.043+0000 7fd0da600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d407d700 con 0x7fd0d410a400 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.043+0000 7fd0d9200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d410c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.043+0000 7fd0d9200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d410c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48486/0 (socket says 172.21.15.42:48486) 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.043+0000 7fd0d9200700 1 -- 172.21.15.42:0/1300195320 learned_addr learned my addr 172.21.15.42:0/1300195320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.044+0000 7fd0d9200700 1 -- 172.21.15.42:0/1300195320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d410cd60 con 0x7fd0d410a400 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.044+0000 7fd0d9200700 1 --2- 172.21.15.42:0/1300195320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d410c820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80098a0 tx=0x7fd0c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac99ac4087ccba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.044+0000 7fd0d3400700 1 -- 172.21.15.42:0/1300195320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c8004160 con 0x7fd0d410a400 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.044+0000 7fd0d3400700 1 -- 172.21.15.42:0/1300195320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd0c8004300 con 0x7fd0d410a400 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.044+0000 7fd0d3400700 1 -- 172.21.15.42:0/1300195320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c80119e0 con 0x7fd0d410a400 2024-05-27T10:02:51.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.045+0000 7fd0da600700 1 -- 172.21.15.42:0/1300195320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 msgr2=0x7fd0d410c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.045+0000 7fd0da600700 1 --2- 172.21.15.42:0/1300195320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d410c820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80098a0 tx=0x7fd0c8009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.046+0000 7fd0da600700 1 -- 172.21.15.42:0/1300195320 shutdown_connections 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.046+0000 7fd0da600700 1 --2- 172.21.15.42:0/1300195320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d410c820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.046+0000 7fd0da600700 1 -- 172.21.15.42:0/1300195320 >> 172.21.15.42:0/1300195320 conn(0x7fd0d4103ea0 msgr2=0x7fd0d4106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.046+0000 7fd0da600700 1 -- 172.21.15.42:0/1300195320 shutdown_connections 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.046+0000 7fd0da600700 1 -- 172.21.15.42:0/1300195320 wait complete. 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.047+0000 7fd0da600700 1 Processor -- start 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.047+0000 7fd0da600700 1 -- start start 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.047+0000 7fd0da600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d4109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.047+0000 7fd0da600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410dc50 con 0x7fd0d410a400 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.048+0000 7fd0d9200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d4109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.048+0000 7fd0d9200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d4109b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48496/0 (socket says 172.21.15.42:48496) 2024-05-27T10:02:51.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.048+0000 7fd0d9200700 1 -- 172.21.15.42:0/2472703722 learned_addr learned my addr 172.21.15.42:0/2472703722 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.048+0000 7fd0d9200700 1 -- 172.21.15.42:0/2472703722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d410a0c0 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.048+0000 7fd0d9200700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d4109b80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80116a0 tx=0x7fd0c8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.049+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c8003970 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.049+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd0c8003b10 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.049+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c8019550 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.049+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d4108310 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.049+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d41087f0 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.050+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d4064e10 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.050+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fd0c801a430 con 0x7fd0d410a400 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.051+0000 7fd0d1600700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd0bc03e410 0x7fd0bc040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:51.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.051+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd0c80535b0 con 0x7fd0d410a400 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.051+0000 7fd0d3e00700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd0bc03e410 0x7fd0bc040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.052+0000 7fd0d3e00700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd0bc03e410 0x7fd0bc040920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0006fd0 tx=0x7fd0c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.058+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0c800bcc0 con 0x7fd0d410a400 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.094+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd0c8018c20 con 0x7fd0d410a400 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.384+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd0d4108e90 con 0x7fd0d410a400 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.385+0000 7fd0d1600700 1 -- 172.21.15.42:0/2472703722 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd0c803d790 con 0x7fd0d410a400 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd0bc03e410 msgr2=0x7fd0bc040920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd0bc03e410 0x7fd0bc040920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0006fd0 tx=0x7fd0c0006e40 comp rx=0 tx=0).stop 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 msgr2=0x7fd0d4109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d4109b80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80116a0 tx=0x7fd0c8011a60 comp rx=0 tx=0).stop 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 shutdown_connections 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd0bc03e410 0x7fd0bc040920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:51.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 --2- 172.21.15.42:0/2472703722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0d410a400 0x7fd0d4109b80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:51.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 >> 172.21.15.42:0/2472703722 conn(0x7fd0d4103ea0 msgr2=0x7fd0d4104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:51.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 shutdown_connections 2024-05-27T10:02:51.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:50.388+0000 7fd0da600700 1 -- 172.21.15.42:0/2472703722 wait complete. 2024-05-27T10:02:51.072 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2024-05-27T10:02:51.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:51 smithi042 ceph-mon[31661]: mgrmap e4: x(active, since 2s) 2024-05-27T10:02:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:51 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2472703722' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T10:02:52.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:52 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1532757866' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T10:02:52.663 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:52.663 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T10:02:52.663 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:02:52.663 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T10:02:52.664 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T10:02:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.766+0000 7fd4d7a00700 1 Processor -- start 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.766+0000 7fd4d7a00700 1 -- start start 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.766+0000 7fd4d7a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d007d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.766+0000 7fd4d7a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d007d840 con 0x7fd4d007ee40 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.767+0000 7fd4d5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d007d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.767+0000 7fd4d5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d007d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48510/0 (socket says 172.21.15.42:48510) 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.767+0000 7fd4d5000700 1 -- 172.21.15.42:0/2876766461 learned_addr learned my addr 172.21.15.42:0/2876766461 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:52.667 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.767+0000 7fd4d5000700 1 -- 172.21.15.42:0/2876766461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d007d9c0 con 0x7fd4d007ee40 2024-05-27T10:02:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.768+0000 7fd4d5000700 1 --2- 172.21.15.42:0/2876766461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d007d270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098a0 tx=0x7fd4c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4717038f53b05e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.768+0000 7fd4cf400700 1 -- 172.21.15.42:0/2876766461 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c4004160 con 0x7fd4d007ee40 2024-05-27T10:02:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.768+0000 7fd4cf400700 1 -- 172.21.15.42:0/2876766461 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd4c4004300 con 0x7fd4d007ee40 2024-05-27T10:02:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.768+0000 7fd4cf400700 1 -- 172.21.15.42:0/2876766461 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c40119e0 con 0x7fd4d007ee40 2024-05-27T10:02:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.769+0000 7fd4d7a00700 1 -- 172.21.15.42:0/2876766461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 msgr2=0x7fd4d007d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.769+0000 7fd4d7a00700 1 --2- 172.21.15.42:0/2876766461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d007d270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098a0 tx=0x7fd4c4009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.769+0000 7fd4d7a00700 1 -- 172.21.15.42:0/2876766461 shutdown_connections 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.769+0000 7fd4d7a00700 1 --2- 172.21.15.42:0/2876766461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d007d270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.769+0000 7fd4d7a00700 1 -- 172.21.15.42:0/2876766461 >> 172.21.15.42:0/2876766461 conn(0x7fd4d01051c0 msgr2=0x7fd4d0107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.769+0000 7fd4d7a00700 1 -- 172.21.15.42:0/2876766461 shutdown_connections 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.770+0000 7fd4d7a00700 1 -- 172.21.15.42:0/2876766461 wait complete. 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.771+0000 7fd4d7a00700 1 Processor -- start 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.771+0000 7fd4d7a00700 1 -- start start 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.771+0000 7fd4d7a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.771+0000 7fd4d7a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d0110060 con 0x7fd4d007ee40 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.772+0000 7fd4d5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.772+0000 7fd4d5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48512/0 (socket says 172.21.15.42:48512) 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.772+0000 7fd4d5000700 1 -- 172.21.15.42:0/1532757866 learned_addr learned my addr 172.21.15.42:0/1532757866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.772+0000 7fd4d5000700 1 -- 172.21.15.42:0/1532757866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d01a04e0 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.772+0000 7fd4d5000700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d019ffa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40119b0 tx=0x7fd4c4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.773+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c4003af0 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.773+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd4c401a8a0 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.773+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d01a0770 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.773+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c4024830 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.773+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d01a3380 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.774+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd4c4022020 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.774+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d0064e10 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.775+0000 7fd4cd600700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd4b803e530 0x7fd4b8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.775+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4c4053650 con 0x7fd4d007ee40 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.775+0000 7fd4cfe00700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd4b803e530 0x7fd4b8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:52.674 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.776+0000 7fd4cfe00700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd4b803e530 0x7fd4b8040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006fd0 tx=0x7fd4bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:51.781+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4c40533b0 con 0x7fd4d007ee40 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.017+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd4d01a4b80 con 0x7fd4d007ee40 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.022+0000 7fd4cd600700 1 -- 172.21.15.42:0/1532757866 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fd4c401aae0 con 0x7fd4d007ee40 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd4b803e530 msgr2=0x7fd4b8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd4b803e530 0x7fd4b8040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006fd0 tx=0x7fd4bc006e40 comp rx=0 tx=0).stop 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 msgr2=0x7fd4d019ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d019ffa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40119b0 tx=0x7fd4c4011a90 comp rx=0 tx=0).stop 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 shutdown_connections 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fd4b803e530 0x7fd4b8040a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 --2- 172.21.15.42:0/1532757866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d007ee40 0x7fd4d019ffa0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 >> 172.21.15.42:0/1532757866 conn(0x7fd4d01051c0 msgr2=0x7fd4d01059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 shutdown_connections 2024-05-27T10:02:52.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:52.025+0000 7fd4d7a00700 1 -- 172.21.15.42:0/1532757866 wait complete. 2024-05-27T10:02:52.676 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2024-05-27T10:02:53.936 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:53 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3429676794' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-27T10:02:54.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: ignoring --setuser ceph since I am not root 2024-05-27T10:02:54.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: ignoring --setgroup ceph since I am not root 2024-05-27T10:02:54.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:54.112+0000 7f5933c1a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:02:54.443 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.348+0000 7fbb29a00700 1 Processor -- start 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.349+0000 7fbb29a00700 1 -- start start 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.349+0000 7fbb29a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb24109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.349+0000 7fbb29a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2410a4e0 con 0x7fbb24109ae0 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.350+0000 7fbb22a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb24109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.350+0000 7fbb22a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb24109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43804/0 (socket says 172.21.15.42:43804) 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.350+0000 7fbb22a00700 1 -- 172.21.15.42:0/2925997785 learned_addr learned my addr 172.21.15.42:0/2925997785 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.350+0000 7fbb22a00700 1 -- 172.21.15.42:0/2925997785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2410acb0 con 0x7fbb24109ae0 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.351+0000 7fbb22a00700 1 --2- 172.21.15.42:0/2925997785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb24109ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb140098a0 tx=0x7fbb14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9960a0d0c86ee06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.351+0000 7fbb21600700 1 -- 172.21.15.42:0/2925997785 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb14004160 con 0x7fbb24109ae0 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.351+0000 7fbb21600700 1 -- 172.21.15.42:0/2925997785 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fbb14004300 con 0x7fbb24109ae0 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.351+0000 7fbb21600700 1 -- 172.21.15.42:0/2925997785 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb140119e0 con 0x7fbb24109ae0 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 -- 172.21.15.42:0/2925997785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 msgr2=0x7fbb24109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 --2- 172.21.15.42:0/2925997785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb24109ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb140098a0 tx=0x7fbb14009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 -- 172.21.15.42:0/2925997785 shutdown_connections 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 --2- 172.21.15.42:0/2925997785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb24109ef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 -- 172.21.15.42:0/2925997785 >> 172.21.15.42:0/2925997785 conn(0x7fbb241051a0 msgr2=0x7fbb241075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 -- 172.21.15.42:0/2925997785 shutdown_connections 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.352+0000 7fbb29a00700 1 -- 172.21.15.42:0/2925997785 wait complete. 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.353+0000 7fbb29a00700 1 Processor -- start 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.353+0000 7fbb29a00700 1 -- start start 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.354+0000 7fbb29a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb2419ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.354+0000 7fbb29a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2410dee0 con 0x7fbb24109ae0 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.354+0000 7fbb22a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb2419ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.354+0000 7fbb22a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb2419ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43816/0 (socket says 172.21.15.42:43816) 2024-05-27T10:02:54.446 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.354+0000 7fbb22a00700 1 -- 172.21.15.42:0/3429676794 learned_addr learned my addr 172.21.15.42:0/3429676794 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.354+0000 7fbb22a00700 1 -- 172.21.15.42:0/3429676794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241a04a0 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.355+0000 7fbb22a00700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb2419ff60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbb140046e0 tx=0x7fbb14011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.355+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb140039e0 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.355+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fbb14003b80 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.355+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb14024700 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.355+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb241a0730 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.355+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241a3340 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.356+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb24064e10 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.357+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fbb14022020 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.357+0000 7fbb1b400700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fbb0803e580 0x7fbb08040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.358+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb14053580 con 0x7fbb24109ae0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.358+0000 7fbb22000700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fbb0803e580 0x7fbb08040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.359+0000 7fbb22000700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fbb0803e580 0x7fbb08040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006fd0 tx=0x7fbb10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.363+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb1400f2e0 con 0x7fbb24109ae0 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.668+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fbb2402e890 con 0x7fbb24109ae0 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.721+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fbb140094f0 con 0x7fbb24109ae0 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.725+0000 7fbb1b400700 1 -- 172.21.15.42:0/3429676794 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fbb1402c350 con 0x7fbb24109ae0 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fbb0803e580 msgr2=0x7fbb08040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fbb0803e580 0x7fbb08040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006fd0 tx=0x7fbb10006e40 comp rx=0 tx=0).stop 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 msgr2=0x7fbb2419ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb2419ff60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbb140046e0 tx=0x7fbb14011a60 comp rx=0 tx=0).stop 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 shutdown_connections 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7fbb0803e580 0x7fbb08040a90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 --2- 172.21.15.42:0/3429676794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb24109ae0 0x7fbb2419ff60 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbb140046e0 tx=0x7fbb14011a60 comp rx=0 tx=0).stop 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.727+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 >> 172.21.15.42:0/3429676794 conn(0x7fbb241051a0 msgr2=0x7fbb2407dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:54.448 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.728+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 shutdown_connections 2024-05-27T10:02:54.449 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:53.728+0000 7fbb29a00700 1 -- 172.21.15.42:0/3429676794 wait complete. 2024-05-27T10:02:54.732 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:54.365+0000 7f5933c1a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:02:54.732 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:54.506+0000 7f5933c1a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:02:55.059 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:54 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3429676794' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-27T10:02:55.059 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:54 smithi042 ceph-mon[31661]: mgrmap e5: x(active, since 5s) 2024-05-27T10:02:55.059 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:54.932+0000 7f5933c1a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:02:55.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:55 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:55.057+0000 7f5933c1a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:02:56.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:02:55 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/787576036' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.151+0000 7f550ee00700 1 Processor -- start 2024-05-27T10:02:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.151+0000 7f550ee00700 1 -- start start 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.151+0000 7f550ee00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f5508109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.151+0000 7f550ee00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f550810a520 con 0x7f5508109b20 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.152+0000 7f5507e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f5508109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.152+0000 7f5507e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f5508109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43846/0 (socket says 172.21.15.42:43846) 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.152+0000 7f5507e00700 1 -- 172.21.15.42:0/3802379175 learned_addr learned my addr 172.21.15.42:0/3802379175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.152+0000 7f5507e00700 1 -- 172.21.15.42:0/3802379175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550810acf0 con 0x7f5508109b20 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.153+0000 7f5507e00700 1 --2- 172.21.15.42:0/3802379175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f5508109f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54f80098a0 tx=0x7f54f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee11ba5c6380fa93 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.153+0000 7f5506a00700 1 -- 172.21.15.42:0/3802379175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f8004160 con 0x7f5508109b20 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.153+0000 7f5506a00700 1 -- 172.21.15.42:0/3802379175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f54f8004300 con 0x7f5508109b20 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.153+0000 7f5506a00700 1 -- 172.21.15.42:0/3802379175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f80119e0 con 0x7f5508109b20 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 -- 172.21.15.42:0/3802379175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 msgr2=0x7f5508109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 --2- 172.21.15.42:0/3802379175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f5508109f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54f80098a0 tx=0x7f54f8009bb0 comp rx=0 tx=0).stop 2024-05-27T10:02:56.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 -- 172.21.15.42:0/3802379175 shutdown_connections 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 --2- 172.21.15.42:0/3802379175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f5508109f30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 -- 172.21.15.42:0/3802379175 >> 172.21.15.42:0/3802379175 conn(0x7f55081051c0 msgr2=0x7f5508107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 -- 172.21.15.42:0/3802379175 shutdown_connections 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.154+0000 7f550ee00700 1 -- 172.21.15.42:0/3802379175 wait complete. 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.155+0000 7f550ee00700 1 Processor -- start 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.155+0000 7f550ee00700 1 -- start start 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.156+0000 7f550ee00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f550819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.156+0000 7f550ee00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f550810df20 con 0x7f5508109b20 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.156+0000 7f5507e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f550819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.156+0000 7f5507e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f550819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43850/0 (socket says 172.21.15.42:43850) 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.156+0000 7f5507e00700 1 -- 172.21.15.42:0/787576036 learned_addr learned my addr 172.21.15.42:0/787576036 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.156+0000 7f5507e00700 1 -- 172.21.15.42:0/787576036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550819e340 con 0x7f5508109b20 2024-05-27T10:02:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.157+0000 7f5507e00700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f550819de00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f54f80119b0 tx=0x7f54f8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.157+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f8003970 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.157+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f54f8003b10 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.157+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54f8024610 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.157+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550819e5d0 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.157+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081a11e0 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.158+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5508064e10 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.159+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f54f801aae0 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.160+0000 7f5504c00700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 0x7f54ec040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.160+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54f8053950 con 0x7f5508109b20 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.160+0000 7f5507400700 1 -- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 msgr2=0x7f54ec040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.160+0000 7f5507400700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 0x7f54ec040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:02:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.170+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54f800f2e0 con 0x7f5508109b20 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.361+0000 7f5507400700 1 -- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 msgr2=0x7f54ec040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.361+0000 7f5507400700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 0x7f54ec040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.458+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f55081a13b0 con 0x7f5508109b20 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.459+0000 7f5504c00700 1 -- 172.21.15.42:0/787576036 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f55081a13b0 con 0x7f5508109b20 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 msgr2=0x7f54ec040af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 0x7f54ec040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 msgr2=0x7f550819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f550819de00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f54f80119b0 tx=0x7f54f8011a90 comp rx=0 tx=0).stop 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 shutdown_connections 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f54ec03e5e0 0x7f54ec040af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 --2- 172.21.15.42:0/787576036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5508109b20 0x7f550819de00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 >> 172.21.15.42:0/787576036 conn(0x7f55081051c0 msgr2=0x7f5508105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:02:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 shutdown_connections 2024-05-27T10:02:56.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:55.462+0000 7f550ee00700 1 -- 172.21.15.42:0/787576036 wait complete. 2024-05-27T10:02:56.156 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-05-27T10:02:56.156 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2024-05-27T10:02:56.838 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:56.689+0000 7f5933c1a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:02:57.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:56.937+0000 7f5933c1a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:02:57.449 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:57.180+0000 7f5933c1a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:02:57.714 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:57.584+0000 7f5933c1a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:02:57.979 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:57.712+0000 7f5933c1a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:02:58.247 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:57.977+0000 7f5933c1a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:02:58.247 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:58.246+0000 7f5933c1a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:02:59.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:59.079+0000 7f5933c1a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:02:59.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:59.381+0000 7f5933c1a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:03:00.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:02:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:02:59.964+0000 7f5933c1a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:03:00.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:00.097+0000 7f5933c1a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:03:00.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:00.579+0000 7f5933c1a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:03:01.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:01.056+0000 7f5933c1a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:03:01.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:01.186+0000 7f5933c1a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:03:01.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:01.526+0000 7f5933c1a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:03:01.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:01.716+0000 7f5933c1a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:03:02.264 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:01.851+0000 7f5933c1a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:03:02.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:02.262+0000 7f5933c1a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:03:02.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:02.392+0000 7f5933c1a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:03:03.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:03.190+0000 7f5933c1a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:03:03.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:03.494+0000 7f5933c1a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:03:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: Active manager daemon x restarted 2024-05-27T10:03:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: Activating manager daemon x 2024-05-27T10:03:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: osdmap e2: 0 total, 0 up, 0 in 2024-05-27T10:03:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: mgrmap e6: x(active, starting, since 0.00652866s) 2024-05-27T10:03:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:03:03.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T10:03:03.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T10:03:03.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T10:03:03.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T10:03:03.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:03 smithi042 ceph-mon[31661]: Manager daemon x is now available 2024-05-27T10:03:04.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: Found migration_current of "None". Setting to last migration. 2024-05-27T10:03:04.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:04 smithi042 ceph-mon[31661]: mgrmap e7: x(active, since 1.01907s) 2024-05-27T10:03:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:03:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.836+0000 7f55fd800700 1 Processor -- start 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.836+0000 7f55fd800700 1 -- start start 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.837+0000 7f55fd800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f807dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.837+0000 7f55fd800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55f807e0b0 con 0x7f55f807f6b0 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.837+0000 7f55f6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f807dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.837+0000 7f55f6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f807dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43866/0 (socket says 172.21.15.42:43866) 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.837+0000 7f55f6a00700 1 -- 172.21.15.42:0/142869321 learned_addr learned my addr 172.21.15.42:0/142869321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.838+0000 7f55f6a00700 1 -- 172.21.15.42:0/142869321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f807e230 con 0x7f55f807f6b0 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.838+0000 7f55f6a00700 1 --2- 172.21.15.42:0/142869321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f807dae0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55e80098a0 tx=0x7f55e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc4c0f1f59f7057 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.838+0000 7f55f5600700 1 -- 172.21.15.42:0/142869321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e8004160 con 0x7f55f807f6b0 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.838+0000 7f55f5600700 1 -- 172.21.15.42:0/142869321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f55e8004300 con 0x7f55f807f6b0 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.838+0000 7f55f5600700 1 -- 172.21.15.42:0/142869321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e80119e0 con 0x7f55f807f6b0 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.839+0000 7f55fd800700 1 -- 172.21.15.42:0/142869321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 msgr2=0x7f55f807dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.839+0000 7f55fd800700 1 --2- 172.21.15.42:0/142869321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f807dae0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55e80098a0 tx=0x7f55e8009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.839+0000 7f55fd800700 1 -- 172.21.15.42:0/142869321 shutdown_connections 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.839+0000 7f55fd800700 1 --2- 172.21.15.42:0/142869321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f807dae0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.839+0000 7f55fd800700 1 -- 172.21.15.42:0/142869321 >> 172.21.15.42:0/142869321 conn(0x7f55f8105140 msgr2=0x7f55f81075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.840+0000 7f55fd800700 1 -- 172.21.15.42:0/142869321 shutdown_connections 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.840+0000 7f55fd800700 1 -- 172.21.15.42:0/142869321 wait complete. 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.841+0000 7f55fd800700 1 Processor -- start 2024-05-27T10:03:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.841+0000 7f55fd800700 1 -- start start 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.842+0000 7f55fd800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.842+0000 7f55fd800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8110070 con 0x7f55f807f6b0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.842+0000 7f55f6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.842+0000 7f55f6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f819ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43880/0 (socket says 172.21.15.42:43880) 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.842+0000 7f55f6a00700 1 -- 172.21.15.42:0/1749726695 learned_addr learned my addr 172.21.15.42:0/1749726695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.843+0000 7f55f6a00700 1 -- 172.21.15.42:0/1749726695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f81a04b0 con 0x7f55f807f6b0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.843+0000 7f55f6a00700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f819ff70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f55e80119b0 tx=0x7f55e8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.843+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e8003af0 con 0x7f55f807f6b0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.843+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f55e801a8a0 con 0x7f55f807f6b0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.843+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e8024830 con 0x7f55f807f6b0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.844+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f81a0740 con 0x7f55f807f6b0 2024-05-27T10:03:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.844+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a3350 con 0x7f55f807f6b0 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.845+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f55e8022020 con 0x7f55f807f6b0 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.846+0000 7f55ef400700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.846+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 --> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55dc041150 con 0x7f55dc03e530 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.846+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f55e8053600 con 0x7f55f807f6b0 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.846+0000 7f55f6000700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:56.846+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:57.047+0000 7f55f6000700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:57.047+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:57.448+0000 7f55f6000700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:57.448+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:58.249+0000 7f55f6000700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:03:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:58.249+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:59.851+0000 7f55f6000700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:02:59.851+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:03.055+0000 7f55f6000700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3950108791 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:03.055+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:03.503+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f55e800b3e0 con 0x7f55f807f6b0 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:03.504+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 msgr2=0x7f55dc040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:03.504+0000 7f55ef400700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.510+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f55e8055170 con 0x7f55f807f6b0 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.510+0000 7f55ef400700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f55dc041e70 0x7f55dc044290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.510+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55e8021dc0 con 0x7f55dc041e70 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.512+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f55dc041e70 0x7f55dc044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.515+0000 7f55f6000700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f55dc041e70 0x7f55dc044290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55e4003a10 tx=0x7f55e4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.519+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f55e8021dc0 con 0x7f55dc041e70 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.534+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f55f8064e10 con 0x7f55dc041e70 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.535+0000 7f55ef400700 1 -- 172.21.15.42:0/1749726695 <== mgr.14120 v2:172.21.15.42:6800/3098170734 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f55f8064e10 con 0x7f55dc041e70 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f55dc041e70 msgr2=0x7f55dc044290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f55dc041e70 0x7f55dc044290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55e4003a10 tx=0x7f55e4009290 comp rx=0 tx=0).stop 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 msgr2=0x7f55f819ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f819ff70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f55e80119b0 tx=0x7f55e8011a90 comp rx=0 tx=0).stop 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 shutdown_connections 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f55dc041e70 0x7f55dc044290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f807f6b0 0x7f55f819ff70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 --2- 172.21.15.42:0/1749726695 >> [v2:172.21.15.42:6800/3950108791,v1:172.21.15.42:6801/3950108791] conn(0x7f55dc03e530 0x7f55dc040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 >> 172.21.15.42:0/1749726695 conn(0x7f55f8105140 msgr2=0x7f55f8105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 shutdown_connections 2024-05-27T10:03:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:04.536+0000 7f55fd800700 1 -- 172.21.15.42:0/1749726695 wait complete. 2024-05-27T10:03:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2024-05-27T10:03:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2024-05-27T10:03:05.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:05 smithi042 ceph-mon[31661]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T10:03:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:05 smithi042 ceph-mon[31661]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T10:03:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:05 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: [27/May/2024:10:03:05] ENGINE Bus STARTING 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: [27/May/2024:10:03:05] ENGINE Serving on http://172.21.15.42:8765 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: [27/May/2024:10:03:05] ENGINE Serving on https://172.21.15.42:7150 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: [27/May/2024:10:03:05] ENGINE Bus STARTED 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: mgrmap e8: x(active, since 2s) 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:06.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:06 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:03:07.085 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.928+0000 7f6b19200700 1 Processor -- start 2024-05-27T10:03:07.085 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.929+0000 7f6b19200700 1 -- start start 2024-05-27T10:03:07.085 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.929+0000 7f6b19200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1410bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:07.085 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.929+0000 7f6b19200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410c4e0 con 0x7f6b1410bae0 2024-05-27T10:03:07.085 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.930+0000 7f6b13400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1410bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.930+0000 7f6b13400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1410bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52088/0 (socket says 172.21.15.42:52088) 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.930+0000 7f6b13400700 1 -- 172.21.15.42:0/1379194280 learned_addr learned my addr 172.21.15.42:0/1379194280 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.930+0000 7f6b13400700 1 -- 172.21.15.42:0/1379194280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1410ccb0 con 0x7f6b1410bae0 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.931+0000 7f6b13400700 1 --2- 172.21.15.42:0/1379194280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1410bef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b040098a0 tx=0x7f6b04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dac987e65f41cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.931+0000 7f6b12000700 1 -- 172.21.15.42:0/1379194280 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b04004160 con 0x7f6b1410bae0 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.931+0000 7f6b12000700 1 -- 172.21.15.42:0/1379194280 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6b04004300 con 0x7f6b1410bae0 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.931+0000 7f6b12000700 1 -- 172.21.15.42:0/1379194280 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b040119e0 con 0x7f6b1410bae0 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 -- 172.21.15.42:0/1379194280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 msgr2=0x7f6b1410bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 --2- 172.21.15.42:0/1379194280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1410bef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b040098a0 tx=0x7f6b04009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 -- 172.21.15.42:0/1379194280 shutdown_connections 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 --2- 172.21.15.42:0/1379194280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1410bef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 -- 172.21.15.42:0/1379194280 >> 172.21.15.42:0/1379194280 conn(0x7f6b141074b0 msgr2=0x7f6b141098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 -- 172.21.15.42:0/1379194280 shutdown_connections 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.932+0000 7f6b19200700 1 -- 172.21.15.42:0/1379194280 wait complete. 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.933+0000 7f6b19200700 1 Processor -- start 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.934+0000 7f6b19200700 1 -- start start 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.934+0000 7f6b19200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.934+0000 7f6b19200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14110060 con 0x7f6b1410bae0 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.934+0000 7f6b13400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.934+0000 7f6b13400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1419ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52102/0 (socket says 172.21.15.42:52102) 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.934+0000 7f6b13400700 1 -- 172.21.15.42:0/4243931298 learned_addr learned my addr 172.21.15.42:0/4243931298 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.935+0000 7f6b13400700 1 -- 172.21.15.42:0/4243931298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141a0440 con 0x7f6b1410bae0 2024-05-27T10:03:07.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.935+0000 7f6b13400700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1419ff00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b040046e0 tx=0x7f6b04011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.935+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b040039e0 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.935+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6b04003b80 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.935+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b04024700 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.935+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141a06d0 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.936+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a1030 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.937+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14064e10 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.937+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6b04022020 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.937+0000 7f6b0be00700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f6af803e580 0x7f6af8040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.938+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b04053910 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.938+0000 7f6b12a00700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f6af803e580 0x7f6af8040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.939+0000 7f6b12a00700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f6af803e580 0x7f6af8040a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b00006fd0 tx=0x7f6b00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:05.944+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b0400f2e0 con 0x7f6b1410bae0 2024-05-27T10:03:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.202+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6b14109340 con 0x7f6af803e580 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.209+0000 7f6b0be00700 1 -- 172.21.15.42:0/4243931298 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b14109340 con 0x7f6af803e580 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f6af803e580 msgr2=0x7f6af8040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f6af803e580 0x7f6af8040a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b00006fd0 tx=0x7f6b00006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 msgr2=0x7f6b1419ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1419ff00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b040046e0 tx=0x7f6b04011a60 comp rx=0 tx=0).stop 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 shutdown_connections 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f6af803e580 0x7f6af8040a90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 --2- 172.21.15.42:0/4243931298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b1410bae0 0x7f6b1419ff00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 >> 172.21.15.42:0/4243931298 conn(0x7f6b141074b0 msgr2=0x7f6b14108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 shutdown_connections 2024-05-27T10:03:07.089 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:06.213+0000 7f6b19200700 1 -- 172.21.15.42:0/4243931298 wait complete. 2024-05-27T10:03:07.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:07 smithi042 ceph-mon[31661]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:08.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.743+0000 7fd3b4600700 1 Processor -- start 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.743+0000 7fd3b4600700 1 -- start start 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.744+0000 7fd3b4600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac07d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.744+0000 7fd3b4600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac07d860 con 0x7fd3ac07ee60 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.744+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.744+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac07d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52110/0 (socket says 172.21.15.42:52110) 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.744+0000 7fd3b1c00700 1 -- 172.21.15.42:0/4078195086 learned_addr learned my addr 172.21.15.42:0/4078195086 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.745+0000 7fd3b1c00700 1 -- 172.21.15.42:0/4078195086 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac07d9e0 con 0x7fd3ac07ee60 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.745+0000 7fd3b1c00700 1 --2- 172.21.15.42:0/4078195086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac07d290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00098a0 tx=0x7fd3a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e7345e7aa32e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.745+0000 7fd3abe00700 1 -- 172.21.15.42:0/4078195086 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a0004160 con 0x7fd3ac07ee60 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.745+0000 7fd3abe00700 1 -- 172.21.15.42:0/4078195086 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd3a0004300 con 0x7fd3ac07ee60 2024-05-27T10:03:08.641 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.745+0000 7fd3abe00700 1 -- 172.21.15.42:0/4078195086 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a00119e0 con 0x7fd3ac07ee60 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.746+0000 7fd3b4600700 1 -- 172.21.15.42:0/4078195086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 msgr2=0x7fd3ac07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.746+0000 7fd3b4600700 1 --2- 172.21.15.42:0/4078195086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac07d290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00098a0 tx=0x7fd3a0009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.746+0000 7fd3b4600700 1 -- 172.21.15.42:0/4078195086 shutdown_connections 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.746+0000 7fd3b4600700 1 --2- 172.21.15.42:0/4078195086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac07d290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.746+0000 7fd3b4600700 1 -- 172.21.15.42:0/4078195086 >> 172.21.15.42:0/4078195086 conn(0x7fd3ac1050c0 msgr2=0x7fd3ac107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.747+0000 7fd3b4600700 1 -- 172.21.15.42:0/4078195086 shutdown_connections 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.747+0000 7fd3b4600700 1 -- 172.21.15.42:0/4078195086 wait complete. 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b4600700 1 Processor -- start 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b4600700 1 -- start start 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b4600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac19ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b4600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac110070 con 0x7fd3ac07ee60 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac19ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac19ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52114/0 (socket says 172.21.15.42:52114) 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.748+0000 7fd3b1c00700 1 -- 172.21.15.42:0/2583201697 learned_addr learned my addr 172.21.15.42:0/2583201697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.749+0000 7fd3b1c00700 1 -- 172.21.15.42:0/2583201697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac1a0490 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.749+0000 7fd3b1c00700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac19ff50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00116a0 tx=0x7fd3a0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.749+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a00039f0 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.749+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd3a0003b90 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.749+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac1a0720 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.750+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a001a600 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.750+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac1a3330 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.751+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac064e10 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.751+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd3a0024a50 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.752+0000 7fd3aa000700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd39403e4e0 0x7fd3940409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.752+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd3a0053bf0 con 0x7fd3ac07ee60 2024-05-27T10:03:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.752+0000 7fd3b1200700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd39403e4e0 0x7fd3940409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.753+0000 7fd3b1200700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd39403e4e0 0x7fd3940409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd398006fd0 tx=0x7fd398006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.758+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3a000fdd0 con 0x7fd3ac07ee60 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.996+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd3ac106000 con 0x7fd39403e4e0 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:07.998+0000 7fd3aa000700 1 -- 172.21.15.42:0/2583201697 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd3ac106000 con 0x7fd39403e4e0 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.001+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd39403e4e0 msgr2=0x7fd3940409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.001+0000 7fd3b4600700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd39403e4e0 0x7fd3940409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd398006fd0 tx=0x7fd398006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 msgr2=0x7fd3ac19ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac19ff50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00116a0 tx=0x7fd3a0011a60 comp rx=0 tx=0).stop 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 shutdown_connections 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd39403e4e0 0x7fd3940409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 --2- 172.21.15.42:0/2583201697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ac07ee60 0x7fd3ac19ff50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 >> 172.21.15.42:0/2583201697 conn(0x7fd3ac1050c0 msgr2=0x7fd3ac1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:08.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 shutdown_connections 2024-05-27T10:03:08.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:08.002+0000 7fd3b4600700 1 -- 172.21.15.42:0/2583201697 wait complete. 2024-05-27T10:03:08.645 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2024-05-27T10:03:09.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:08 smithi042 ceph-mon[31661]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: Generating public/private rsa key pair. 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: Your identification has been saved in /tmp/tmpnzyxaiu8/key. 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: Your public key has been saved in /tmp/tmpnzyxaiu8/key.pub. 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: The key fingerprint is: 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: SHA256:YvZscUE3kbb695u2JuXCZ7nP2Apklglfh6TOOWsIc08 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: The key's randomart image is: 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: +---[RSA 3072]----+ 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | . +o. | 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | . .o+ . | 2024-05-27T10:03:10.095 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | o...o .| 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | *.= . | 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | + S o.E | 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | o + *.B o . | 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | + ..* o .| 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | . ..=.@.| 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: | .XBX| 2024-05-27T10:03:10.096 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: +----[SHA256]-----+ 2024-05-27T10:03:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.334+0000 7fc0e8000700 1 Processor -- start 2024-05-27T10:03:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.334+0000 7fc0e8000700 1 -- start start 2024-05-27T10:03:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.335+0000 7fc0e8000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e0109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.335+0000 7fc0e8000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010a520 con 0x7fc0e0109b20 2024-05-27T10:03:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.335+0000 7fc0e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e0109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.335+0000 7fc0e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e0109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52122/0 (socket says 172.21.15.42:52122) 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.335+0000 7fc0e5600700 1 -- 172.21.15.42:0/3053823952 learned_addr learned my addr 172.21.15.42:0/3053823952 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.336+0000 7fc0e5600700 1 -- 172.21.15.42:0/3053823952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e010acf0 con 0x7fc0e0109b20 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.336+0000 7fc0e5600700 1 --2- 172.21.15.42:0/3053823952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e0109f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40098a0 tx=0x7fc0d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf613c2cb3edeba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.336+0000 7fc0dfe00700 1 -- 172.21.15.42:0/3053823952 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0d4004160 con 0x7fc0e0109b20 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.336+0000 7fc0dfe00700 1 -- 172.21.15.42:0/3053823952 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc0d4004300 con 0x7fc0e0109b20 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.336+0000 7fc0dfe00700 1 -- 172.21.15.42:0/3053823952 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0d40119e0 con 0x7fc0e0109b20 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.337+0000 7fc0e8000700 1 -- 172.21.15.42:0/3053823952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 msgr2=0x7fc0e0109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.337+0000 7fc0e8000700 1 --2- 172.21.15.42:0/3053823952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e0109f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40098a0 tx=0x7fc0d4009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.338+0000 7fc0e8000700 1 -- 172.21.15.42:0/3053823952 shutdown_connections 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.338+0000 7fc0e8000700 1 --2- 172.21.15.42:0/3053823952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e0109f30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.338+0000 7fc0e8000700 1 -- 172.21.15.42:0/3053823952 >> 172.21.15.42:0/3053823952 conn(0x7fc0e01051c0 msgr2=0x7fc0e0107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.338+0000 7fc0e8000700 1 -- 172.21.15.42:0/3053823952 shutdown_connections 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.338+0000 7fc0e8000700 1 -- 172.21.15.42:0/3053823952 wait complete. 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.339+0000 7fc0e8000700 1 Processor -- start 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.339+0000 7fc0e8000700 1 -- start start 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.339+0000 7fc0e8000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.339+0000 7fc0e8000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010df20 con 0x7fc0e0109b20 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.340+0000 7fc0e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.340+0000 7fc0e5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52138/0 (socket says 172.21.15.42:52138) 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.340+0000 7fc0e5600700 1 -- 172.21.15.42:0/189374841 learned_addr learned my addr 172.21.15.42:0/189374841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.340+0000 7fc0e5600700 1 -- 172.21.15.42:0/189374841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e019e340 con 0x7fc0e0109b20 2024-05-27T10:03:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.341+0000 7fc0e5600700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e019de00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40119b0 tx=0x7fc0d4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.341+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0d4003970 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.341+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc0d4003b10 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.341+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0d4024610 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.341+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e019e5d0 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.341+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e01a11e0 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.342+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc0d402d3f0 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.342+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e0064e10 con 0x7fc0e0109b20 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.343+0000 7fc0de000700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc0c803e530 0x7fc0c8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:10.523 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.343+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc0d4053990 con 0x7fc0e0109b20 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.343+0000 7fc0e4c00700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc0c803e530 0x7fc0c8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.344+0000 7fc0e4c00700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc0c803e530 0x7fc0c8040a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc006fd0 tx=0x7fc0cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.349+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0d40094f0 con 0x7fc0e0109b20 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.584+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fc0e0106090 con 0x7fc0c803e530 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.747+0000 7fc0de000700 1 -- 172.21.15.42:0/189374841 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc0e0106090 con 0x7fc0c803e530 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.750+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc0c803e530 msgr2=0x7fc0c8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.750+0000 7fc0e8000700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc0c803e530 0x7fc0c8040a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc006fd0 tx=0x7fc0cc006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 msgr2=0x7fc0e019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e019de00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40119b0 tx=0x7fc0d4011a90 comp rx=0 tx=0).stop 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 shutdown_connections 2024-05-27T10:03:10.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc0c803e530 0x7fc0c8040a40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:10.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 --2- 172.21.15.42:0/189374841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0e0109b20 0x7fc0e019de00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:10.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 >> 172.21.15.42:0/189374841 conn(0x7fc0e01051c0 msgr2=0x7fc0e0105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:10.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 shutdown_connections 2024-05-27T10:03:10.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:09.751+0000 7fc0e8000700 1 -- 172.21.15.42:0/189374841 wait complete. 2024-05-27T10:03:10.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:10 smithi042 ceph-mon[31661]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:10.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:10 smithi042 ceph-mon[31661]: Generating ssh key... 2024-05-27T10:03:10.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:10 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:10.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:10 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:12.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:12 smithi042 ceph-mon[31661]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:12.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:03:12.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.287+0000 7eff13e00700 1 Processor -- start 2024-05-27T10:03:12.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.288+0000 7eff13e00700 1 -- start start 2024-05-27T10:03:12.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.288+0000 7eff13e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:12.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.288+0000 7eff13e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10a4c0 con 0x7eff0c109ac0 2024-05-27T10:03:12.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.289+0000 7eff11400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:12.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.289+0000 7eff11400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c109ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52146/0 (socket says 172.21.15.42:52146) 2024-05-27T10:03:12.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.289+0000 7eff11400700 1 -- 172.21.15.42:0/2425271366 learned_addr learned my addr 172.21.15.42:0/2425271366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:12.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.289+0000 7eff11400700 1 -- 172.21.15.42:0/2425271366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c10ac90 con 0x7eff0c109ac0 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.290+0000 7eff11400700 1 --2- 172.21.15.42:0/2425271366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c109ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff00009e80 tx=0x7eff0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870df3f32e1234d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.290+0000 7eff0be00700 1 -- 172.21.15.42:0/2425271366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00017070 con 0x7eff0c109ac0 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.290+0000 7eff0be00700 1 -- 172.21.15.42:0/2425271366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7eff0000b8c0 con 0x7eff0c109ac0 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.290+0000 7eff0be00700 1 -- 172.21.15.42:0/2425271366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00004420 con 0x7eff0c109ac0 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 -- 172.21.15.42:0/2425271366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 msgr2=0x7eff0c109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 --2- 172.21.15.42:0/2425271366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c109ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff00009e80 tx=0x7eff0000b330 comp rx=0 tx=0).stop 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 -- 172.21.15.42:0/2425271366 shutdown_connections 2024-05-27T10:03:12.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 --2- 172.21.15.42:0/2425271366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c109ed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:12.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 -- 172.21.15.42:0/2425271366 >> 172.21.15.42:0/2425271366 conn(0x7eff0c105140 msgr2=0x7eff0c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:12.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 -- 172.21.15.42:0/2425271366 shutdown_connections 2024-05-27T10:03:12.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.291+0000 7eff13e00700 1 -- 172.21.15.42:0/2425271366 wait complete. 2024-05-27T10:03:12.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.292+0000 7eff13e00700 1 Processor -- start 2024-05-27T10:03:12.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.293+0000 7eff13e00700 1 -- start start 2024-05-27T10:03:12.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.293+0000 7eff13e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.293+0000 7eff13e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10e040 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.293+0000 7eff11400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.293+0000 7eff11400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c1a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52156/0 (socket says 172.21.15.42:52156) 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.293+0000 7eff11400700 1 -- 172.21.15.42:0/1824404371 learned_addr learned my addr 172.21.15.42:0/1824404371 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.294+0000 7eff11400700 1 -- 172.21.15.42:0/1824404371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c1a05a0 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.294+0000 7eff11400700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c1a0060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff00004110 tx=0x7eff00009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.294+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00020020 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.294+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7eff00003df0 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.294+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00017040 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.294+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c1a0830 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.295+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1a3440 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.296+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c064e10 con 0x7eff0c109ac0 2024-05-27T10:03:12.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.296+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7eff00023020 con 0x7eff0c109ac0 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.296+0000 7eff0a000700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7efef403e580 0x7efef4040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.297+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff0000f050 con 0x7eff0c109ac0 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.297+0000 7eff10a00700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7efef403e580 0x7efef4040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.298+0000 7eff10a00700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7efef403e580 0x7efef4040a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efef8006fd0 tx=0x7efef8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.304+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff0002a020 con 0x7eff0c109ac0 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.541+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7eff0c07e1c0 con 0x7efef403e580 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.543+0000 7eff0a000700 1 -- 172.21.15.42:0/1824404371 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7eff0c07e1c0 con 0x7efef403e580 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7efef403e580 msgr2=0x7efef4040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:12.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7efef403e580 0x7efef4040a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efef8006fd0 tx=0x7efef8006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 msgr2=0x7eff0c1a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c1a0060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff00004110 tx=0x7eff00009390 comp rx=0 tx=0).stop 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 shutdown_connections 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7efef403e580 0x7efef4040a90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 --2- 172.21.15.42:0/1824404371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff0c109ac0 0x7eff0c1a0060 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 >> 172.21.15.42:0/1824404371 conn(0x7eff0c105140 msgr2=0x7eff0c07dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 shutdown_connections 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:11.548+0000 7eff13e00700 1 -- 172.21.15.42:0/1824404371 wait complete. 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-27T10:03:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2024-05-27T10:03:12.289 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2024-05-27T10:03:14.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:14 smithi042 ceph-mon[31661]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:15.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:15 smithi042 ceph-mon[31661]: Deploying cephadm binary to smithi042 2024-05-27T10:03:20.392 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2024-05-27T10:03:20.392 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.039+0000 7f125a000700 1 Processor -- start 2024-05-27T10:03:20.392 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.039+0000 7f125a000700 1 -- start start 2024-05-27T10:03:20.392 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.040+0000 7f125a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125407d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:20.392 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.040+0000 7f125a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f125407d770 con 0x7f125407ee00 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.040+0000 7f1253400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.040+0000 7f1253400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125407d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58710/0 (socket says 172.21.15.42:58710) 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.041+0000 7f1253400700 1 -- 172.21.15.42:0/2362979525 learned_addr learned my addr 172.21.15.42:0/2362979525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.041+0000 7f1253400700 1 -- 172.21.15.42:0/2362979525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125407d8f0 con 0x7f125407ee00 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.041+0000 7f1253400700 1 --2- 172.21.15.42:0/2362979525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125407d230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12440098a0 tx=0x7f1244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52b39707ce6f9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.042+0000 7f1252000700 1 -- 172.21.15.42:0/2362979525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1244004160 con 0x7f125407ee00 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.042+0000 7f1252000700 1 -- 172.21.15.42:0/2362979525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1244004300 con 0x7f125407ee00 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.042+0000 7f1252000700 1 -- 172.21.15.42:0/2362979525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12440119e0 con 0x7f125407ee00 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.042+0000 7f125a000700 1 -- 172.21.15.42:0/2362979525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 msgr2=0x7f125407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:20.393 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.042+0000 7f125a000700 1 --2- 172.21.15.42:0/2362979525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125407d230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12440098a0 tx=0x7f1244009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.043+0000 7f125a000700 1 -- 172.21.15.42:0/2362979525 shutdown_connections 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.043+0000 7f125a000700 1 --2- 172.21.15.42:0/2362979525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125407d230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.043+0000 7f125a000700 1 -- 172.21.15.42:0/2362979525 >> 172.21.15.42:0/2362979525 conn(0x7f12541051c0 msgr2=0x7f1254107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.043+0000 7f125a000700 1 -- 172.21.15.42:0/2362979525 shutdown_connections 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.043+0000 7f125a000700 1 -- 172.21.15.42:0/2362979525 wait complete. 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.044+0000 7f125a000700 1 Processor -- start 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.044+0000 7f125a000700 1 -- start start 2024-05-27T10:03:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.045+0000 7f125a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.045+0000 7f125a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1254110030 con 0x7f125407ee00 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.045+0000 7f1253400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.045+0000 7f1253400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125419ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58724/0 (socket says 172.21.15.42:58724) 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.045+0000 7f1253400700 1 -- 172.21.15.42:0/1174676599 learned_addr learned my addr 172.21.15.42:0/1174676599 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.045+0000 7f1253400700 1 -- 172.21.15.42:0/1174676599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12541a0480 con 0x7f125407ee00 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.046+0000 7f1253400700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125419ff40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12440119b0 tx=0x7f1244011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.046+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1244003af0 con 0x7f125407ee00 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.046+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f124401a8a0 con 0x7f125407ee00 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.046+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1244024830 con 0x7f125407ee00 2024-05-27T10:03:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.046+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12541a0710 con 0x7f125407ee00 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.046+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12541a3320 con 0x7f125407ee00 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.047+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1244022020 con 0x7f125407ee00 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.047+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1254064e10 con 0x7f125407ee00 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.048+0000 7f124be00700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f123803e580 0x7f1238040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.048+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1244053b50 con 0x7f125407ee00 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.048+0000 7f1252a00700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f123803e580 0x7f1238040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.049+0000 7f1252a00700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f123803e580 0x7f1238040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1240006fd0 tx=0x7f1240006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.054+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f124400b3e0 con 0x7f125407ee00 2024-05-27T10:03:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:13.296+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) -- 0x7f12541a34f0 con 0x7f123803e580 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.653+0000 7f124be00700 1 -- 172.21.15.42:0/1174676599 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f12541a34f0 con 0x7f123803e580 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.657+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f123803e580 msgr2=0x7f1238040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.657+0000 7f125a000700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f123803e580 0x7f1238040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1240006fd0 tx=0x7f1240006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.657+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 msgr2=0x7f125419ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.657+0000 7f125a000700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125419ff40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12440119b0 tx=0x7f1244011a90 comp rx=0 tx=0).stop 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.658+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 shutdown_connections 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.658+0000 7f125a000700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f123803e580 0x7f1238040a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.658+0000 7f125a000700 1 --2- 172.21.15.42:0/1174676599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125407ee00 0x7f125419ff40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.658+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 >> 172.21.15.42:0/1174676599 conn(0x7f12541051c0 msgr2=0x7f12541059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:20.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.658+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 shutdown_connections 2024-05-27T10:03:20.398 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:19.658+0000 7f125a000700 1 -- 172.21.15.42:0/1174676599 wait complete. 2024-05-27T10:03:20.398 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mon service... 2024-05-27T10:03:20.914 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:20 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:20.914 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:20 smithi042 ceph-mon[31661]: Added host smithi042 2024-05-27T10:03:20.915 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:20 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:03:22.313 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-27T10:03:22.314 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.358+0000 7f4837200700 1 Processor -- start 2024-05-27T10:03:22.314 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.359+0000 7f4837200700 1 -- start start 2024-05-27T10:03:22.314 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.359+0000 7f4837200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483810bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:22.314 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.359+0000 7f4837200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f483810c4e0 con 0x7f483810bae0 2024-05-27T10:03:22.315 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.359+0000 7f4835e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483810bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:22.315 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.360+0000 7f4835e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483810bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58736/0 (socket says 172.21.15.42:58736) 2024-05-27T10:03:22.315 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.360+0000 7f4835e00700 1 -- 172.21.15.42:0/2872653478 learned_addr learned my addr 172.21.15.42:0/2872653478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:22.315 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.360+0000 7f4835e00700 1 -- 172.21.15.42:0/2872653478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483810ccb0 con 0x7f483810bae0 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.361+0000 7f4835e00700 1 --2- 172.21.15.42:0/2872653478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483810bef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f48280098a0 tx=0x7f4828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28acceef2fd8cb1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.361+0000 7f4834a00700 1 -- 172.21.15.42:0/2872653478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4828004160 con 0x7f483810bae0 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.361+0000 7f4834a00700 1 -- 172.21.15.42:0/2872653478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4828004300 con 0x7f483810bae0 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.361+0000 7f4834a00700 1 -- 172.21.15.42:0/2872653478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48280119e0 con 0x7f483810bae0 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 -- 172.21.15.42:0/2872653478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 msgr2=0x7f483810bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 --2- 172.21.15.42:0/2872653478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483810bef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f48280098a0 tx=0x7f4828009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 -- 172.21.15.42:0/2872653478 shutdown_connections 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 --2- 172.21.15.42:0/2872653478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483810bef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:22.317 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 -- 172.21.15.42:0/2872653478 >> 172.21.15.42:0/2872653478 conn(0x7f48381074b0 msgr2=0x7f48381098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 -- 172.21.15.42:0/2872653478 shutdown_connections 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.362+0000 7f4837200700 1 -- 172.21.15.42:0/2872653478 wait complete. 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.363+0000 7f4837200700 1 Processor -- start 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.363+0000 7f4837200700 1 -- start start 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.364+0000 7f4837200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.364+0000 7f4837200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4838110060 con 0x7f483810bae0 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.364+0000 7f4835e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:22.318 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.364+0000 7f4835e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483819ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58742/0 (socket says 172.21.15.42:58742) 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.364+0000 7f4835e00700 1 -- 172.21.15.42:0/3145618937 learned_addr learned my addr 172.21.15.42:0/3145618937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.364+0000 7f4835e00700 1 -- 172.21.15.42:0/3145618937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48381a0440 con 0x7f483810bae0 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.365+0000 7f4835e00700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483819ff00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48280046e0 tx=0x7f4828011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.365+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48280039e0 con 0x7f483810bae0 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.365+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4828003b80 con 0x7f483810bae0 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.365+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4828024700 con 0x7f483810bae0 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.365+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381a06d0 con 0x7f483810bae0 2024-05-27T10:03:22.321 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.365+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381a1030 con 0x7f483810bae0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.367+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4838064e10 con 0x7f483810bae0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.367+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4828022020 con 0x7f483810bae0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.367+0000 7f482ea00700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f481c03e580 0x7f481c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.367+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48280538f0 con 0x7f483810bae0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.368+0000 7f4835400700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f481c03e580 0x7f481c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.369+0000 7f4835400700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f481c03e580 0x7f481c040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.373+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f482800b3e0 con 0x7f483810bae0 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.641+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f48381a18b0 con 0x7f481c03e580 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.652+0000 7f482ea00700 1 -- 172.21.15.42:0/3145618937 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f48381a18b0 con 0x7f481c03e580 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f481c03e580 msgr2=0x7f481c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f481c03e580 0x7f481c040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 msgr2=0x7f483819ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:22.322 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483819ff00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48280046e0 tx=0x7f4828011a60 comp rx=0 tx=0).stop 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 shutdown_connections 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f481c03e580 0x7f481c040a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 --2- 172.21.15.42:0/3145618937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f483810bae0 0x7f483819ff00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.655+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 >> 172.21.15.42:0/3145618937 conn(0x7f48381074b0 msgr2=0x7f4838108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.656+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 shutdown_connections 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:21.656+0000 7f4837200700 1 -- 172.21.15.42:0/3145618937 wait complete. 2024-05-27T10:03:22.323 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mgr service... 2024-05-27T10:03:23.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:22 smithi042 ceph-mon[31661]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:23.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:22 smithi042 ceph-mon[31661]: Saving service mon spec with placement count:5 2024-05-27T10:03:23.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:22 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:24.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:24 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:24.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:24 smithi042 ceph-mon[31661]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:03:24.200 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:24 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:24.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.922+0000 7fd438800700 1 Processor -- start 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.922+0000 7fd438800700 1 -- start start 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.922+0000 7fd438800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43007d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.922+0000 7fd438800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd43007d820 con 0x7fd43007ee20 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.923+0000 7fd435e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43007d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.923+0000 7fd435e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43007d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35160/0 (socket says 172.21.15.42:35160) 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.923+0000 7fd435e00700 1 -- 172.21.15.42:0/1998742108 learned_addr learned my addr 172.21.15.42:0/1998742108 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.923+0000 7fd435e00700 1 -- 172.21.15.42:0/1998742108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43007d9a0 con 0x7fd43007ee20 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.924+0000 7fd435e00700 1 --2- 172.21.15.42:0/1998742108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43007d250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4240098a0 tx=0x7fd424009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c339fd9b1c26d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:24.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.924+0000 7fd434a00700 1 -- 172.21.15.42:0/1998742108 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd424004160 con 0x7fd43007ee20 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.924+0000 7fd434a00700 1 -- 172.21.15.42:0/1998742108 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd424004300 con 0x7fd43007ee20 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.924+0000 7fd434a00700 1 -- 172.21.15.42:0/1998742108 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd424011960 con 0x7fd43007ee20 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.925+0000 7fd438800700 1 -- 172.21.15.42:0/1998742108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 msgr2=0x7fd43007d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.925+0000 7fd438800700 1 --2- 172.21.15.42:0/1998742108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43007d250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4240098a0 tx=0x7fd424009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.925+0000 7fd438800700 1 -- 172.21.15.42:0/1998742108 shutdown_connections 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.925+0000 7fd438800700 1 --2- 172.21.15.42:0/1998742108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43007d250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.925+0000 7fd438800700 1 -- 172.21.15.42:0/1998742108 >> 172.21.15.42:0/1998742108 conn(0x7fd4301051c0 msgr2=0x7fd430107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.926+0000 7fd438800700 1 -- 172.21.15.42:0/1998742108 shutdown_connections 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.926+0000 7fd438800700 1 -- 172.21.15.42:0/1998742108 wait complete. 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.927+0000 7fd438800700 1 Processor -- start 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.927+0000 7fd438800700 1 -- start start 2024-05-27T10:03:24.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.927+0000 7fd438800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.927+0000 7fd438800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd430110120 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.928+0000 7fd435e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.928+0000 7fd435e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35162/0 (socket says 172.21.15.42:35162) 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.928+0000 7fd435e00700 1 -- 172.21.15.42:0/3393526382 learned_addr learned my addr 172.21.15.42:0/3393526382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.928+0000 7fd435e00700 1 -- 172.21.15.42:0/3393526382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4301a04e0 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.928+0000 7fd435e00700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43019ffa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd424011670 tx=0x7fd424011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.929+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd424003a60 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.929+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd42401b8d0 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.929+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd424023790 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.929+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4301a0770 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.929+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4301a3380 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.930+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd430064e10 con 0x7fd43007ee20 2024-05-27T10:03:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.930+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd42401b440 con 0x7fd43007ee20 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.931+0000 7fd42aa00700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd41803e580 0x7fd418040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.931+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd424054020 con 0x7fd43007ee20 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.931+0000 7fd435400700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd41803e580 0x7fd418040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.932+0000 7fd435400700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd41803e580 0x7fd418040a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd420006fd0 tx=0x7fd420006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:23.937+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd42401f030 con 0x7fd43007ee20 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.198+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd4301a3610 con 0x7fd41803e580 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.205+0000 7fd42aa00700 1 -- 172.21.15.42:0/3393526382 <== mgr.14120 v2:172.21.15.42:6800/3098170734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd4301a3610 con 0x7fd41803e580 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd41803e580 msgr2=0x7fd418040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd41803e580 0x7fd418040a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd420006fd0 tx=0x7fd420006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 msgr2=0x7fd43019ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43019ffa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd424011670 tx=0x7fd424011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:24.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 shutdown_connections 2024-05-27T10:03:24.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fd41803e580 0x7fd418040a90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:24.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 --2- 172.21.15.42:0/3393526382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43007ee20 0x7fd43019ffa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:24.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 >> 172.21.15.42:0/3393526382 conn(0x7fd4301051c0 msgr2=0x7fd4301059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:24.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 shutdown_connections 2024-05-27T10:03:24.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:24.209+0000 7fd438800700 1 -- 172.21.15.42:0/3393526382 wait complete. 2024-05-27T10:03:25.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:25 smithi042 ceph-mon[31661]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:25.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:25 smithi042 ceph-mon[31661]: Saving service mgr spec with placement count:2 2024-05-27T10:03:25.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:25 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:26.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:26 smithi042 ceph-mon[31661]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:03:26.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:26 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/209819351' entity='client.admin' 2024-05-27T10:03:27.002 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.564+0000 7f2524800700 1 Processor -- start 2024-05-27T10:03:27.002 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.565+0000 7f2524800700 1 -- start start 2024-05-27T10:03:27.002 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.565+0000 7f2524800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c07d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:27.002 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.565+0000 7f2524800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f251c07d770 con 0x7f251c07ee00 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.566+0000 7f2521e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c07d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.566+0000 7f2521e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c07d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35170/0 (socket says 172.21.15.42:35170) 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.566+0000 7f2521e00700 1 -- 172.21.15.42:0/1906558488 learned_addr learned my addr 172.21.15.42:0/1906558488 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.567+0000 7f2521e00700 1 -- 172.21.15.42:0/1906558488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c07d8f0 con 0x7f251c07ee00 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.567+0000 7f2521e00700 1 --2- 172.21.15.42:0/1906558488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c07d230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25140098a0 tx=0x7f2514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a9889a31e8ef51 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.567+0000 7f2520a00700 1 -- 172.21.15.42:0/1906558488 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514004160 con 0x7f251c07ee00 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.567+0000 7f2520a00700 1 -- 172.21.15.42:0/1906558488 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2514004300 con 0x7f251c07ee00 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.567+0000 7f2520a00700 1 -- 172.21.15.42:0/1906558488 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514011960 con 0x7f251c07ee00 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.568+0000 7f2524800700 1 -- 172.21.15.42:0/1906558488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 msgr2=0x7f251c07d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.568+0000 7f2524800700 1 --2- 172.21.15.42:0/1906558488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c07d230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25140098a0 tx=0x7f2514009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.569+0000 7f2524800700 1 -- 172.21.15.42:0/1906558488 shutdown_connections 2024-05-27T10:03:27.003 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.569+0000 7f2524800700 1 --2- 172.21.15.42:0/1906558488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c07d230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.569+0000 7f2524800700 1 -- 172.21.15.42:0/1906558488 >> 172.21.15.42:0/1906558488 conn(0x7f251c1051c0 msgr2=0x7f251c107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.569+0000 7f2524800700 1 -- 172.21.15.42:0/1906558488 shutdown_connections 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.569+0000 7f2524800700 1 -- 172.21.15.42:0/1906558488 wait complete. 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.570+0000 7f2524800700 1 Processor -- start 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.570+0000 7f2524800700 1 -- start start 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.571+0000 7f2524800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.571+0000 7f2524800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1100e0 con 0x7f251c07ee00 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.571+0000 7f2521e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.571+0000 7f2521e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c19ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35178/0 (socket says 172.21.15.42:35178) 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.571+0000 7f2521e00700 1 -- 172.21.15.42:0/209819351 learned_addr learned my addr 172.21.15.42:0/209819351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.571+0000 7f2521e00700 1 -- 172.21.15.42:0/209819351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a0500 con 0x7f251c07ee00 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.572+0000 7f2521e00700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c19ffc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2514011670 tx=0x7f2514011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.572+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514003a60 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.572+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a0790 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.572+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f251401b8d0 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.572+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514023790 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.573+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a33a0 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.574+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2514023a40 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.574+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2500005360 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.575+0000 7f2512a00700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f250803e580 0x7f2508040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.575+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f25140542d0 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.576+0000 7f2521400700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f250803e580 0x7f2508040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.577+0000 7f2521400700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f250803e580 0x7f2508040a90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f250c006fd0 tx=0x7f250c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.585+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f251401f030 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.822+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2500005650 con 0x7f251c07ee00 2024-05-27T10:03:27.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.831+0000 7f2512a00700 1 -- 172.21.15.42:0/209819351 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f251400b3e0 con 0x7f251c07ee00 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f250803e580 msgr2=0x7f2508040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f250803e580 0x7f2508040a90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f250c006fd0 tx=0x7f250c006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 msgr2=0x7f251c19ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c19ffc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2514011670 tx=0x7f2514011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 shutdown_connections 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f250803e580 0x7f2508040a90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 --2- 172.21.15.42:0/209819351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f251c07ee00 0x7f251c19ffc0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2514011670 tx=0x7f2514011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.834+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 >> 172.21.15.42:0/209819351 conn(0x7f251c1051c0 msgr2=0x7f251c105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.835+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 shutdown_connections 2024-05-27T10:03:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:25.835+0000 7f2524800700 1 -- 172.21.15.42:0/209819351 wait complete. 2024-05-27T10:03:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:28 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:28 smithi042 ceph-mon[31661]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:03:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:28 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2189413232' entity='client.admin' 2024-05-27T10:03:28.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.662+0000 7f323ce00700 1 Processor -- start 2024-05-27T10:03:28.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.662+0000 7f323ce00700 1 -- start start 2024-05-27T10:03:28.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.663+0000 7f323ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f3238109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.663+0000 7f323ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f323810a520 con 0x7f3238109b20 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.663+0000 7f3235e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f3238109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.664+0000 7f3235e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f3238109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35184/0 (socket says 172.21.15.42:35184) 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.664+0000 7f3235e00700 1 -- 172.21.15.42:0/2241559570 learned_addr learned my addr 172.21.15.42:0/2241559570 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.664+0000 7f3235e00700 1 -- 172.21.15.42:0/2241559570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323810acf0 con 0x7f3238109b20 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.664+0000 7f3235e00700 1 --2- 172.21.15.42:0/2241559570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f3238109f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3228009e80 tx=0x7f322800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd061b8943de2154 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.665+0000 7f3234a00700 1 -- 172.21.15.42:0/2241559570 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228017070 con 0x7f3238109b20 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.665+0000 7f3234a00700 1 -- 172.21.15.42:0/2241559570 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f322800b8c0 con 0x7f3238109b20 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.665+0000 7f3234a00700 1 -- 172.21.15.42:0/2241559570 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228004420 con 0x7f3238109b20 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 -- 172.21.15.42:0/2241559570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 msgr2=0x7f3238109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 --2- 172.21.15.42:0/2241559570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f3238109f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3228009e80 tx=0x7f322800b330 comp rx=0 tx=0).stop 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 -- 172.21.15.42:0/2241559570 shutdown_connections 2024-05-27T10:03:28.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 --2- 172.21.15.42:0/2241559570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f3238109f30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 -- 172.21.15.42:0/2241559570 >> 172.21.15.42:0/2241559570 conn(0x7f32381051c0 msgr2=0x7f3238107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 -- 172.21.15.42:0/2241559570 shutdown_connections 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.666+0000 7f323ce00700 1 -- 172.21.15.42:0/2241559570 wait complete. 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.667+0000 7f323ce00700 1 Processor -- start 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.668+0000 7f323ce00700 1 -- start start 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.668+0000 7f323ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f32381a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.668+0000 7f323ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f323810e0a0 con 0x7f3238109b20 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.668+0000 7f3235e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f32381a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.668+0000 7f3235e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f32381a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35196/0 (socket says 172.21.15.42:35196) 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.668+0000 7f3235e00700 1 -- 172.21.15.42:0/2189413232 learned_addr learned my addr 172.21.15.42:0/2189413232 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.669+0000 7f3235e00700 1 -- 172.21.15.42:0/2189413232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32381a05a0 con 0x7f3238109b20 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.669+0000 7f3235e00700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f32381a0060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3228015040 tx=0x7f322800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:28.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.669+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228017040 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.669+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3228022070 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.669+0000 7f323ce00700 1 -- 172.21.15.42:0/2189413232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32381a0830 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.669+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32280125c0 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.670+0000 7f323ce00700 1 -- 172.21.15.42:0/2189413232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32381a3440 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.671+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3238064e10 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.672+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f322801d440 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.672+0000 7f322ea00700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f321c03e580 0x7f321c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.672+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f322800f050 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.673+0000 7f3235400700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f321c03e580 0x7f321c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.673+0000 7f3235400700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f321c03e580 0x7f321c040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3224006fd0 tx=0x7f3224006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.678+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32280263c0 con 0x7f3238109b20 2024-05-27T10:03:28.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.984+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f32380643c0 con 0x7f3238109b20 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.988+0000 7f322ea00700 1 -- 172.21.15.42:0/2189413232 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f32280534f0 con 0x7f3238109b20 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f321c03e580 msgr2=0x7f321c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f321c03e580 0x7f321c040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3224006fd0 tx=0x7f3224006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 msgr2=0x7f32381a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f32381a0060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3228015040 tx=0x7f322800bd50 comp rx=0 tx=0).stop 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 shutdown_connections 2024-05-27T10:03:28.695 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f321c03e580 0x7f321c040a90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:28.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 --2- 172.21.15.42:0/2189413232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238109b20 0x7f32381a0060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:28.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.991+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 >> 172.21.15.42:0/2189413232 conn(0x7f32381051c0 msgr2=0x7f323807d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:28.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.992+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 shutdown_connections 2024-05-27T10:03:28.696 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:27.992+0000 7f3223e00700 1 -- 172.21.15.42:0/2189413232 wait complete. 2024-05-27T10:03:28.696 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2024-05-27T10:03:30.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:30 smithi042 ceph-mon[31661]: from='mgr.14120 172.21.15.42:0/792007310' entity='mgr.x' 2024-05-27T10:03:30.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:30 smithi042 ceph-mon[31661]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:03:30.489 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:30 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/107534677' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-27T10:03:30.756 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: ignoring --setuser ceph since I am not root 2024-05-27T10:03:30.756 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: ignoring --setgroup ceph since I am not root 2024-05-27T10:03:30.756 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:30.754+0000 7f19b1197200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:03:31.327 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:31.015+0000 7f19b1197200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:03:31.327 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:31.152+0000 7f19b1197200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:03:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:31 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/107534677' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-27T10:03:31.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:31 smithi042 ceph-mon[31661]: mgrmap e9: x(active, since 26s) 2024-05-27T10:03:31.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:31.572+0000 7f19b1197200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:03:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.376+0000 7fc5eae00700 1 Processor -- start 2024-05-27T10:03:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.376+0000 7fc5eae00700 1 -- start start 2024-05-27T10:03:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.377+0000 7fc5eae00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e4109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.377+0000 7fc5eae00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410a520 con 0x7fc5e4109b20 2024-05-27T10:03:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.377+0000 7fc5e3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e4109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.377+0000 7fc5e3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e4109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35202/0 (socket says 172.21.15.42:35202) 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.377+0000 7fc5e3e00700 1 -- 172.21.15.42:0/3092301434 learned_addr learned my addr 172.21.15.42:0/3092301434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.378+0000 7fc5e3e00700 1 -- 172.21.15.42:0/3092301434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e410acf0 con 0x7fc5e4109b20 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.378+0000 7fc5e3e00700 1 --2- 172.21.15.42:0/3092301434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e4109f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40098a0 tx=0x7fc5d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31651e2285a71370 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.378+0000 7fc5e2a00700 1 -- 172.21.15.42:0/3092301434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5d4004160 con 0x7fc5e4109b20 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.378+0000 7fc5e2a00700 1 -- 172.21.15.42:0/3092301434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d4004300 con 0x7fc5e4109b20 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.378+0000 7fc5e2a00700 1 -- 172.21.15.42:0/3092301434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5d4011960 con 0x7fc5e4109b20 2024-05-27T10:03:31.640 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 -- 172.21.15.42:0/3092301434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 msgr2=0x7fc5e4109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:31.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 --2- 172.21.15.42:0/3092301434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e4109f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40098a0 tx=0x7fc5d4009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:31.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 -- 172.21.15.42:0/3092301434 shutdown_connections 2024-05-27T10:03:31.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 --2- 172.21.15.42:0/3092301434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e4109f30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:31.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 -- 172.21.15.42:0/3092301434 >> 172.21.15.42:0/3092301434 conn(0x7fc5e41051c0 msgr2=0x7fc5e4107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:31.644 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 -- 172.21.15.42:0/3092301434 shutdown_connections 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.379+0000 7fc5eae00700 1 -- 172.21.15.42:0/3092301434 wait complete. 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.381+0000 7fc5eae00700 1 Processor -- start 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.381+0000 7fc5eae00700 1 -- start start 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.381+0000 7fc5eae00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.381+0000 7fc5eae00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410e0a0 con 0x7fc5e4109b20 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.382+0000 7fc5e3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.382+0000 7fc5e3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35216/0 (socket says 172.21.15.42:35216) 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.382+0000 7fc5e3e00700 1 -- 172.21.15.42:0/107534677 learned_addr learned my addr 172.21.15.42:0/107534677 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.382+0000 7fc5e3e00700 1 -- 172.21.15.42:0/107534677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e419e340 con 0x7fc5e4109b20 2024-05-27T10:03:31.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.382+0000 7fc5e3e00700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e419de00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4006af0 tx=0x7fc5d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.383+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5d4003af0 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.383+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e419e5d0 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.383+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d401b8d0 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.383+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5d4023820 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.383+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41a11e0 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.385+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc5d401b440 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.385+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e4064e10 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.385+0000 7fc5e0c00700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc5c803e580 0x7fc5c8040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.386+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc5d4053be0 con 0x7fc5e4109b20 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.386+0000 7fc5e3400700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc5c803e580 0x7fc5c8040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.387+0000 7fc5e3400700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc5c803e580 0x7fc5c8040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0006fd0 tx=0x7fc5d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.391+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5d401f030 con 0x7fc5e4109b20 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:29.688+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc5e41a1820 con 0x7fc5e4109b20 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.352+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc5d400b3e0 con 0x7fc5e4109b20 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.357+0000 7fc5e0c00700 1 -- 172.21.15.42:0/107534677 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fc5d404c150 con 0x7fc5e4109b20 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.357+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc5c803e580 msgr2=0x7fc5c8040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.357+0000 7fc5eae00700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc5c803e580 0x7fc5c8040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0006fd0 tx=0x7fc5d0006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.357+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 msgr2=0x7fc5e419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.357+0000 7fc5eae00700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e419de00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4006af0 tx=0x7fc5d4011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.358+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 shutdown_connections 2024-05-27T10:03:31.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.358+0000 7fc5eae00700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc5c803e580 0x7fc5c8040a90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:31.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.358+0000 7fc5eae00700 1 --2- 172.21.15.42:0/107534677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109b20 0x7fc5e419de00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:31.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.358+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 >> 172.21.15.42:0/107534677 conn(0x7fc5e41051c0 msgr2=0x7fc5e4105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:31.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.358+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 shutdown_connections 2024-05-27T10:03:31.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:30.358+0000 7fc5eae00700 1 -- 172.21.15.42:0/107534677 wait complete. 2024-05-27T10:03:31.910 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:31.701+0000 7f19b1197200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:03:33.579 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:33 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/136129374' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T10:03:33.579 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:33.336+0000 7f19b1197200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:03:33.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:33.578+0000 7f19b1197200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:03:33.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:33.804+0000 7f19b1197200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.995+0000 7f42b0e00700 1 Processor -- start 2024-05-27T10:03:34.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.995+0000 7f42b0e00700 1 -- start start 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.996+0000 7f42b0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac10c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.996+0000 7f42b0e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac07ce70 con 0x7f42ac10a580 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.997+0000 7f42a9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac10c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.997+0000 7f42a9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac10c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57116/0 (socket says 172.21.15.42:57116) 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.997+0000 7f42a9e00700 1 -- 172.21.15.42:0/1104661852 learned_addr learned my addr 172.21.15.42:0/1104661852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.997+0000 7f42a9e00700 1 -- 172.21.15.42:0/1104661852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac10cee0 con 0x7f42ac10a580 2024-05-27T10:03:34.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.997+0000 7f42a9e00700 1 --2- 172.21.15.42:0/1104661852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac10c9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098a0 tx=0x7f429c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955903b3b8a292c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:34.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.998+0000 7f42a8a00700 1 -- 172.21.15.42:0/1104661852 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c004160 con 0x7f42ac10a580 2024-05-27T10:03:34.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.998+0000 7f42a8a00700 1 -- 172.21.15.42:0/1104661852 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f429c004300 con 0x7f42ac10a580 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.998+0000 7f42a8a00700 1 -- 172.21.15.42:0/1104661852 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c011960 con 0x7f42ac10a580 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 -- 172.21.15.42:0/1104661852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 msgr2=0x7f42ac10c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 --2- 172.21.15.42:0/1104661852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac10c9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098a0 tx=0x7f429c009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 -- 172.21.15.42:0/1104661852 shutdown_connections 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 --2- 172.21.15.42:0/1104661852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac10c9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 -- 172.21.15.42:0/1104661852 >> 172.21.15.42:0/1104661852 conn(0x7f42ac104080 msgr2=0x7f42ac1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 -- 172.21.15.42:0/1104661852 shutdown_connections 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:32.999+0000 7f42b0e00700 1 -- 172.21.15.42:0/1104661852 wait complete. 2024-05-27T10:03:34.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.000+0000 7f42b0e00700 1 Processor -- start 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.001+0000 7f42b0e00700 1 -- start start 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.001+0000 7f42b0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.001+0000 7f42b0e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10de80 con 0x7f42ac10a580 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.001+0000 7f42a9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.001+0000 7f42a9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57132/0 (socket says 172.21.15.42:57132) 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.001+0000 7f42a9e00700 1 -- 172.21.15.42:0/136129374 learned_addr learned my addr 172.21.15.42:0/136129374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.002+0000 7f42a9e00700 1 -- 172.21.15.42:0/136129374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac10a240 con 0x7f42ac10a580 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.002+0000 7f42a9e00700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac109d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f429c011670 tx=0x7f429c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:34.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.002+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c01f070 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.002+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f429c003b20 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.002+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac108490 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.002+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c019580 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.003+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac108970 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.004+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c005360 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.004+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f429c022070 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.005+0000 7f42a2a00700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 0x7f4290040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.005+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f429c053af0 con 0x7f42ac10a580 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.005+0000 7f42a9400700 1 -- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 msgr2=0x7f4290040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:34.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.005+0000 7f42a9400700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 0x7f4290040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.011+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f429c0172b0 con 0x7f42ac10a580 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.206+0000 7f42a9400700 1 -- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 msgr2=0x7f4290040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.206+0000 7f42a9400700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 0x7f4290040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.308+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f428c005e60 con 0x7f42ac10a580 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.309+0000 7f42a2a00700 1 -- 172.21.15.42:0/136129374 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f429c019de0 con 0x7f42ac10a580 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.313+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 msgr2=0x7f4290040af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.313+0000 7f42b0e00700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 0x7f4290040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.313+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 msgr2=0x7f42ac109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac109d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f429c011670 tx=0x7f429c011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:34.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 shutdown_connections 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7f429003e5e0 0x7f4290040af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 --2- 172.21.15.42:0/136129374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac10a580 0x7f42ac109d00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 >> 172.21.15.42:0/136129374 conn(0x7f42ac104080 msgr2=0x7f42ac105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 shutdown_connections 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:33.314+0000 7f42b0e00700 1 -- 172.21.15.42:0/136129374 wait complete. 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-05-27T10:03:34.040 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2024-05-27T10:03:34.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:34.224+0000 7f19b1197200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:03:34.629 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:34.358+0000 7f19b1197200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:03:34.629 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:34.626+0000 7f19b1197200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:03:34.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:34.902+0000 7f19b1197200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:03:36.036 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:35.729+0000 7f19b1197200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:03:36.037 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:36.034+0000 7f19b1197200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:03:37.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:36.620+0000 7f19b1197200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:03:37.094 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:36.755+0000 7f19b1197200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:03:37.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:37.239+0000 7f19b1197200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:03:38.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:37.717+0000 7f19b1197200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:03:38.094 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:37.850+0000 7f19b1197200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:03:38.519 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:38.190+0000 7f19b1197200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:03:38.519 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:38.380+0000 7f19b1197200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:03:38.519 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:38.517+0000 7f19b1197200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:03:39.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:38.931+0000 7f19b1197200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:03:39.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:39.064+0000 7f19b1197200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:03:40.176 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:39.870+0000 7f19b1197200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:03:40.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:03:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:03:40.175+0000 7f19b1197200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:03:40.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: Active manager daemon x restarted 2024-05-27T10:03:40.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: Activating manager daemon x 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: osdmap e3: 0 total, 0 up, 0 in 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: mgrmap e10: x(active, starting, since 0.00621944s) 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T10:03:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:40 smithi042 ceph-mon[31661]: Manager daemon x is now available 2024-05-27T10:03:41.558 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:03:41.558 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T10:03:41.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T10:03:41.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:41 smithi042 ceph-mon[31661]: mgrmap e11: x(active, since 1.01016s) 2024-05-27T10:03:41.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-05-27T10:03:41.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-27T10:03:41.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T10:03:41.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.782+0000 7fc56e200700 1 Processor -- start 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.782+0000 7fc56e200700 1 -- start start 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.783+0000 7fc56e200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56807d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.783+0000 7fc56e200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc56807d820 con 0x7fc56807ee20 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.783+0000 7fc567400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56807d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.783+0000 7fc567400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56807d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57138/0 (socket says 172.21.15.42:57138) 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.783+0000 7fc567400700 1 -- 172.21.15.42:0/227229630 learned_addr learned my addr 172.21.15.42:0/227229630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:41.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.783+0000 7fc567400700 1 -- 172.21.15.42:0/227229630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56807d9a0 con 0x7fc56807ee20 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.784+0000 7fc567400700 1 --2- 172.21.15.42:0/227229630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56807d250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5580098a0 tx=0x7fc558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69259973a26629fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.784+0000 7fc566000700 1 -- 172.21.15.42:0/227229630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc558004160 con 0x7fc56807ee20 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.784+0000 7fc566000700 1 -- 172.21.15.42:0/227229630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc558004300 con 0x7fc56807ee20 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.784+0000 7fc566000700 1 -- 172.21.15.42:0/227229630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc558011960 con 0x7fc56807ee20 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 -- 172.21.15.42:0/227229630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 msgr2=0x7fc56807d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 --2- 172.21.15.42:0/227229630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56807d250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5580098a0 tx=0x7fc558009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 -- 172.21.15.42:0/227229630 shutdown_connections 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 --2- 172.21.15.42:0/227229630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56807d250 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 -- 172.21.15.42:0/227229630 >> 172.21.15.42:0/227229630 conn(0x7fc5681050c0 msgr2=0x7fc568107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 -- 172.21.15.42:0/227229630 shutdown_connections 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.785+0000 7fc56e200700 1 -- 172.21.15.42:0/227229630 wait complete. 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.787+0000 7fc56e200700 1 Processor -- start 2024-05-27T10:03:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.787+0000 7fc56e200700 1 -- start start 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.787+0000 7fc56e200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56819ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.787+0000 7fc56e200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681100a0 con 0x7fc56807ee20 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.787+0000 7fc567400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56819ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.788+0000 7fc567400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56819ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57146/0 (socket says 172.21.15.42:57146) 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.788+0000 7fc567400700 1 -- 172.21.15.42:0/1504359830 learned_addr learned my addr 172.21.15.42:0/1504359830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.788+0000 7fc567400700 1 -- 172.21.15.42:0/1504359830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5681a04d0 con 0x7fc56807ee20 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.788+0000 7fc567400700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56819ff90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc558011670 tx=0x7fc558011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.788+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc558003a60 con 0x7fc56807ee20 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.789+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5681a0760 con 0x7fc56807ee20 2024-05-27T10:03:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.789+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc55801b8d0 con 0x7fc56807ee20 2024-05-27T10:03:41.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.789+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc558023790 con 0x7fc56807ee20 2024-05-27T10:03:41.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.789+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5681a3370 con 0x7fc56807ee20 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.790+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fc558023a90 con 0x7fc56807ee20 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.791+0000 7fc55fe00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.791+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 --> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc54c0411a0 con 0x7fc54c03e580 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.791+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc558054100 con 0x7fc56807ee20 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.791+0000 7fc566a00700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 msgr2=0x7fc54c040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.791+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.992+0000 7fc566a00700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 msgr2=0x7fc54c040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:34.992+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:35.393+0000 7fc566a00700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 msgr2=0x7fc54c040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:35.393+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T10:03:41.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:36.194+0000 7fc566a00700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 msgr2=0x7fc54c040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:36.194+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:37.796+0000 7fc566a00700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 msgr2=0x7fc54c040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3098170734 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:37.796+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:40.183+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fc55802c3f0 con 0x7fc56807ee20 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:40.183+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 msgr2=0x7fc54c040a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:40.184+0000 7fc55fe00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.186+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fc5580559b0 con 0x7fc56807ee20 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.186+0000 7fc55fe00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc558053c90 0x7fc54c043fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.186+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc54c0411a0 con 0x7fc558053c90 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.187+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc558053c90 0x7fc54c043fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.188+0000 7fc566a00700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc558053c90 0x7fc54c043fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc554003a10 tx=0x7fc5540092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:41.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.189+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc54c0411a0 con 0x7fc558053c90 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.199+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc568064e10 con 0x7fc558053c90 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.199+0000 7fc55fe00700 1 -- 172.21.15.42:0/1504359830 <== mgr.14152 v2:172.21.15.42:6800/2062230330 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc568064e10 con 0x7fc558053c90 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc558053c90 msgr2=0x7fc54c043fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc558053c90 0x7fc54c043fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc554003a10 tx=0x7fc5540092b0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 msgr2=0x7fc56819ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56819ff90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc558011670 tx=0x7fc558011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 shutdown_connections 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc558053c90 0x7fc54c043fc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc56807ee20 0x7fc56819ff90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 --2- 172.21.15.42:0/1504359830 >> [v2:172.21.15.42:6800/3098170734,v1:172.21.15.42:6801/3098170734] conn(0x7fc54c03e580 0x7fc54c040a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 >> 172.21.15.42:0/1504359830 conn(0x7fc5681050c0 msgr2=0x7fc5681059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 shutdown_connections 2024-05-27T10:03:41.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:41.200+0000 7fc56e200700 1 -- 172.21.15.42:0/1504359830 wait complete. 2024-05-27T10:03:41.894 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2024-05-27T10:03:41.894 INFO:teuthology.orchestra.run.smithi042.stdout:Generating a dashboard self-signed certificate... 2024-05-27T10:03:42.514 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:42 smithi042 ceph-mon[31661]: [27/May/2024:10:03:40] ENGINE Bus STARTING 2024-05-27T10:03:42.514 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:42 smithi042 ceph-mon[31661]: [27/May/2024:10:03:40] ENGINE Serving on https://172.21.15.42:7150 2024-05-27T10:03:42.514 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:42 smithi042 ceph-mon[31661]: [27/May/2024:10:03:40] ENGINE Serving on http://172.21.15.42:8765 2024-05-27T10:03:42.514 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:42 smithi042 ceph-mon[31661]: [27/May/2024:10:03:40] ENGINE Bus STARTED 2024-05-27T10:03:42.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:42 smithi042 ceph-mon[31661]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T10:03:42.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:42 smithi042 ceph-mon[31661]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T10:03:43.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:43 smithi042 ceph-mon[31661]: mgrmap e12: x(active, since 2s) 2024-05-27T10:03:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:03:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:03:43.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.670+0000 7f508ce00700 1 Processor -- start 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.670+0000 7f508ce00700 1 -- start start 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.671+0000 7f508ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508807d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.671+0000 7f508ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f508807d770 con 0x7f508807ee00 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.671+0000 7f5087400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508807d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.672+0000 7f5087400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508807d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57194/0 (socket says 172.21.15.42:57194) 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.672+0000 7f5087400700 1 -- 172.21.15.42:0/2869181426 learned_addr learned my addr 172.21.15.42:0/2869181426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.672+0000 7f5087400700 1 -- 172.21.15.42:0/2869181426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508807d8f0 con 0x7f508807ee00 2024-05-27T10:03:43.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.672+0000 7f5087400700 1 --2- 172.21.15.42:0/2869181426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508807d230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f50780098a0 tx=0x7f5078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7a6ed920207b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.673+0000 7f5086a00700 1 -- 172.21.15.42:0/2869181426 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5078004160 con 0x7f508807ee00 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.673+0000 7f5086a00700 1 -- 172.21.15.42:0/2869181426 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5078004300 con 0x7f508807ee00 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.673+0000 7f5086a00700 1 -- 172.21.15.42:0/2869181426 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5078011960 con 0x7f508807ee00 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 -- 172.21.15.42:0/2869181426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 msgr2=0x7f508807d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 --2- 172.21.15.42:0/2869181426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508807d230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f50780098a0 tx=0x7f5078009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 -- 172.21.15.42:0/2869181426 shutdown_connections 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 --2- 172.21.15.42:0/2869181426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508807d230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 -- 172.21.15.42:0/2869181426 >> 172.21.15.42:0/2869181426 conn(0x7f5088104f90 msgr2=0x7f5088107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 -- 172.21.15.42:0/2869181426 shutdown_connections 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.674+0000 7f508ce00700 1 -- 172.21.15.42:0/2869181426 wait complete. 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.675+0000 7f508ce00700 1 Processor -- start 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.675+0000 7f508ce00700 1 -- start start 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.676+0000 7f508ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.676+0000 7f508ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f508810ff10 con 0x7f508807ee00 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.676+0000 7f5087400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.676+0000 7f5087400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57210/0 (socket says 172.21.15.42:57210) 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.676+0000 7f5087400700 1 -- 172.21.15.42:0/2684913722 learned_addr learned my addr 172.21.15.42:0/2684913722 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.676+0000 7f5087400700 1 -- 172.21.15.42:0/2684913722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50881a0290 con 0x7f508807ee00 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.677+0000 7f5087400700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508819fd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5078011670 tx=0x7f5078011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.677+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5078003aa0 con 0x7f508807ee00 2024-05-27T10:03:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.677+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f507801b8d0 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.677+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50780197f0 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.677+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881a0520 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.677+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a3120 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.678+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5088064e10 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.679+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f507801b440 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.679+0000 7f5084c00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f506c03e530 0x7f506c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.679+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50780540a0 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.680+0000 7f507fe00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f506c03e530 0x7f506c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.680+0000 7f507fe00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f506c03e530 0x7f506c040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5070006fd0 tx=0x7f5070006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.685+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f507801f030 con 0x7f508807ee00 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:42.933+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5088105ea0 con 0x7f506c03e530 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.118+0000 7f5084c00700 1 -- 172.21.15.42:0/2684913722 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5088105ea0 con 0x7f506c03e530 2024-05-27T10:03:43.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f506c03e530 msgr2=0x7f506c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f506c03e530 0x7f506c040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5070006fd0 tx=0x7f5070006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 msgr2=0x7f508819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508819fd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5078011670 tx=0x7f5078011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 shutdown_connections 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f506c03e530 0x7f506c040a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 --2- 172.21.15.42:0/2684913722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f508807ee00 0x7f508819fd50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.123+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 >> 172.21.15.42:0/2684913722 conn(0x7f5088104f90 msgr2=0x7f5088105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.124+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 shutdown_connections 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:43.124+0000 7f508ce00700 1 -- 172.21.15.42:0/2684913722 wait complete. 2024-05-27T10:03:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2024-05-27T10:03:44.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:44 smithi042 ceph-mon[31661]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:44.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$2m14KzhJ291iRqVSvk5hR.keBkxJdrG6e8ncoBfmE/28rY/7Fuf42", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716804225, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.679+0000 7f82a2a00700 1 Processor -- start 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.679+0000 7f82a2a00700 1 -- start start 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.680+0000 7f82a2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c109ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.680+0000 7f82a2a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10a4b0 con 0x7f829c109ab0 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.680+0000 7f829be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c109ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.680+0000 7f829be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c109ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49216/0 (socket says 172.21.15.42:49216) 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.680+0000 7f829be00700 1 -- 172.21.15.42:0/4220144637 learned_addr learned my addr 172.21.15.42:0/4220144637 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:46.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.681+0000 7f829be00700 1 -- 172.21.15.42:0/4220144637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c10ac80 con 0x7f829c109ab0 2024-05-27T10:03:46.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.681+0000 7f829be00700 1 --2- 172.21.15.42:0/4220144637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c109ec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098a0 tx=0x7f828c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f1c4805972f505 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:46.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.682+0000 7f829aa00700 1 -- 172.21.15.42:0/4220144637 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f828c004160 con 0x7f829c109ab0 2024-05-27T10:03:46.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.682+0000 7f829aa00700 1 -- 172.21.15.42:0/4220144637 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f828c004300 con 0x7f829c109ab0 2024-05-27T10:03:46.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.682+0000 7f829aa00700 1 -- 172.21.15.42:0/4220144637 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f828c011960 con 0x7f829c109ab0 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.682+0000 7f82a2a00700 1 -- 172.21.15.42:0/4220144637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 msgr2=0x7f829c109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.682+0000 7f82a2a00700 1 --2- 172.21.15.42:0/4220144637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c109ec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098a0 tx=0x7f828c009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.683+0000 7f82a2a00700 1 -- 172.21.15.42:0/4220144637 shutdown_connections 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.683+0000 7f82a2a00700 1 --2- 172.21.15.42:0/4220144637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c109ec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.683+0000 7f82a2a00700 1 -- 172.21.15.42:0/4220144637 >> 172.21.15.42:0/4220144637 conn(0x7f829c105130 msgr2=0x7f829c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.683+0000 7f82a2a00700 1 -- 172.21.15.42:0/4220144637 shutdown_connections 2024-05-27T10:03:46.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.683+0000 7f82a2a00700 1 -- 172.21.15.42:0/4220144637 wait complete. 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.684+0000 7f82a2a00700 1 Processor -- start 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.685+0000 7f82a2a00700 1 -- start start 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.685+0000 7f82a2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c07f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.685+0000 7f82a2a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10e030 con 0x7f829c109ab0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.685+0000 7f829be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c07f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.685+0000 7f829be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c07f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49226/0 (socket says 172.21.15.42:49226) 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.685+0000 7f829be00700 1 -- 172.21.15.42:0/1896680268 learned_addr learned my addr 172.21.15.42:0/1896680268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.686+0000 7f829be00700 1 -- 172.21.15.42:0/1896680268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c07f9d0 con 0x7f829c109ab0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.686+0000 7f829be00700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c07f490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f828c011980 tx=0x7f828c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.686+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f828c01f070 con 0x7f829c109ab0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.687+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f828c003aa0 con 0x7f829c109ab0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.687+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f828c019560 con 0x7f829c109ab0 2024-05-27T10:03:46.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.687+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c07dc20 con 0x7f829c109ab0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.687+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c07e100 con 0x7f829c109ab0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.688+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f828c022070 con 0x7f829c109ab0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.688+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c064e10 con 0x7f829c109ab0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.689+0000 7f8298c00700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f828003e5d0 0x7f8280040ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.689+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f828c053a90 con 0x7f829c109ab0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.689+0000 7f829b400700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f828003e5d0 0x7f8280040ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.690+0000 7f829b400700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f828003e5d0 0x7f8280040ae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8288006fd0 tx=0x7f8288006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.695+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f828c0172b0 con 0x7f829c109ab0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:44.956+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f829c07e7b0 con 0x7f828003e5d0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.218+0000 7f8298c00700 1 -- 172.21.15.42:0/1896680268 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f829c07e7b0 con 0x7f828003e5d0 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f828003e5d0 msgr2=0x7f8280040ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f828003e5d0 0x7f8280040ae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8288006fd0 tx=0x7f8288006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:46.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 msgr2=0x7f829c07f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c07f490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f828c011980 tx=0x7f828c011a60 comp rx=0 tx=0).stop 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 shutdown_connections 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f828003e5d0 0x7f8280040ae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 --2- 172.21.15.42:0/1896680268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f829c109ab0 0x7f829c07f490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.221+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 >> 172.21.15.42:0/1896680268 conn(0x7f829c105130 msgr2=0x7f829c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.222+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 shutdown_connections 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:45.222+0000 7f82a2a00700 1 -- 172.21.15.42:0/1896680268 wait complete. 2024-05-27T10:03:46.061 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2024-05-27T10:03:46.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:46 smithi042 ceph-mon[31661]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:03:46.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:46 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:03:46.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:46 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:03:48.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:47 smithi042 ceph-mon[31661]: mgrmap e13: x(active, since 6s) 2024-05-27T10:03:48.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:47 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/576199594' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-27T10:03:48.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.762+0000 7fa13c600700 1 Processor -- start 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.762+0000 7fa13c600700 1 -- start start 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.763+0000 7fa13c600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13407d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.763+0000 7fa13c600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa13407d820 con 0x7fa13407ee20 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.763+0000 7fa139c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13407d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.763+0000 7fa139c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13407d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49228/0 (socket says 172.21.15.42:49228) 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.763+0000 7fa139c00700 1 -- 172.21.15.42:0/2807128608 learned_addr learned my addr 172.21.15.42:0/2807128608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.764+0000 7fa139c00700 1 -- 172.21.15.42:0/2807128608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13407d9a0 con 0x7fa13407ee20 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.764+0000 7fa139c00700 1 --2- 172.21.15.42:0/2807128608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13407d250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa1280098a0 tx=0x7fa128009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dadb80719aa41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.764+0000 7fa133e00700 1 -- 172.21.15.42:0/2807128608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128004160 con 0x7fa13407ee20 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.764+0000 7fa133e00700 1 -- 172.21.15.42:0/2807128608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa128004300 con 0x7fa13407ee20 2024-05-27T10:03:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.764+0000 7fa133e00700 1 -- 172.21.15.42:0/2807128608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128011960 con 0x7fa13407ee20 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.765+0000 7fa13c600700 1 -- 172.21.15.42:0/2807128608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 msgr2=0x7fa13407d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.765+0000 7fa13c600700 1 --2- 172.21.15.42:0/2807128608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13407d250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa1280098a0 tx=0x7fa128009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.765+0000 7fa13c600700 1 -- 172.21.15.42:0/2807128608 shutdown_connections 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.765+0000 7fa13c600700 1 --2- 172.21.15.42:0/2807128608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13407d250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.765+0000 7fa13c600700 1 -- 172.21.15.42:0/2807128608 >> 172.21.15.42:0/2807128608 conn(0x7fa1341051c0 msgr2=0x7fa134107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.766+0000 7fa13c600700 1 -- 172.21.15.42:0/2807128608 shutdown_connections 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.766+0000 7fa13c600700 1 -- 172.21.15.42:0/2807128608 wait complete. 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa13c600700 1 Processor -- start 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa13c600700 1 -- start start 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa13c600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13410cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa13c600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa134110120 con 0x7fa13407ee20 2024-05-27T10:03:48.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa139c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13410cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa139c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13410cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49232/0 (socket says 172.21.15.42:49232) 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.767+0000 7fa139c00700 1 -- 172.21.15.42:0/576199594 learned_addr learned my addr 172.21.15.42:0/576199594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.768+0000 7fa139c00700 1 -- 172.21.15.42:0/576199594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13410d220 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.768+0000 7fa139c00700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13410cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa128011670 tx=0x7fa128011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.768+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128003a60 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.768+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13410d4b0 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.768+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa12801b8d0 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.768+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128023790 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.769+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa134109900 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.770+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fa128023a90 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.770+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa134064e10 con 0x7fa13407ee20 2024-05-27T10:03:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.770+0000 7fa132000700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa11c03e580 0x7fa11c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.770+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1280540c0 con 0x7fa13407ee20 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.771+0000 7fa139200700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa11c03e580 0x7fa11c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.772+0000 7fa139200700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa11c03e580 0x7fa11c040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.776+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa12801f030 con 0x7fa13407ee20 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:46.787+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa12802c3f0 con 0x7fa13407ee20 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.010+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa13410a060 con 0x7fa13407ee20 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.011+0000 7fa132000700 1 -- 172.21.15.42:0/576199594 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa12800b3e0 con 0x7fa13407ee20 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa11c03e580 msgr2=0x7fa11c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa11c03e580 0x7fa11c040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 msgr2=0x7fa13410cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13410cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa128011670 tx=0x7fa128011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 shutdown_connections 2024-05-27T10:03:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa11c03e580 0x7fa11c040a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 --2- 172.21.15.42:0/576199594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa13407ee20 0x7fa13410cce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 >> 172.21.15.42:0/576199594 conn(0x7fa1341051c0 msgr2=0x7fa1341059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 shutdown_connections 2024-05-27T10:03:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:47.014+0000 7fa13c600700 1 -- 172.21.15.42:0/576199594 wait complete. 2024-05-27T10:03:48.230 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T10:03:48.230 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-05-27T10:03:48.252 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T10:03:48.252 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-05-27T10:03:48.252 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-05-27T10:03:48.252 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-27T10:03:48.255 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2024-05-27T10:03:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042.front.sepia.ceph.com:8443/ 2024-05-27T10:03:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2024-05-27T10:03:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: Password: bc56ukj5sz 2024-05-27T10:03:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:48.256 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config directory 2024-05-27T10:03:48.257 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling autotune for osd_memory_target 2024-05-27T10:03:50.430 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.392+0000 7f74a6c00700 1 Processor -- start 2024-05-27T10:03:50.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.392+0000 7f74a6c00700 1 -- start start 2024-05-27T10:03:50.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.392+0000 7f74a6c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a0108800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:50.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.392+0000 7f74a6c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74a007cdf0 con 0x7f74a01083f0 2024-05-27T10:03:50.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.393+0000 7f749fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a0108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:50.431 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.393+0000 7f749fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a0108800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49236/0 (socket says 172.21.15.42:49236) 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.393+0000 7f749fe00700 1 -- 172.21.15.42:0/1702867021 learned_addr learned my addr 172.21.15.42:0/1702867021 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.394+0000 7f749fe00700 1 -- 172.21.15.42:0/1702867021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a0108d40 con 0x7f74a01083f0 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.394+0000 7f749fe00700 1 --2- 172.21.15.42:0/1702867021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a0108800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f74900098a0 tx=0x7f7490009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fdd852b713baa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.394+0000 7f749f400700 1 -- 172.21.15.42:0/1702867021 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7490004160 con 0x7f74a01083f0 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.394+0000 7f749f400700 1 -- 172.21.15.42:0/1702867021 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7490004300 con 0x7f74a01083f0 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.394+0000 7f749f400700 1 -- 172.21.15.42:0/1702867021 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7490011960 con 0x7f74a01083f0 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 -- 172.21.15.42:0/1702867021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 msgr2=0x7f74a0108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 --2- 172.21.15.42:0/1702867021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a0108800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f74900098a0 tx=0x7f7490009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:50.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 -- 172.21.15.42:0/1702867021 shutdown_connections 2024-05-27T10:03:50.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 --2- 172.21.15.42:0/1702867021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a0108800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:50.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 -- 172.21.15.42:0/1702867021 >> 172.21.15.42:0/1702867021 conn(0x7f74a0104080 msgr2=0x7f74a01064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:50.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 -- 172.21.15.42:0/1702867021 shutdown_connections 2024-05-27T10:03:50.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.395+0000 7f74a6c00700 1 -- 172.21.15.42:0/1702867021 wait complete. 2024-05-27T10:03:50.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f74a6c00700 1 Processor -- start 2024-05-27T10:03:50.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f74a6c00700 1 -- start start 2024-05-27T10:03:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f74a6c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f74a6c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0109da0 con 0x7f74a01083f0 2024-05-27T10:03:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f749fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f749fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a019ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49252/0 (socket says 172.21.15.42:49252) 2024-05-27T10:03:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.397+0000 7f749fe00700 1 -- 172.21.15.42:0/1699057814 learned_addr learned my addr 172.21.15.42:0/1699057814 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.398+0000 7f749fe00700 1 -- 172.21.15.42:0/1699057814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01a0460 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.398+0000 7f749fe00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a019ff20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7490011670 tx=0x7f7490011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.398+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7490003aa0 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.398+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f749001b8d0 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.398+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74900197f0 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.399+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a06f0 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.399+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a3300 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.400+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a0064e10 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.400+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f749001b440 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.401+0000 7f749d600700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748403e630 0x7f7484040b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.401+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7490054110 con 0x7f74a01083f0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.401+0000 7f7497e00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748403e630 0x7f7484040b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.402+0000 7f7497e00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748403e630 0x7f7484040b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7488006fd0 tx=0x7f7488006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:50.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.406+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f749001f030 con 0x7f74a01083f0 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.632+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f74a002e8c0 con 0x7f74a01083f0 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.633+0000 7f749d600700 1 -- 172.21.15.42:0/1699057814 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f7490019dc0 con 0x7f74a01083f0 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.635+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748403e630 msgr2=0x7f7484040b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.635+0000 7f74a6c00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748403e630 0x7f7484040b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7488006fd0 tx=0x7f7488006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.635+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 msgr2=0x7f74a019ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.635+0000 7f74a6c00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a019ff20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7490011670 tx=0x7f7490011a30 comp rx=0 tx=0).stop 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.636+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 shutdown_connections 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.636+0000 7f74a6c00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748403e630 0x7f7484040b40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.636+0000 7f74a6c00700 1 --2- 172.21.15.42:0/1699057814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a01083f0 0x7f74a019ff20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:50.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.636+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 >> 172.21.15.42:0/1699057814 conn(0x7f74a0104080 msgr2=0x7f74a0193f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:50.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.636+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 shutdown_connections 2024-05-27T10:03:50.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:49.636+0000 7f74a6c00700 1 -- 172.21.15.42:0/1699057814 wait complete. 2024-05-27T10:03:52.237 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.174+0000 7f481b000700 1 Processor -- start 2024-05-27T10:03:52.237 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.175+0000 7f481b000700 1 -- start start 2024-05-27T10:03:52.237 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.175+0000 7f481b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f4814109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:52.237 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.175+0000 7f481b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f481410a520 con 0x7f4814109b20 2024-05-27T10:03:52.237 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.175+0000 7f4813e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f4814109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.176+0000 7f4813e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f4814109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49266/0 (socket says 172.21.15.42:49266) 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.176+0000 7f4813e00700 1 -- 172.21.15.42:0/1819760880 learned_addr learned my addr 172.21.15.42:0/1819760880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.176+0000 7f4813e00700 1 -- 172.21.15.42:0/1819760880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481410acf0 con 0x7f4814109b20 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.177+0000 7f4813e00700 1 --2- 172.21.15.42:0/1819760880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f4814109f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48040098a0 tx=0x7f4804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b7b238735039f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.177+0000 7f4813400700 1 -- 172.21.15.42:0/1819760880 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4804004160 con 0x7f4814109b20 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.177+0000 7f4813400700 1 -- 172.21.15.42:0/1819760880 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4804004300 con 0x7f4814109b20 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.177+0000 7f4813400700 1 -- 172.21.15.42:0/1819760880 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4804011960 con 0x7f4814109b20 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 -- 172.21.15.42:0/1819760880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 msgr2=0x7f4814109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 --2- 172.21.15.42:0/1819760880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f4814109f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48040098a0 tx=0x7f4804009bb0 comp rx=0 tx=0).stop 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 -- 172.21.15.42:0/1819760880 shutdown_connections 2024-05-27T10:03:52.238 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 --2- 172.21.15.42:0/1819760880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f4814109f30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 -- 172.21.15.42:0/1819760880 >> 172.21.15.42:0/1819760880 conn(0x7f48141051c0 msgr2=0x7f4814107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 -- 172.21.15.42:0/1819760880 shutdown_connections 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.178+0000 7f481b000700 1 -- 172.21.15.42:0/1819760880 wait complete. 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.179+0000 7f481b000700 1 Processor -- start 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.180+0000 7f481b000700 1 -- start start 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.180+0000 7f481b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f481407ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.180+0000 7f481b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f481410e0a0 con 0x7f4814109b20 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.180+0000 7f4813e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f481407ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.180+0000 7f4813e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f481407ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49278/0 (socket says 172.21.15.42:49278) 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.180+0000 7f4813e00700 1 -- 172.21.15.42:0/1586257393 learned_addr learned my addr 172.21.15.42:0/1586257393 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.181+0000 7f4813e00700 1 -- 172.21.15.42:0/1586257393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481407efa0 con 0x7f4814109b20 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.181+0000 7f4813e00700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f481407ea60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48040119b0 tx=0x7f4804011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.181+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4804003aa0 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.181+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f480401b8d0 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.181+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481407d1f0 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.181+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4804023860 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.182+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481407d6d0 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.183+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f480401b440 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.183+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4814064e10 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.184+0000 7f4811600700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47f803e630 0x7f47f8040b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.184+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4804054080 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.184+0000 7f480b400700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47f803e630 0x7f47f8040b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.185+0000 7f480b400700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47f803e630 0x7f47f8040b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006fd0 tx=0x7f47fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.190+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f480401f030 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.556+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f481407ddc0 con 0x7f4814109b20 2024-05-27T10:03:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.564+0000 7f4811600700 1 -- 172.21.15.42:0/1586257393 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f480400edf0 con 0x7f4814109b20 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.567+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47f803e630 msgr2=0x7f47f8040b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.567+0000 7f481b000700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47f803e630 0x7f47f8040b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006fd0 tx=0x7f47fc006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 msgr2=0x7f481407ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f481407ea60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48040119b0 tx=0x7f4804011a90 comp rx=0 tx=0).stop 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 shutdown_connections 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47f803e630 0x7f47f8040b40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 --2- 172.21.15.42:0/1586257393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4814109b20 0x7f481407ea60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 >> 172.21.15.42:0/1586257393 conn(0x7f48141051c0 msgr2=0x7f4814105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 shutdown_connections 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-05-27T10:03:51.568+0000 7f481b000700 1 -- 172.21.15.42:0/1586257393 wait complete. 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-27T10:03:52.241 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2024-05-27T10:03:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.243 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-27T10:03:52.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:03:52.243 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2024-05-27T10:03:52.322 INFO:tasks.cephadm:Fetching config... 2024-05-27T10:03:52.323 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:03:52.323 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-27T10:03:52.379 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-27T10:03:52.380 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:03:52.380 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-27T10:03:52.423 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-27T10:03:52.424 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:03:52.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-27T10:03:52.543 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-27T10:03:52.543 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:03:52.543 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-27T10:03:52.642 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-27T10:03:52.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T10:03:52.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:52 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1586257393' entity='client.admin' 2024-05-27T10:03:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:03:52.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T10:03:52.965 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:03:53.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T10:03:53.132 INFO:teuthology.orchestra.run.smithi143.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCOpPX44LG+fl/TMGTg213iiOd9LXdBakL2Gf2VV3fZK3qmmsCCvfKCChn09jP5LZ7lNVyLridbA4gyMaT+t0SMxReE1lVb1srshjwNGjYVY7k87Ic/qoxQjTm7oFtlKN4wOQSccjcwMf8xtWdKsTDK3YeInH1AuS2zyU86JdrBy2i2wVd38QZhYXtAumbnlBsHbcx7jVYqAm30T1f5d2ivM2/l1qvp2mfTUh5Jg3OHTYXMng2kkoM0GNwh+gEC8CC3RWKy6L6O5Ortz5Jd7GxVBJe9uqYKd7IQZ2cQwmRndnEINqfTy9LsT6sj8lhKZVXQyR0g1Q1/JjyCul+g6Ww2nb6dHJiUVKe+ap6Wrez7mpyY7aKjtvA6km5DhoqXusjzbE7Qf9F0ioaH46hz1BAgXRTV3XkNFSHOCvHGIyBJNrrfJADC33Hu1I46xZOWCNKqfKQkfzCCO8zERstz6GZrBdf5+qXyrGYvjEYE7vHXUxspMPBBG7/xWMRIACcLE= ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:03:53.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-27T10:03:53.622 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:03:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.692+0000 7f266fa00700 1 -- 172.21.15.42:0/401662550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 msgr2=0x7f2668104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.692+0000 7f266fa00700 1 --2- 172.21.15.42:0/401662550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668104020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f265c009910 tx=0x7f265c009c20 comp rx=0 tx=0).stop 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.692+0000 7f266fa00700 1 -- 172.21.15.42:0/401662550 shutdown_connections 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.692+0000 7f266fa00700 1 --2- 172.21.15.42:0/401662550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668104020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.693+0000 7f266fa00700 1 -- 172.21.15.42:0/401662550 >> 172.21.15.42:0/401662550 conn(0x7f26680ff250 msgr2=0x7f2668101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.693+0000 7f266fa00700 1 -- 172.21.15.42:0/401662550 shutdown_connections 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.693+0000 7f266fa00700 1 -- 172.21.15.42:0/401662550 wait complete. 2024-05-27T10:03:56.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.694+0000 7f266fa00700 1 Processor -- start 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.694+0000 7f266fa00700 1 -- start start 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.694+0000 7f266fa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.694+0000 7f266fa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2668109a80 con 0x7f2668103c10 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.695+0000 7f266d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.695+0000 7f266d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32804/0 (socket says 172.21.15.42:32804) 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.695+0000 7f266d000700 1 -- 172.21.15.42:0/4236548681 learned_addr learned my addr 172.21.15.42:0/4236548681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:03:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.695+0000 7f266d000700 1 -- 172.21.15.42:0/4236548681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2668199bd0 con 0x7f2668103c10 2024-05-27T10:03:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.696+0000 7f266d000700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668199690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f265c005950 tx=0x7f265c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.696+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f265c027070 con 0x7f2668103c10 2024-05-27T10:03:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.696+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265c00bdf0 con 0x7f2668103c10 2024-05-27T10:03:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.696+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2668199e60 con 0x7f2668103c10 2024-05-27T10:03:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.696+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f265c00fdb0 con 0x7f2668103c10 2024-05-27T10:03:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.696+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266819ca70 con 0x7f2668103c10 2024-05-27T10:03:56.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.698+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f265c02a070 con 0x7f2668103c10 2024-05-27T10:03:56.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.698+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26681928d0 con 0x7f2668103c10 2024-05-27T10:03:56.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.698+0000 7f2665600700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f265003e570 0x7f2650040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:03:56.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.698+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f265c053860 con 0x7f2668103c10 2024-05-27T10:03:56.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.699+0000 7f2667e00700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f265003e570 0x7f2650040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:03:56.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.699+0000 7f2667e00700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f265003e570 0x7f2650040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2654006fd0 tx=0x7f2654006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:03:56.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.705+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f265c013070 con 0x7f2668103c10 2024-05-27T10:03:56.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.941+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f266802e6f0 con 0x7f2668103c10 2024-05-27T10:03:56.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.943+0000 7f2665600700 1 -- 172.21.15.42:0/4236548681 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f265c00f730 con 0x7f2668103c10 2024-05-27T10:03:56.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f265003e570 msgr2=0x7f2650040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:56.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f265003e570 0x7f2650040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2654006fd0 tx=0x7f2654006e40 comp rx=0 tx=0).stop 2024-05-27T10:03:56.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 msgr2=0x7f2668199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:03:56.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668199690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f265c005950 tx=0x7f265c005810 comp rx=0 tx=0).stop 2024-05-27T10:03:56.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 shutdown_connections 2024-05-27T10:03:56.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f265003e570 0x7f2650040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:56.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 --2- 172.21.15.42:0/4236548681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2668103c10 0x7f2668199690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:03:56.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 >> 172.21.15.42:0/4236548681 conn(0x7f26680ff250 msgr2=0x7f26680fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:03:56.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 shutdown_connections 2024-05-27T10:03:56.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:03:56.947+0000 7f266fa00700 1 -- 172.21.15.42:0/4236548681 wait complete. 2024-05-27T10:03:57.787 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-27T10:03:57.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-27T10:03:58.056 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:57 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4236548681' entity='client.admin' 2024-05-27T10:03:58.901 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:04:00.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:00.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:00.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:00.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:00.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:00.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:03:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 -- 172.21.15.42:0/1934636158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 msgr2=0x7f10f8075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 --2- 172.21.15.42:0/1934636158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f8075ba0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009910 tx=0x7f10ec009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 -- 172.21.15.42:0/1934636158 shutdown_connections 2024-05-27T10:04:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 --2- 172.21.15.42:0/1934636158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f8075ba0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 -- 172.21.15.42:0/1934636158 >> 172.21.15.42:0/1934636158 conn(0x7f10f80ff210 msgr2=0x7f10f81016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 -- 172.21.15.42:0/1934636158 shutdown_connections 2024-05-27T10:04:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.039+0000 7f10ff600700 1 -- 172.21.15.42:0/1934636158 wait complete. 2024-05-27T10:04:01.040 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:00 smithi042 ceph-mon[31661]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.040+0000 7f10ff600700 1 Processor -- start 2024-05-27T10:04:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.040+0000 7f10ff600700 1 -- start start 2024-05-27T10:04:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.041+0000 7f10ff600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.041+0000 7f10ff600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f10f810bc30 con 0x7f10f8077770 2024-05-27T10:04:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.041+0000 7f10fcc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.041+0000 7f10fcc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32820/0 (socket says 172.21.15.42:32820) 2024-05-27T10:04:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.041+0000 7f10fcc00700 1 -- 172.21.15.42:0/4213619067 learned_addr learned my addr 172.21.15.42:0/4213619067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:04:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.042+0000 7f10fcc00700 1 -- 172.21.15.42:0/4213619067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f819bd60 con 0x7f10f8077770 2024-05-27T10:04:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.042+0000 7f10fcc00700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f819b820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005790 tx=0x7f10ec004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.043+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec027070 con 0x7f10f8077770 2024-05-27T10:04:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.043+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ec00bdf0 con 0x7f10f8077770 2024-05-27T10:04:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.043+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec00fd90 con 0x7f10f8077770 2024-05-27T10:04:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.043+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f819bff0 con 0x7f10f8077770 2024-05-27T10:04:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.043+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f819ec00 con 0x7f10f8077770 2024-05-27T10:04:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.044+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f810c3b0 con 0x7f10f8077770 2024-05-27T10:04:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.045+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f10ec02a070 con 0x7f10f8077770 2024-05-27T10:04:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.045+0000 7f10f5600700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10e003e570 0x7f10e0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.046+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f10ec053ac0 con 0x7f10f8077770 2024-05-27T10:04:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.046+0000 7f10f7e00700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10e003e570 0x7f10e0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.047+0000 7f10f7e00700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10e003e570 0x7f10e0040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f10e4006fd0 tx=0x7f10e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.052+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10ec013070 con 0x7f10f8077770 2024-05-27T10:04:01.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.294+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f10f802e6f0 con 0x7f10e003e570 2024-05-27T10:04:01.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.303+0000 7f10f5600700 1 -- 172.21.15.42:0/4213619067 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10f802e6f0 con 0x7f10e003e570 2024-05-27T10:04:01.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10e003e570 msgr2=0x7f10e0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:01.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10e003e570 0x7f10e0040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f10e4006fd0 tx=0x7f10e4006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:01.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 msgr2=0x7f10f819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:01.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f819b820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005790 tx=0x7f10ec004f60 comp rx=0 tx=0).stop 2024-05-27T10:04:01.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 shutdown_connections 2024-05-27T10:04:01.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10e003e570 0x7f10e0040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:01.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 --2- 172.21.15.42:0/4213619067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10f8077770 0x7f10f819b820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:01.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.306+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 >> 172.21.15.42:0/4213619067 conn(0x7f10f80ff210 msgr2=0x7f10f80fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:01.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.307+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 shutdown_connections 2024-05-27T10:04:01.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:01.307+0000 7f10ff600700 1 -- 172.21.15.42:0/4213619067 wait complete. 2024-05-27T10:04:02.151 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-05-27T10:04:02.151 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:04:02.152 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-05-27T10:04:02.209 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:04:02.209 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:02.303 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-05-27T10:04:02.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch host add smithi104 2024-05-27T10:04:02.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:02 smithi042 ceph-mon[31661]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:02.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:02.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:02.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:02.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:02.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:02 smithi042 ceph-mon[31661]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:04:02.715 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:04:03.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:03 smithi042 ceph-mon[31661]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:03.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:03 smithi042 ceph-mon[31661]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:04.442 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:04 smithi042 ceph-mon[31661]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:04.442 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:04 smithi042 ceph-mon[31661]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:04.442 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:04.442 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:04.442 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 -- 172.21.15.42:0/768278152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 msgr2=0x7f94cc105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 --2- 172.21.15.42:0/768278152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc105150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00b0a0 tx=0x7f94bc00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 -- 172.21.15.42:0/768278152 shutdown_connections 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 --2- 172.21.15.42:0/768278152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc105150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 -- 172.21.15.42:0/768278152 >> 172.21.15.42:0/768278152 conn(0x7f94cc0785a0 msgr2=0x7f94cc078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 -- 172.21.15.42:0/768278152 shutdown_connections 2024-05-27T10:04:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.025+0000 7f94d1c00700 1 -- 172.21.15.42:0/768278152 wait complete. 2024-05-27T10:04:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.027+0000 7f94d1c00700 1 Processor -- start 2024-05-27T10:04:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.027+0000 7f94d1c00700 1 -- start start 2024-05-27T10:04:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.027+0000 7f94d1c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.027+0000 7f94d1c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc10dfe0 con 0x7f94cc104d40 2024-05-27T10:04:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.028+0000 7f94caa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.028+0000 7f94caa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59552/0 (socket says 172.21.15.42:59552) 2024-05-27T10:04:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.028+0000 7f94caa00700 1 -- 172.21.15.42:0/2669933951 learned_addr learned my addr 172.21.15.42:0/2669933951 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:04:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.028+0000 7f94caa00700 1 -- 172.21.15.42:0/2669933951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc19fac0 con 0x7f94cc104d40 2024-05-27T10:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.028+0000 7f94caa00700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc19f580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94bc000f80 tx=0x7f94bc009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.029+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94bc017040 con 0x7f94cc104d40 2024-05-27T10:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.029+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94bc022070 con 0x7f94cc104d40 2024-05-27T10:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.029+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc19fd50 con 0x7f94cc104d40 2024-05-27T10:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.029+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94bc004880 con 0x7f94cc104d40 2024-05-27T10:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.029+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc199710 con 0x7f94cc104d40 2024-05-27T10:04:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.030+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc06b6c0 con 0x7f94cc104d40 2024-05-27T10:04:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.031+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f94bc007880 con 0x7f94cc104d40 2024-05-27T10:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.031+0000 7f94c3e00700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f94b003e5b0 0x7f94b0040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.031+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f94bc00f050 con 0x7f94cc104d40 2024-05-27T10:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.032+0000 7f94c2000700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f94b003e5b0 0x7f94b0040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.032+0000 7f94c2000700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f94b003e5b0 0x7f94b0040ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.037+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94bc01c990 con 0x7f94cc104d40 2024-05-27T10:04:05.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:05.280+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7f94cc064e10 con 0x7f94b003e5b0 2024-05-27T10:04:05.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:05 smithi042 ceph-mon[31661]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:06.596 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:06 smithi042 ceph-mon[31661]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:07.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:07 smithi042 ceph-mon[31661]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:07.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:07 smithi042 ceph-mon[31661]: Deploying cephadm binary to smithi104 2024-05-27T10:04:09.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:09 smithi042 ceph-mon[31661]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:11.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.578+0000 7f94c3e00700 1 -- 172.21.15.42:0/2669933951 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f94cc064e10 con 0x7f94b003e5b0 2024-05-27T10:04:11.579 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-05-27T10:04:11.579 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:11 smithi042 ceph-mon[31661]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:11.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f94b003e5b0 msgr2=0x7f94b0040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:11.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f94b003e5b0 0x7f94b0040ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:11.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 msgr2=0x7f94cc19f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc19f580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94bc000f80 tx=0x7f94bc009790 comp rx=0 tx=0).stop 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 shutdown_connections 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f94b003e5b0 0x7f94b0040ac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 --2- 172.21.15.42:0/2669933951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94cc104d40 0x7f94cc19f580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.583+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 >> 172.21.15.42:0/2669933951 conn(0x7f94cc0785a0 msgr2=0x7f94cc109f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.584+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 shutdown_connections 2024-05-27T10:04:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:11.584+0000 7f94d1c00700 1 -- 172.21.15.42:0/2669933951 wait complete. 2024-05-27T10:04:12.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T10:04:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:04:12.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:12.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:12 smithi042 ceph-mon[31661]: Added host smithi104 2024-05-27T10:04:12.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:12.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:13.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:13 smithi042 ceph-mon[31661]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:13.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.073+0000 7fdf82c00700 1 -- 172.21.15.42:0/300454451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 msgr2=0x7fdf7c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.073+0000 7fdf82c00700 1 --2- 172.21.15.42:0/300454451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1041f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009910 tx=0x7fdf6c009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.073+0000 7fdf82c00700 1 -- 172.21.15.42:0/300454451 shutdown_connections 2024-05-27T10:04:15.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.073+0000 7fdf82c00700 1 --2- 172.21.15.42:0/300454451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1041f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:15.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.073+0000 7fdf82c00700 1 -- 172.21.15.42:0/300454451 >> 172.21.15.42:0/300454451 conn(0x7fdf7c0ff690 msgr2=0x7fdf7c101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:15.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.074+0000 7fdf82c00700 1 -- 172.21.15.42:0/300454451 shutdown_connections 2024-05-27T10:04:15.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.074+0000 7fdf82c00700 1 -- 172.21.15.42:0/300454451 wait complete. 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.075+0000 7fdf82c00700 1 Processor -- start 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.075+0000 7fdf82c00700 1 -- start start 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.076+0000 7fdf82c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.076+0000 7fdf82c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c10df80 con 0x7fdf7c103de0 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.076+0000 7fdf7be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.076+0000 7fdf7be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45860/0 (socket says 172.21.15.42:45860) 2024-05-27T10:04:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.076+0000 7fdf7be00700 1 -- 172.21.15.42:0/3118874102 learned_addr learned my addr 172.21.15.42:0/3118874102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:04:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.076+0000 7fdf7be00700 1 -- 172.21.15.42:0/3118874102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c1a1be0 con 0x7fdf7c103de0 2024-05-27T10:04:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.077+0000 7fdf7be00700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1a16a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005660 tx=0x7fdf6c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.077+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf6c026070 con 0x7fdf7c103de0 2024-05-27T10:04:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.077+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf6c00b6f0 con 0x7fdf7c103de0 2024-05-27T10:04:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.077+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf6c0037b0 con 0x7fdf7c103de0 2024-05-27T10:04:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.077+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c1a1ed0 con 0x7fdf7c103de0 2024-05-27T10:04:15.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.077+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c19b8c0 con 0x7fdf7c103de0 2024-05-27T10:04:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.078+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c105290 con 0x7fdf7c103de0 2024-05-27T10:04:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.079+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fdf6c029070 con 0x7fdf7c103de0 2024-05-27T10:04:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.080+0000 7fdf79600700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdf6003e570 0x7fdf60040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.080+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf6c01e070 con 0x7fdf7c103de0 2024-05-27T10:04:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.080+0000 7fdf73400700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdf6003e570 0x7fdf60040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.081+0000 7fdf73400700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdf6003e570 0x7fdf60040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:15.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.085+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf6c030080 con 0x7fdf7c103de0 2024-05-27T10:04:15.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.327+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf7c19cdf0 con 0x7fdf6003e570 2024-05-27T10:04:15.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.329+0000 7fdf79600700 1 -- 172.21.15.42:0/3118874102 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fdf7c19cdf0 con 0x7fdf6003e570 2024-05-27T10:04:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:04:15.330 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-05-27T10:04:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdf6003e570 msgr2=0x7fdf60040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdf6003e570 0x7fdf60040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 msgr2=0x7fdf7c1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1a16a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005660 tx=0x7fdf6c005740 comp rx=0 tx=0).stop 2024-05-27T10:04:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 shutdown_connections 2024-05-27T10:04:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdf6003e570 0x7fdf60040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 --2- 172.21.15.42:0/3118874102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf7c103de0 0x7fdf7c1a16a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 >> 172.21.15.42:0/3118874102 conn(0x7fdf7c0ff690 msgr2=0x7fdf7c075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.333+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 shutdown_connections 2024-05-27T10:04:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:15.334+0000 7fdf82c00700 1 -- 172.21.15.42:0/3118874102 wait complete. 2024-05-27T10:04:15.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:15 smithi042 ceph-mon[31661]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:15.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:16.094 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi143 2024-05-27T10:04:16.094 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:04:16.094 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.conf 2024-05-27T10:04:16.151 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:04:16.151 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:16.247 INFO:tasks.cephadm:Adding host smithi143 to orchestrator... 2024-05-27T10:04:16.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch host add smithi143 2024-05-27T10:04:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:04:16.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:16 smithi042 ceph-mon[31661]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:04:17.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:17 smithi042 ceph-mon[31661]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:18.668 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:18 smithi042 ceph-mon[31661]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.910+0000 7f0e4c800700 1 -- 172.21.15.42:0/533186434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 msgr2=0x7f0e4410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.910+0000 7f0e4c800700 1 --2- 172.21.15.42:0/533186434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e4410a040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009910 tx=0x7f0e38009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.910+0000 7f0e4c800700 1 -- 172.21.15.42:0/533186434 shutdown_connections 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.910+0000 7f0e4c800700 1 --2- 172.21.15.42:0/533186434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e4410a040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.910+0000 7f0e4c800700 1 -- 172.21.15.42:0/533186434 >> 172.21.15.42:0/533186434 conn(0x7f0e440ff650 msgr2=0x7f0e44101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.910+0000 7f0e4c800700 1 -- 172.21.15.42:0/533186434 shutdown_connections 2024-05-27T10:04:18.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.911+0000 7f0e4c800700 1 -- 172.21.15.42:0/533186434 wait complete. 2024-05-27T10:04:18.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.912+0000 7f0e4c800700 1 Processor -- start 2024-05-27T10:04:18.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.912+0000 7f0e4c800700 1 -- start start 2024-05-27T10:04:18.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.912+0000 7f0e4c800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e441a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:18.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.912+0000 7f0e4c800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410dfc0 con 0x7f0e44109c30 2024-05-27T10:04:18.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.912+0000 7f0e49e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e441a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:18.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e49e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e441a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45886/0 (socket says 172.21.15.42:45886) 2024-05-27T10:04:18.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e49e00700 1 -- 172.21.15.42:0/2020636679 learned_addr learned my addr 172.21.15.42:0/2020636679 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:04:18.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e49e00700 1 -- 172.21.15.42:0/2020636679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e441a1b70 con 0x7f0e44109c30 2024-05-27T10:04:18.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e49e00700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e441a1630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e380092e0 tx=0x7f0e38005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:18.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e38026070 con 0x7f0e44109c30 2024-05-27T10:04:18.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e3800b750 con 0x7f0e44109c30 2024-05-27T10:04:18.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.913+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441a1e00 con 0x7f0e44109c30 2024-05-27T10:04:18.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.914+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e38003810 con 0x7f0e44109c30 2024-05-27T10:04:18.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.914+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4419b870 con 0x7f0e44109c30 2024-05-27T10:04:18.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.915+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44105200 con 0x7f0e44109c30 2024-05-27T10:04:18.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.915+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0e38029070 con 0x7f0e44109c30 2024-05-27T10:04:18.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.915+0000 7f0e3ea00700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0e2c03e570 0x7f0e2c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:18.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.916+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0e3801e070 con 0x7f0e44109c30 2024-05-27T10:04:18.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.916+0000 7f0e49400700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0e2c03e570 0x7f0e2c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:18.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.916+0000 7f0e49400700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0e2c03e570 0x7f0e2c040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:18.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:18.922+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e38013070 con 0x7f0e44109c30 2024-05-27T10:04:19.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:19.165+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}) -- 0x7f0e4419cd80 con 0x7f0e2c03e570 2024-05-27T10:04:20.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:19 smithi042 ceph-mon[31661]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:21.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:21 smithi042 ceph-mon[31661]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:04:22.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:22 smithi042 ceph-mon[31661]: Deploying cephadm binary to smithi143 2024-05-27T10:04:22.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:22 smithi042 ceph-mon[31661]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:23.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:23 smithi042 ceph-mon[31661]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:25.668 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:25 smithi042 ceph-mon[31661]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:25.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.667+0000 7f0e3ea00700 1 -- 172.21.15.42:0/2020636679 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0e4419cd80 con 0x7f0e2c03e570 2024-05-27T10:04:25.669 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi143' with addr '172.21.15.143' 2024-05-27T10:04:25.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.671+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0e2c03e570 msgr2=0x7f0e2c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:25.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.671+0000 7f0e4c800700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0e2c03e570 0x7f0e2c040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:25.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.671+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 msgr2=0x7f0e441a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:25.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.671+0000 7f0e4c800700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e441a1630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e380092e0 tx=0x7f0e38005710 comp rx=0 tx=0).stop 2024-05-27T10:04:25.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.672+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 shutdown_connections 2024-05-27T10:04:25.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.672+0000 7f0e4c800700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0e2c03e570 0x7f0e2c040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:25.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.672+0000 7f0e4c800700 1 --2- 172.21.15.42:0/2020636679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e44109c30 0x7f0e441a1630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:25.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.672+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 >> 172.21.15.42:0/2020636679 conn(0x7f0e440ff650 msgr2=0x7f0e440ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:25.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.672+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 shutdown_connections 2024-05-27T10:04:25.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:25.672+0000 7f0e4c800700 1 -- 172.21.15.42:0/2020636679 wait complete. 2024-05-27T10:04:26.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:26 smithi042 ceph-mon[31661]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:26.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:26 smithi042 ceph-mon[31661]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:26.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:26.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:26.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:26.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:26.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T10:04:27.052 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:04:27.969 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:27 smithi042 ceph-mon[31661]: Added host smithi143 2024-05-27T10:04:27.969 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:27 smithi042 ceph-mon[31661]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:27.969 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:28.951 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:28.951 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:28 smithi042 ceph-mon[31661]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:28.951 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:29.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.299+0000 7f95d3c00700 1 -- 172.21.15.42:0/654455650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 msgr2=0x7f95cc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.299+0000 7f95d3c00700 1 --2- 172.21.15.42:0/654455650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc076130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009910 tx=0x7f95c0009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.300+0000 7f95d3c00700 1 -- 172.21.15.42:0/654455650 shutdown_connections 2024-05-27T10:04:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.300+0000 7f95d3c00700 1 --2- 172.21.15.42:0/654455650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc076130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.300+0000 7f95d3c00700 1 -- 172.21.15.42:0/654455650 >> 172.21.15.42:0/654455650 conn(0x7f95cc0ff690 msgr2=0x7f95cc101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.300+0000 7f95d3c00700 1 -- 172.21.15.42:0/654455650 shutdown_connections 2024-05-27T10:04:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.300+0000 7f95d3c00700 1 -- 172.21.15.42:0/654455650 wait complete. 2024-05-27T10:04:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.301+0000 7f95d3c00700 1 Processor -- start 2024-05-27T10:04:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.301+0000 7f95d3c00700 1 -- start start 2024-05-27T10:04:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.302+0000 7f95d3c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.302+0000 7f95d3c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1101e0 con 0x7f95cc075d20 2024-05-27T10:04:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.302+0000 7f95d1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.302+0000 7f95d1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc1a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40438/0 (socket says 172.21.15.42:40438) 2024-05-27T10:04:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.302+0000 7f95d1200700 1 -- 172.21.15.42:0/982990693 learned_addr learned my addr 172.21.15.42:0/982990693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:04:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.302+0000 7f95d1200700 1 -- 172.21.15.42:0/982990693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc1a1bf0 con 0x7f95cc075d20 2024-05-27T10:04:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.303+0000 7f95d1200700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc1a16b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95c0005060 tx=0x7f95c0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.303+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c0026070 con 0x7f95cc075d20 2024-05-27T10:04:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.303+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95c000b750 con 0x7f95cc075d20 2024-05-27T10:04:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.303+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc1a1e80 con 0x7f95cc075d20 2024-05-27T10:04:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.303+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c0003850 con 0x7f95cc075d20 2024-05-27T10:04:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.303+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc19b920 con 0x7f95cc075d20 2024-05-27T10:04:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.305+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc077290 con 0x7f95cc075d20 2024-05-27T10:04:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.305+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f95c0029070 con 0x7f95cc075d20 2024-05-27T10:04:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.305+0000 7f95c9600700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f95b403e510 0x7f95b4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.305+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f95c001e070 con 0x7f95cc075d20 2024-05-27T10:04:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.305+0000 7f95cbe00700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f95b403e510 0x7f95b4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:29.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.306+0000 7f95cbe00700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f95b403e510 0x7f95b4040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f95b8006fd0 tx=0x7f95b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:29.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.312+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95c0013070 con 0x7f95cc075d20 2024-05-27T10:04:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.564+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95cc19ce00 con 0x7f95b403e510 2024-05-27T10:04:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.567+0000 7f95c9600700 1 -- 172.21.15.42:0/982990693 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f95cc19ce00 con 0x7f95b403e510 2024-05-27T10:04:29.567 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:04:29.567 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}] 2024-05-27T10:04:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f95b403e510 msgr2=0x7f95b4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f95b403e510 0x7f95b4040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f95b8006fd0 tx=0x7f95b8006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 msgr2=0x7f95cc1a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc1a16b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95c0005060 tx=0x7f95c0005710 comp rx=0 tx=0).stop 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 shutdown_connections 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f95b403e510 0x7f95b4040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 --2- 172.21.15.42:0/982990693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075d20 0x7f95cc1a16b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.570+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 >> 172.21.15.42:0/982990693 conn(0x7f95cc0ff690 msgr2=0x7f95cc10c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.571+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 shutdown_connections 2024-05-27T10:04:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:29.571+0000 7f95d3c00700 1 -- 172.21.15.42:0/982990693 wait complete. 2024-05-27T10:04:29.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:29 smithi042 ceph-mon[31661]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:04:30.380 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-27T10:04:30.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-27T10:04:30.846 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:04:30.995 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:30 smithi042 ceph-mon[31661]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:30.995 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:32.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.964+0000 7fb2d1600700 1 -- 172.21.15.42:0/2791200652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 msgr2=0x7fb2cc1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:32.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.964+0000 7fb2d1600700 1 --2- 172.21.15.42:0/2791200652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1063e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009910 tx=0x7fb2bc009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:32.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.965+0000 7fb2d1600700 1 -- 172.21.15.42:0/2791200652 shutdown_connections 2024-05-27T10:04:32.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.965+0000 7fb2d1600700 1 --2- 172.21.15.42:0/2791200652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1063e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:32.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.965+0000 7fb2d1600700 1 -- 172.21.15.42:0/2791200652 >> 172.21.15.42:0/2791200652 conn(0x7fb2cc101880 msgr2=0x7fb2cc103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:32.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.965+0000 7fb2d1600700 1 -- 172.21.15.42:0/2791200652 shutdown_connections 2024-05-27T10:04:32.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.965+0000 7fb2d1600700 1 -- 172.21.15.42:0/2791200652 wait complete. 2024-05-27T10:04:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.966+0000 7fb2d1600700 1 Processor -- start 2024-05-27T10:04:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.966+0000 7fb2d1600700 1 -- start start 2024-05-27T10:04:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.967+0000 7fb2d1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.967+0000 7fb2d1600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc110170 con 0x7fb2cc105fd0 2024-05-27T10:04:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.967+0000 7fb2caa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.967+0000 7fb2caa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55780/0 (socket says 172.21.15.42:55780) 2024-05-27T10:04:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.967+0000 7fb2caa00700 1 -- 172.21.15.42:0/766381553 learned_addr learned my addr 172.21.15.42:0/766381553 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:04:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.967+0000 7fb2caa00700 1 -- 172.21.15.42:0/766381553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1a1c40 con 0x7fb2cc105fd0 2024-05-27T10:04:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.968+0000 7fb2caa00700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1a1700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005660 tx=0x7fb2bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.968+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2bc026070 con 0x7fb2cc105fd0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.968+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2bc00b6f0 con 0x7fb2cc105fd0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.968+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2bc0037b0 con 0x7fb2cc105fd0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.968+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc1a1f30 con 0x7fb2cc105fd0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.968+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc19b920 con 0x7fb2cc105fd0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.969+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc107480 con 0x7fb2cc105fd0 2024-05-27T10:04:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.970+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb2bc029070 con 0x7fb2cc105fd0 2024-05-27T10:04:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.970+0000 7fb2c3400700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b003e510 0x7fb2b0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.970+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2bc01e070 con 0x7fb2cc105fd0 2024-05-27T10:04:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.970+0000 7fb2ca000700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b003e510 0x7fb2b0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.971+0000 7fb2ca000700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b003e510 0x7fb2b0040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:32.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:32.976+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2bc013070 con 0x7fb2cc105fd0 2024-05-27T10:04:33.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.218+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb2cc19ce50 con 0x7fb2cc105fd0 2024-05-27T10:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.266+0000 7fb2c3400700 1 -- 172.21.15.42:0/766381553 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb2bc029d30 con 0x7fb2cc105fd0 2024-05-27T10:04:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b003e510 msgr2=0x7fb2b0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b003e510 0x7fb2b0040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 msgr2=0x7fb2cc1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1a1700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005660 tx=0x7fb2bc005740 comp rx=0 tx=0).stop 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 shutdown_connections 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b003e510 0x7fb2b0040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 --2- 172.21.15.42:0/766381553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2cc105fd0 0x7fb2cc1a1700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 >> 172.21.15.42:0/766381553 conn(0x7fb2cc101880 msgr2=0x7fb2cc10bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 shutdown_connections 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:04:33.270+0000 7fb2d1600700 1 -- 172.21.15.42:0/766381553 wait complete. 2024-05-27T10:04:33.271 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-05-27T10:04:33.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:33 smithi042 ceph-mon[31661]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:33.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:33 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/766381553' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T10:04:34.090 INFO:tasks.cephadm:Adding mon.a on smithi042 2024-05-27T10:04:34.090 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-05-27T10:04:34.091 INFO:tasks.cephadm:Adding mon.c on smithi143 2024-05-27T10:04:34.091 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c' 2024-05-27T10:04:34.505 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T10:04:34.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:34 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/766381553' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T10:04:34.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:34 smithi042 ceph-mon[31661]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:04:34.615 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T10:04:35.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:35 smithi042 ceph-mon[31661]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 -- 172.21.15.143:0/2023135749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 msgr2=0x7f1318106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 --2- 172.21.15.143:0/2023135749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f1318106af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f130c009910 tx=0x7f130c009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 -- 172.21.15.143:0/2023135749 shutdown_connections 2024-05-27T10:04:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 --2- 172.21.15.143:0/2023135749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f1318106af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 -- 172.21.15.143:0/2023135749 >> 172.21.15.143:0/2023135749 conn(0x7f13180fe130 msgr2=0x7f13181005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 -- 172.21.15.143:0/2023135749 shutdown_connections 2024-05-27T10:04:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.983+0000 7f131fa00700 1 -- 172.21.15.143:0/2023135749 wait complete. 2024-05-27T10:04:36.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.984+0000 7f131fa00700 1 Processor -- start 2024-05-27T10:04:36.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.985+0000 7f131fa00700 1 -- start start 2024-05-27T10:04:36.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.985+0000 7f131fa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f131819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:36.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.985+0000 7f131fa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1318109a40 con 0x7f13181046d0 2024-05-27T10:04:36.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.986+0000 7f131d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f131819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:36.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.986+0000 7f131d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f131819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.143:51744/0 (socket says 172.21.15.143:51744) 2024-05-27T10:04:36.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.986+0000 7f131d000700 1 -- 172.21.15.143:0/3667743973 learned_addr learned my addr 172.21.15.143:0/3667743973 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:04:36.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.986+0000 7f131d000700 1 -- 172.21.15.143:0/3667743973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131819bd60 con 0x7f13181046d0 2024-05-27T10:04:36.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.987+0000 7f131d000700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f131819b820 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f130c005120 tx=0x7f130c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.987+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f130c027070 con 0x7f13181046d0 2024-05-27T10:04:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.987+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131819bff0 con 0x7f13181046d0 2024-05-27T10:04:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.987+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f130c00bdf0 con 0x7f13181046d0 2024-05-27T10:04:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.987+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f130c00fd70 con 0x7f13181046d0 2024-05-27T10:04:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.987+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131819ec00 con 0x7f13181046d0 2024-05-27T10:04:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.989+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131810a1c0 con 0x7f13181046d0 2024-05-27T10:04:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.989+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f130c02a070 con 0x7f13181046d0 2024-05-27T10:04:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.990+0000 7f1315600700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f130003e570 0x7f1300040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.990+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f130c053b00 con 0x7f13181046d0 2024-05-27T10:04:36.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.991+0000 7f1317e00700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f130003e570 0x7f1300040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:36.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:36.992+0000 7f1317e00700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f130003e570 0x7f1300040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1304006fd0 tx=0x7f1304006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:37.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.008+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f130c013070 con 0x7f13181046d0 2024-05-27T10:04:37.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.255+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}) -- 0x7f131802e870 con 0x7f130003e570 2024-05-27T10:04:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.273+0000 7f1315600700 1 -- 172.21.15.143:0/3667743973 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f131802e870 con 0x7f130003e570 2024-05-27T10:04:37.274 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mon update... 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f130003e570 msgr2=0x7f1300040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f130003e570 0x7f1300040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1304006fd0 tx=0x7f1304006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 msgr2=0x7f131819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f131819b820 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f130c005120 tx=0x7f130c005810 comp rx=0 tx=0).stop 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 shutdown_connections 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f130003e570 0x7f1300040a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:37.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 --2- 172.21.15.143:0/3667743973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13181046d0 0x7f131819b820 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:37.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.276+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 >> 172.21.15.143:0/3667743973 conn(0x7f13180fe130 msgr2=0x7f13181005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:37.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.277+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 shutdown_connections 2024-05-27T10:04:37.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:37.277+0000 7f131fa00700 1 -- 172.21.15.143:0/3667743973 wait complete. 2024-05-27T10:04:37.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:04:37.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:38.131 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.b.service 2024-05-27T10:04:38.139 DEBUG:teuthology.orchestra.run.smithi143:mon.c> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.c.service 2024-05-27T10:04:38.144 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T10:04:38.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T10:04:38.813 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:38.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:38 smithi042 ceph-mon[31661]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:38.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:38 smithi042 ceph-mon[31661]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:38.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:38 smithi042 ceph-mon[31661]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c;count:3 2024-05-27T10:04:39.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:04:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:40.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:40 smithi042 ceph-mon[31661]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:40.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:40 smithi042 ceph-mon[31661]: Deploying daemon mon.c on smithi143 2024-05-27T10:04:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:41 smithi042 ceph-mon[31661]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:41.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.965+0000 7f4ad1800700 1 -- 172.21.15.143:0/2733184783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 msgr2=0x7f4acc1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:41.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.965+0000 7f4ad1800700 1 --2- 172.21.15.143:0/2733184783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc1041f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009910 tx=0x7f4abc009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:41.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.965+0000 7f4ad1800700 1 -- 172.21.15.143:0/2733184783 shutdown_connections 2024-05-27T10:04:41.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.965+0000 7f4ad1800700 1 --2- 172.21.15.143:0/2733184783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc1041f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:41.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.965+0000 7f4ad1800700 1 -- 172.21.15.143:0/2733184783 >> 172.21.15.143:0/2733184783 conn(0x7f4acc0ff690 msgr2=0x7f4acc101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:41.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.965+0000 7f4ad1800700 1 -- 172.21.15.143:0/2733184783 shutdown_connections 2024-05-27T10:04:41.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.967+0000 7f4ad1800700 1 -- 172.21.15.143:0/2733184783 wait complete. 2024-05-27T10:04:41.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.967+0000 7f4ad1800700 1 Processor -- start 2024-05-27T10:04:41.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.967+0000 7f4ad1800700 1 -- start start 2024-05-27T10:04:41.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.968+0000 7f4ad1800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:41.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.968+0000 7f4ad1800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc10df80 con 0x7f4acc103de0 2024-05-27T10:04:41.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.969+0000 7f4acaa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:41.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.969+0000 7f4acaa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.143:51764/0 (socket says 172.21.15.143:51764) 2024-05-27T10:04:41.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.969+0000 7f4acaa00700 1 -- 172.21.15.143:0/1962266006 learned_addr learned my addr 172.21.15.143:0/1962266006 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:04:41.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.969+0000 7f4acaa00700 1 -- 172.21.15.143:0/1962266006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc19fa40 con 0x7f4acc103de0 2024-05-27T10:04:41.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.970+0000 7f4acaa00700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc19f500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005660 tx=0x7f4abc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:41.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.970+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4abc026070 con 0x7f4acc103de0 2024-05-27T10:04:41.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.970+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc19fd30 con 0x7f4acc103de0 2024-05-27T10:04:41.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.970+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc199860 con 0x7f4acc103de0 2024-05-27T10:04:41.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.970+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4abc00b6f0 con 0x7f4acc103de0 2024-05-27T10:04:41.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.970+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4abc0037d0 con 0x7f4acc103de0 2024-05-27T10:04:41.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.972+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa8005330 con 0x7f4acc103de0 2024-05-27T10:04:41.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.972+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4abc029070 con 0x7f4acc103de0 2024-05-27T10:04:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.973+0000 7f4ac3400700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4ab003e510 0x7f4ab0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.973+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4abc01e070 con 0x7f4acc103de0 2024-05-27T10:04:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.974+0000 7f4ac3e00700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4ab003e510 0x7f4ab0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.975+0000 7f4ac3e00700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4ab003e510 0x7f4ab0040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006fd0 tx=0x7f4ab4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:41.990+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4abc013070 con 0x7f4acc103de0 2024-05-27T10:04:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.325+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4aa8005620 con 0x7f4acc103de0 2024-05-27T10:04:42.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.327+0000 7f4ac3400700 1 -- 172.21.15.143:0/1962266006 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4abc030090 con 0x7f4acc103de0 2024-05-27T10:04:42.328 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:04:42.328 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":1,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","modified":"2024-05-27T10:02:23.315287Z","created":"2024-05-27T10:02:23.315287Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T10:04:42.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.330+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4ab003e510 msgr2=0x7f4ab0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:42.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.330+0000 7f4ad1800700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4ab003e510 0x7f4ab0040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006fd0 tx=0x7f4ab4006e40 comp rx=0 tx=0).stop 2024-05-27T10:04:42.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 msgr2=0x7f4acc19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:42.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc19f500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005660 tx=0x7f4abc005740 comp rx=0 tx=0).stop 2024-05-27T10:04:42.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 shutdown_connections 2024-05-27T10:04:42.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4ab003e510 0x7f4ab0040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:42.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 --2- 172.21.15.143:0/1962266006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4acc103de0 0x7f4acc19f500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:42.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 >> 172.21.15.143:0/1962266006 conn(0x7f4acc0ff690 msgr2=0x7f4acc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:42.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 shutdown_connections 2024-05-27T10:04:42.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:42.331+0000 7f4ad1800700 1 -- 172.21.15.143:0/1962266006 wait complete. 2024-05-27T10:04:42.334 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 1 2024-05-27T10:04:42.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:42 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/1962266006' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:04:43.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:43 smithi042 ceph-mon[31661]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:44.991 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T10:04:44.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T10:04:45.506 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 podman[35501]: 2024-05-27 10:04:45.159311469 +0000 UTC m=+0.032607483 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:04:45.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 podman[35501]: 2024-05-27 10:04:45.505661985 +0000 UTC m=+0.378958022 container init 945848b5fdc3f069414537a893b4ed078933c990444d720c4b7fac5ef8edb2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:04:45.673 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 podman[35501]: 2024-05-27 10:04:45.51526246 +0000 UTC m=+0.388558480 container start 945848b5fdc3f069414537a893b4ed078933c990444d720c4b7fac5ef8edb2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 bash[35501]: 945848b5fdc3f069414537a893b4ed078933c990444d720c4b7fac5ef8edb2c3 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 systemd[1]: Started Ceph mon.c for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev), process ceph-mon, pid 2 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pidfile_write: ignore empty --pid-file 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: load: jerasure load: lrc 2024-05-27T10:04:45.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: RocksDB version: 7.9.2 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Git sha 0 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Compile date 2024-05-21 11:39:56 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: DB SUMMARY 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: DB Session ID: 4QSMURQCVXJ1ATILRU8U 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: CURRENT file: CURRENT 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.error_if_exists: 0 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.create_if_missing: 0 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.paranoid_checks: 1 2024-05-27T10:04:45.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.env: 0x55b46f27c500 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.info_log: 0x55b4af1ae040 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.statistics: (nil) 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.use_fsync: 0 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_log_file_size: 0 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T10:04:45.812 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.allow_fallocate: 1 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.use_direct_reads: 0 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.db_log_dir: 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.wal_dir: 2024-05-27T10:04:45.813 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.write_buffer_manager: 0x55b4af0788c0 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T10:04:45.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.unordered_write: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.row_cache: None 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.wal_filter: None 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.two_write_queues: 0 2024-05-27T10:04:45.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.wal_compression: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.atomic_flush: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.log_readahead_size: 0 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T10:04:45.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_background_jobs: 2 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_background_compactions: -1 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_subcompactions: 1 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T10:04:45.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_open_files: -1 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_background_flushes: -1 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Compression algorithms supported: 2024-05-27T10:04:45.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kZSTD supported: 0 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kXpressCompression supported: 0 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kZlibCompression supported: 1 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kSnappyCompression supported: 1 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kLZ4Compression supported: 1 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: kBZip2Compression supported: 0 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T10:04:45.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.merge_operator: 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_filter: None 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b4af1ae200) 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: cache_index_and_filter_blocks: 1 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: pin_top_level_index_and_filter: 1 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_type: 0 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: data_block_index_type: 0 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_shortening: 1 2024-05-27T10:04:45.820 INFO:journalctl@ceph.mon.c.smithi143.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: checksum: 4 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: no_block_cache: 0 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache: 0x55b4af0ab1f0 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_name: BinnedLRUCache 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_options: 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: capacity : 536870912 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: num_shard_bits : 4 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: strict_capacity_limit : 0 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: high_pri_pool_ratio: 0.000 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_compressed: (nil) 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: persistent_cache: (nil) 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_size: 4096 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_size_deviation: 10 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_restart_interval: 16 2024-05-27T10:04:45.821 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_block_restart_interval: 1 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: metadata_block_size: 4096 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: partition_filters: 0 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: use_delta_encoding: 1 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: filter_policy: bloomfilter 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: whole_key_filtering: 1 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: verify_compression: 0 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: read_amp_bytes_per_bit: 0 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: format_version: 5 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: enable_index_compression: 1 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_align: 0 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: max_auto_readahead_size: 262144 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: prepopulate_block_cache: 0 2024-05-27T10:04:45.822 INFO:journalctl@ceph.mon.c.smithi143.stdout: initial_auto_readahead_size: 8192 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression: NoCompression 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T10:04:45.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T10:04:45.824 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.num_levels: 7 2024-05-27T10:04:45.824 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T10:04:45.824 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T10:04:45.824 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T10:04:45.824 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T10:04:45.825 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T10:04:45.826 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T10:04:45.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T10:04:45.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.inplace_update_support: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.bloom_locality: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.max_successive_merges: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.ttl: 2592000 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T10:04:45.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.enable_blob_files: false 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.min_blob_size: 0 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T10:04:45.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 573f02cb-c32f-4183-a3f2-f3259595c35a 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804285650915, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804285652410, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716804285, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "573f02cb-c32f-4183-a3f2-f3259595c35a", "db_session_id": "4QSMURQCVXJ1ATILRU8U", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804285652687, "job": 1, "event": "recovery_finished"} 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b4af08fc00 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: DB pointer 0x55b4b3e82000 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T10:04:45.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** DB Stats ** 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T10:04:45.832 INFO:journalctl@ceph.mon.c.smithi143.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T10:04:45.833 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: Block cache BinnedLRUCache@0x55b4af0ab1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-27T10:04:45.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: using public_addr v2:172.21.15.143:0/0 -> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: starting mon.c rank -1 at public addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] at bind addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(???) e0 preinit fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).mds e1 new map 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout: e1 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout: legacy client fscid: -1 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout: No filesystems configured 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T10:04:45.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Deploying cephadm binary to smithi104 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Added host smithi104 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.837 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Deploying cephadm binary to smithi143 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.838 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Added host smithi143 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/766381553' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/766381553' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c;count:3 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: Deploying daemon mon.c on smithi143 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/1962266006' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:04:45.841 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.842 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:45.842 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:45 smithi143 ceph-mon[35545]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-27T10:04:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:45 smithi042 ceph-mon[31661]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.793+0000 7fa5efa00700 1 -- 172.21.15.143:0/3341571586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 msgr2=0x7fa5d0005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.793+0000 7fa5efa00700 1 --2- 172.21.15.143:0/3341571586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5d0005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc005fa0 tx=0x7fa5dc00f4a0 comp rx=0 tx=0).stop 2024-05-27T10:04:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.793+0000 7fa5efa00700 1 -- 172.21.15.143:0/3341571586 shutdown_connections 2024-05-27T10:04:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.793+0000 7fa5efa00700 1 --2- 172.21.15.143:0/3341571586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5d0005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.793+0000 7fa5efa00700 1 -- 172.21.15.143:0/3341571586 >> 172.21.15.143:0/3341571586 conn(0x7fa5e80ff760 msgr2=0x7fa5e8101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:50.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.794+0000 7fa5efa00700 1 -- 172.21.15.143:0/3341571586 shutdown_connections 2024-05-27T10:04:50.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.794+0000 7fa5efa00700 1 -- 172.21.15.143:0/3341571586 wait complete. 2024-05-27T10:04:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.796+0000 7fa5efa00700 1 Processor -- start 2024-05-27T10:04:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.796+0000 7fa5efa00700 1 -- start start 2024-05-27T10:04:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.797+0000 7fa5efa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5e8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.797+0000 7fa5efa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5e8075ff0 0x7fa5e8076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.797+0000 7fa5efa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810baa0 con 0x7fa5e8103eb0 2024-05-27T10:04:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.797+0000 7fa5efa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810b920 con 0x7fa5e8075ff0 2024-05-27T10:04:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.798+0000 7fa5ed000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5e8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.798+0000 7fa5ed000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5e8075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.143:45454/0 (socket says 172.21.15.143:45454) 2024-05-27T10:04:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.798+0000 7fa5ed000700 1 -- 172.21.15.143:0/786167015 learned_addr learned my addr 172.21.15.143:0/786167015 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:04:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.798+0000 7fa5e7e00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5e8075ff0 0x7fa5e8076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.799+0000 7fa5ed000700 1 -- 172.21.15.143:0/786167015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5e8075ff0 msgr2=0x7fa5e8076400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:50.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.799+0000 7fa5ed000700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5e8075ff0 0x7fa5e8076400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:50.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.799+0000 7fa5ed000700 1 -- 172.21.15.143:0/786167015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e80779e0 con 0x7fa5e8103eb0 2024-05-27T10:04:50.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.799+0000 7fa5ed000700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5e8075ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc01e040 tx=0x7fa5dc01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.799+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa5dc027450 con 0x7fa5e8103eb0 2024-05-27T10:04:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.800+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e81a5c00 con 0x7fa5e8103eb0 2024-05-27T10:04:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.800+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5dc02c070 con 0x7fa5e8103eb0 2024-05-27T10:04:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.800+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa5dc025690 con 0x7fa5e8103eb0 2024-05-27T10:04:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.800+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e81a6160 con 0x7fa5e8103eb0 2024-05-27T10:04:50.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.802+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e8105360 con 0x7fa5e8103eb0 2024-05-27T10:04:50.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.802+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa5dc027b10 con 0x7fa5e8103eb0 2024-05-27T10:04:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.803+0000 7fa5e5600700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa5c803e2d0 0x7fa5c80407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.803+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa5dc019070 con 0x7fa5e8103eb0 2024-05-27T10:04:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.804+0000 7fa5e7e00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa5c803e2d0 0x7fa5c80407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.805+0000 7fa5e7e00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa5c803e2d0 0x7fa5c80407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa5e81032e0 tx=0x7fa5d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:50.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:50.820+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5dc02fa50 con 0x7fa5e8103eb0 2024-05-27T10:04:51.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:04:51.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: mon.a calling monitor election 2024-05-27T10:04:51.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:51.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: mon.c calling monitor election 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: monmap epoch 2 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: last_changed 2024-05-27T10:04:45.721549+0000 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:04:51.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: min_mon_release 19 (squid) 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: election_strategy: 1 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: fsmap 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: mgrmap e13: x(active, since 70s) 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: overall HEALTH_OK 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:04:51.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:51.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: mon.a calling monitor election 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: mon.c calling monitor election 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:51.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: monmap epoch 2 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: last_changed 2024-05-27T10:04:45.721549+0000 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: min_mon_release 19 (squid) 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: election_strategy: 1 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: fsmap 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: mgrmap e13: x(active, since 70s) 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: overall HEALTH_OK 2024-05-27T10:04:51.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:51.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:04:51.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.162+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa5e802e5e0 con 0x7fa5e8103eb0 2024-05-27T10:04:51.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.163+0000 7fa5e5600700 1 -- 172.21.15.143:0/786167015 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa5dc030ca0 con 0x7fa5e8103eb0 2024-05-27T10:04:51.163 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:04:51.163 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","modified":"2024-05-27T10:04:45.721549Z","created":"2024-05-27T10:02:23.315287Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T10:04:51.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa5c803e2d0 msgr2=0x7fa5c80407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:51.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa5c803e2d0 0x7fa5c80407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa5e81032e0 tx=0x7fa5d4008040 comp rx=0 tx=0).stop 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 msgr2=0x7fa5e8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5e8075ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc01e040 tx=0x7fa5dc01ca60 comp rx=0 tx=0).stop 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 shutdown_connections 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa5c803e2d0 0x7fa5c80407e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8103eb0 0x7fa5e8075ab0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 --2- 172.21.15.143:0/786167015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5e8075ff0 0x7fa5e8076400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.166+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 >> 172.21.15.143:0/786167015 conn(0x7fa5e80ff760 msgr2=0x7fa5e81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.167+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 shutdown_connections 2024-05-27T10:04:51.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:51.167+0000 7fa5efa00700 1 -- 172.21.15.143:0/786167015 wait complete. 2024-05-27T10:04:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 2 2024-05-27T10:04:52.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:51 smithi042 ceph-mon[31661]: Deploying daemon mon.b on smithi104 2024-05-27T10:04:52.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:51 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/786167015' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:04:52.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:52.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:51 smithi143 ceph-mon[35545]: Deploying daemon mon.b on smithi104 2024-05-27T10:04:52.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:51 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/786167015' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:04:52.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:52.997 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T10:04:52.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T10:04:53.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:52 smithi042 ceph-mon[31661]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:53.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:52 smithi143 ceph-mon[35545]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:53.430 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.391+0000 7f2334000700 1 -- 172.21.15.143:0/11450920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c075b50 msgr2=0x7f232c075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.391+0000 7f2334000700 1 --2- 172.21.15.143:0/11450920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c075b50 0x7f232c075f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2320009910 tx=0x7f2320009c20 comp rx=0 tx=0).stop 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.391+0000 7f2334000700 1 -- 172.21.15.143:0/11450920 shutdown_connections 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.391+0000 7f2334000700 1 --2- 172.21.15.143:0/11450920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c075b50 0x7f232c075f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.391+0000 7f2334000700 1 -- 172.21.15.143:0/11450920 >> 172.21.15.143:0/11450920 conn(0x7f232c0ff760 msgr2=0x7f232c101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.392+0000 7f2334000700 1 -- 172.21.15.143:0/11450920 shutdown_connections 2024-05-27T10:04:55.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.392+0000 7f2334000700 1 -- 172.21.15.143:0/11450920 wait complete. 2024-05-27T10:04:55.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.393+0000 7f2334000700 1 Processor -- start 2024-05-27T10:04:55.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.393+0000 7f2334000700 1 -- start start 2024-05-27T10:04:55.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2334000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:55.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2334000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c1a0690 0x7f232c1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:55.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2334000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1146c0 con 0x7f232c1a0690 2024-05-27T10:04:55.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2334000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f232c114540 con 0x7f232c075b50 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59136/0 (socket says 172.21.15.143:59136) 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 -- 172.21.15.143:0/1935912933 learned_addr learned my addr 172.21.15.143:0/1935912933 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 -- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 msgr2=0x7f232c1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 -- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 msgr2=0x7f232c1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2330c00700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c1a0690 0x7f232c1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.394+0000 7f2331600700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:04:55.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.395+0000 7f2330c00700 1 -- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 msgr2=0x7f232c1a0150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:04:55.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.395+0000 7f2330c00700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:55.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.395+0000 7f2330c00700 1 -- 172.21.15.143:0/1935912933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1a0c90 con 0x7f232c1a0690 2024-05-27T10:04:55.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.396+0000 7f2330c00700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c1a0690 0x7f232c1a4980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f231800b4d0 tx=0x7f231800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:55.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.396+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2318011630 con 0x7f232c1a0690 2024-05-27T10:04:55.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.396+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1a4ec0 con 0x7f232c1a0690 2024-05-27T10:04:55.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.397+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2318011c60 con 0x7f232c1a0690 2024-05-27T10:04:55.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.397+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2318010c90 con 0x7f232c1a0690 2024-05-27T10:04:55.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.397+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a5450 con 0x7f232c1a0690 2024-05-27T10:04:55.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.398+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c077290 con 0x7f232c1a0690 2024-05-27T10:04:55.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.399+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2318018020 con 0x7f232c1a0690 2024-05-27T10:04:55.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.399+0000 7f232a000700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f231403e2b0 0x7f23140407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:04:55.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.399+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2318052280 con 0x7f232c1a0690 2024-05-27T10:04:55.399 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:04:55 smithi143 ceph-mon[35545]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.400+0000 7f2331600700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f231403e2b0 0x7f23140407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:04:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.401+0000 7f2331600700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f231403e2b0 0x7f23140407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2320006010 tx=0x7f232001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:04:55.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.417+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f231801f020 con 0x7f232c1a0690 2024-05-27T10:04:55.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:04:55 smithi042 ceph-mon[31661]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:55.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.745+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f232c1a5790 con 0x7f232c1a0690 2024-05-27T10:04:55.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.746+0000 7f232a000700 1 -- 172.21.15.143:0/1935912933 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f231801f200 con 0x7f232c1a0690 2024-05-27T10:04:55.746 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:04:55.747 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","modified":"2024-05-27T10:04:45.721549Z","created":"2024-05-27T10:02:23.315287Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f231403e2b0 msgr2=0x7f23140407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f231403e2b0 0x7f23140407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2320006010 tx=0x7f232001a040 comp rx=0 tx=0).stop 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c1a0690 msgr2=0x7f232c1a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c1a0690 0x7f232c1a4980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f231800b4d0 tx=0x7f231800b890 comp rx=0 tx=0).stop 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 shutdown_connections 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f231403e2b0 0x7f23140407c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f232c075b50 0x7f232c1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 --2- 172.21.15.143:0/1935912933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f232c1a0690 0x7f232c1a4980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:04:55.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 >> 172.21.15.143:0/1935912933 conn(0x7f232c0ff760 msgr2=0x7f232c1009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:04:55.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 shutdown_connections 2024-05-27T10:04:55.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:04:55.750+0000 7f2334000700 1 -- 172.21.15.143:0/1935912933 wait complete. 2024-05-27T10:04:55.751 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 2 2024-05-27T10:04:56.552 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 podman[34794]: 2024-05-27 10:04:56.448886359 +0000 UTC m=+0.377294435 container start 407d38b71fe497a029e9fbc818697d9f7cf528aeb3e45981f24aa80273551ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T10:04:56.552 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 bash[34794]: 407d38b71fe497a029e9fbc818697d9f7cf528aeb3e45981f24aa80273551ab0 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 systemd[1]: Started Ceph mon.b for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev), process ceph-mon, pid 2 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pidfile_write: ignore empty --pid-file 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: load: jerasure load: lrc 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: RocksDB version: 7.9.2 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Git sha 0 2024-05-27T10:04:56.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Compile date 2024-05-21 11:39:56 2024-05-27T10:04:56.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: DB SUMMARY 2024-05-27T10:04:56.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: DB Session ID: ZPUOR4XSGE069IR7C5VG 2024-05-27T10:04:56.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: CURRENT file: CURRENT 2024-05-27T10:04:56.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T10:04:56.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T10:04:56.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-27T10:04:56.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.error_if_exists: 0 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.create_if_missing: 0 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.paranoid_checks: 1 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.env: 0x55881d3f2500 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.info_log: 0x55882a35e040 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.statistics: (nil) 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.use_fsync: 0 2024-05-27T10:04:56.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_log_file_size: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.allow_fallocate: 1 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.use_direct_reads: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.db_log_dir: 2024-05-27T10:04:56.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.wal_dir: 2024-05-27T10:04:56.956 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T10:04:56.956 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T10:04:56.956 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T10:04:56.956 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T10:04:56.956 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.write_buffer_manager: 0x55882a2288c0 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T10:04:56.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.unordered_write: 0 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.row_cache: None 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.wal_filter: None 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.two_write_queues: 0 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.wal_compression: 0 2024-05-27T10:04:56.958 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.atomic_flush: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.log_readahead_size: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_background_jobs: 2 2024-05-27T10:04:56.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_background_compactions: -1 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_subcompactions: 1 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_open_files: -1 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T10:04:56.960 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_background_flushes: -1 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Compression algorithms supported: 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kZSTD supported: 0 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kXpressCompression supported: 0 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kZlibCompression supported: 1 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kSnappyCompression supported: 1 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kLZ4Compression supported: 1 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: kBZip2Compression supported: 0 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T10:04:56.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.merge_operator: 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_filter: None 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55882a35e200) 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks: 1 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T10:04:56.962 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_top_level_index_and_filter: 1 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_type: 0 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_index_type: 0 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_shortening: 1 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: checksum: 4 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: no_block_cache: 0 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache: 0x55882a25b1f0 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_name: BinnedLRUCache 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_options: 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: capacity : 536870912 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_shard_bits : 4 2024-05-27T10:04:56.963 INFO:journalctl@ceph.mon.b.smithi104.stdout: strict_capacity_limit : 0 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: high_pri_pool_ratio: 0.000 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_compressed: (nil) 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: persistent_cache: (nil) 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size: 4096 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size_deviation: 10 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_restart_interval: 16 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_block_restart_interval: 1 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: metadata_block_size: 4096 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: partition_filters: 0 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: use_delta_encoding: 1 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: filter_policy: bloomfilter 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: whole_key_filtering: 1 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: verify_compression: 0 2024-05-27T10:04:56.964 INFO:journalctl@ceph.mon.b.smithi104.stdout: read_amp_bytes_per_bit: 0 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: format_version: 5 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_index_compression: 1 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_align: 0 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: max_auto_readahead_size: 262144 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: prepopulate_block_cache: 0 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: initial_auto_readahead_size: 8192 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression: NoCompression 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T10:04:56.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.num_levels: 7 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T10:04:56.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T10:04:56.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T10:04:56.968 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T10:04:56.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T10:04:56.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T10:04:56.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T10:04:56.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T10:04:56.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T10:04:56.969 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T10:04:56.970 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T10:04:56.970 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T10:04:56.970 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T10:04:56.970 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.inplace_update_support: 0 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.bloom_locality: 0 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.max_successive_merges: 0 2024-05-27T10:04:56.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.ttl: 2592000 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.enable_blob_files: false 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.min_blob_size: 0 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T10:04:56.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5d31d0d9-6735-4d57-8198-de896a7940c8 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804296587874, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804296589274, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716804296, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5d31d0d9-6735-4d57-8198-de896a7940c8", "db_session_id": "ZPUOR4XSGE069IR7C5VG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716804296589679, "job": 1, "event": "recovery_finished"} 2024-05-27T10:04:56.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55882a23fc00 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: DB pointer 0x55882f032000 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** DB Stats ** 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T10:04:56.974 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T10:04:56.975 INFO:journalctl@ceph.mon.b.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T10:04:56.976 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache BinnedLRUCache@0x55882a25b1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: using public_addr v2:172.21.15.104:0/0 -> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: starting mon.b rank -1 at public addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] at bind addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(???) e0 preinit fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).mds e1 new map 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: e1 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: legacy client fscid: -1 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-05-27T10:04:56.977 INFO:journalctl@ceph.mon.b.smithi104.stdout: No filesystems configured 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Deploying cephadm binary to smithi104 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Added host smithi104 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Deploying cephadm binary to smithi143 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Added host smithi143 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/766381553' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/766381553' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:04:56.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c;count:3 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:56.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Deploying daemon mon.c on smithi143 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/1962266006' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.a calling monitor election 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.c calling monitor election 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: monmap epoch 2 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: last_changed 2024-05-27T10:04:45.721549+0000 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: min_mon_release 19 (squid) 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: election_strategy: 1 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: fsmap 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:04:56.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mgrmap e13: x(active, since 70s) 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: overall HEALTH_OK 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: Deploying daemon mon.b on smithi104 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/786167015' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:04:56.986 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:04:56 smithi104 ceph-mon[34812]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-05-27T10:04:57.461 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T10:04:57.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T10:04:57.885 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:05:02.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: mon.a calling monitor election 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: mon.c calling monitor election 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: mon.b calling monitor election 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: monmap epoch 3 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: last_changed 2024-05-27T10:04:56.645779+0000 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: min_mon_release 19 (squid) 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: election_strategy: 1 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: fsmap 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: mgrmap e13: x(active, since 81s) 2024-05-27T10:05:02.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: overall HEALTH_OK 2024-05-27T10:05:02.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: mon.a calling monitor election 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: mon.c calling monitor election 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: mon.b calling monitor election 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: monmap epoch 3 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: last_changed 2024-05-27T10:04:56.645779+0000 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: min_mon_release 19 (squid) 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: election_strategy: 1 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: fsmap 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: mgrmap e13: x(active, since 81s) 2024-05-27T10:05:02.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: overall HEALTH_OK 2024-05-27T10:05:02.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: mon.a calling monitor election 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: mon.c calling monitor election 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: mon.b calling monitor election 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: monmap epoch 3 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: last_changed 2024-05-27T10:04:56.645779+0000 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: created 2024-05-27T10:02:23.315287+0000 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: min_mon_release 19 (squid) 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: election_strategy: 1 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-05-27T10:05:02.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: fsmap 2024-05-27T10:05:02.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T10:05:02.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: mgrmap e13: x(active, since 81s) 2024-05-27T10:05:02.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: overall HEALTH_OK 2024-05-27T10:05:02.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 -- 172.21.15.143:0/636734804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41881060d0 msgr2=0x7f416c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 --2- 172.21.15.143:0/636734804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41881060d0 0x7f416c005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fa0 tx=0x7f417800ffc0 comp rx=0 tx=0).stop 2024-05-27T10:05:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 -- 172.21.15.143:0/636734804 shutdown_connections 2024-05-27T10:05:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 --2- 172.21.15.143:0/636734804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41881060d0 0x7f416c005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 -- 172.21.15.143:0/636734804 >> 172.21.15.143:0/636734804 conn(0x7f4188101940 msgr2=0x7f4188103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 -- 172.21.15.143:0/636734804 shutdown_connections 2024-05-27T10:05:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.885+0000 7f418de00700 1 -- 172.21.15.143:0/636734804 wait complete. 2024-05-27T10:05:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.888+0000 7f418de00700 1 Processor -- start 2024-05-27T10:05:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.888+0000 7f418de00700 1 -- start start 2024-05-27T10:05:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f418de00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f418819c010 0x7f418819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f418de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418819c960 0x7f41881a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f418de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 0x7f41881a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f418de00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f418810dc70 con 0x7f418819c960 2024-05-27T10:05:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f418de00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f418810daf0 con 0x7f418819c010 2024-05-27T10:05:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f418de00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f418810ddf0 con 0x7f41881a0d50 2024-05-27T10:05:02.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f4187400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 0x7f41881a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:02.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f4187400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 0x7f41881a89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59144/0 (socket says 172.21.15.143:59144) 2024-05-27T10:05:02.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.889+0000 7f4187400700 1 -- 172.21.15.143:0/468152924 learned_addr learned my addr 172.21.15.143:0/468152924 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:05:02.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4186000700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418819c960 0x7f41881a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:02.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4186a00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f418819c010 0x7f418819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:02.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4187400700 1 -- 172.21.15.143:0/468152924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f418819c010 msgr2=0x7f418819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:02.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4187400700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f418819c010 0x7f418819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:02.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4187400700 1 -- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418819c960 msgr2=0x7f41881a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:02.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4187400700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418819c960 0x7f41881a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:02.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4187400700 1 -- 172.21.15.143:0/468152924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41881a8f20 con 0x7f41881a0d50 2024-05-27T10:05:02.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.890+0000 7f4187400700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 0x7f41881a89e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f418000ba50 tx=0x7f418000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:02.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.891+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180019070 con 0x7f41881a0d50 2024-05-27T10:05:02.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.891+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f418000cdc0 con 0x7f41881a0d50 2024-05-27T10:05:02.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.891+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180007350 con 0x7f41881a0d50 2024-05-27T10:05:02.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.891+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41881a90f0 con 0x7f41881a0d50 2024-05-27T10:05:02.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.891+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41881a9630 con 0x7f41881a0d50 2024-05-27T10:05:02.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.892+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4188064e10 con 0x7f41881a0d50 2024-05-27T10:05:02.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.893+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f418001c070 con 0x7f41881a0d50 2024-05-27T10:05:02.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.893+0000 7f418ca00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f416803e9a0 0x7f4168040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:02.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.894+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4180051d00 con 0x7f41881a0d50 2024-05-27T10:05:02.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.894+0000 7f4186a00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f416803e9a0 0x7f4168040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:02.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.895+0000 7f4186a00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f416803e9a0 0x7f4168040eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fd0 tx=0x7f4178005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:02.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:02.902+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4180017120 con 0x7f41881a0d50 2024-05-27T10:05:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:02 smithi143 ceph-mon[35545]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:02 smithi042 ceph-mon[31661]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:03.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:03.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:02 smithi104 ceph-mon[34812]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:03.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:03.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:05:03.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.238+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f41881a99a0 con 0x7f41881a0d50 2024-05-27T10:05:03.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.238+0000 7f418ca00700 1 -- 172.21.15.143:0/468152924 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f418001cc60 con 0x7f41881a0d50 2024-05-27T10:05:03.239 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-27T10:05:03.239 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","modified":"2024-05-27T10:04:56.645779Z","created":"2024-05-27T10:02:23.315287Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-27T10:05:03.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f416803e9a0 msgr2=0x7f4168040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f416803e9a0 0x7f4168040eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fd0 tx=0x7f4178005960 comp rx=0 tx=0).stop 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 msgr2=0x7f41881a89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 0x7f41881a89e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f418000ba50 tx=0x7f418000bd60 comp rx=0 tx=0).stop 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 shutdown_connections 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41881a0d50 0x7f41881a89e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f416803e9a0 0x7f4168040eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f418819c010 0x7f418819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 --2- 172.21.15.143:0/468152924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418819c960 0x7f41881a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.241+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 >> 172.21.15.143:0/468152924 conn(0x7f4188101940 msgr2=0x7f4188109a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.242+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 shutdown_connections 2024-05-27T10:05:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:03.242+0000 7f418de00700 1 -- 172.21.15.143:0/468152924 wait complete. 2024-05-27T10:05:03.243 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 3 2024-05-27T10:05:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:03 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/468152924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:05:04.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:03 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/468152924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:05:04.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:03 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/468152924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T10:05:04.537 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-27T10:05:04.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-27T10:05:04.959 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:05:05.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:05.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:05.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:05.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:05:05.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:05:05.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:05 smithi143 ceph-mon[35545]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:05:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:05 smithi104 ceph-mon[34812]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:05:05.451 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:05 smithi042 ceph-mon[31661]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:05:06.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:05:06.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:06.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 -- 172.21.15.42:0/2904430686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc075d20 msgr2=0x7f0fbc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 --2- 172.21.15.42:0/2904430686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc075d20 0x7f0fbc076130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009910 tx=0x7f0fb0009c20 comp rx=0 tx=0).stop 2024-05-27T10:05:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 -- 172.21.15.42:0/2904430686 shutdown_connections 2024-05-27T10:05:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 --2- 172.21.15.42:0/2904430686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc075d20 0x7f0fbc076130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 -- 172.21.15.42:0/2904430686 >> 172.21.15.42:0/2904430686 conn(0x7f0fbc0ff690 msgr2=0x7f0fbc101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 -- 172.21.15.42:0/2904430686 shutdown_connections 2024-05-27T10:05:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.299+0000 7f0fc4800700 1 -- 172.21.15.42:0/2904430686 wait complete. 2024-05-27T10:05:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.300+0000 7f0fc4800700 1 Processor -- start 2024-05-27T10:05:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.300+0000 7f0fc4800700 1 -- start start 2024-05-27T10:05:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc4800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc075d20 0x7f0fbc19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc4800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fbc19c300 0x7f0fbc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc4800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 0x7f0fbc1a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc4800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc110190 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc4800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc110010 con 0x7f0fbc075d20 2024-05-27T10:05:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc4800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc110310 con 0x7f0fbc19c300 2024-05-27T10:05:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc2800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 0x7f0fbc1a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc2800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 0x7f0fbc1a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40702/0 (socket says 172.21.15.42:40702) 2024-05-27T10:05:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.301+0000 7f0fc2800700 1 -- 172.21.15.42:0/4196379662 learned_addr learned my addr 172.21.15.42:0/4196379662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:05:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc1400700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fbc19c300 0x7f0fbc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc1e00700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc075d20 0x7f0fbc19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc2800700 1 -- 172.21.15.42:0/4196379662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fbc19c300 msgr2=0x7f0fbc1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc2800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fbc19c300 0x7f0fbc1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc2800700 1 -- 172.21.15.42:0/4196379662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc075d20 msgr2=0x7f0fbc19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc2800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc075d20 0x7f0fbc19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc2800700 1 -- 172.21.15.42:0/4196379662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc1a88a0 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fc2800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 0x7f0fbc1a0f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800ba50 tx=0x7f0fb800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.302+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8019070 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.303+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc1a8ad0 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.303+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fb8014440 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.303+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80075b0 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.303+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc1a8fc0 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.304+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc064e10 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.304+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0fb801c070 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.305+0000 7f0fb6a00700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0fa403e9a0 0x7f0fa4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:07.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.305+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0fb8051f90 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.305+0000 7f0fc1e00700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0fa403e9a0 0x7f0fa4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:07.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.306+0000 7f0fc1e00700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0fa403e9a0 0x7f0fa4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b5c0 tx=0x7f0fb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:07.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.316+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fb8022170 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:07.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:07.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:07 smithi042 ceph-mon[31661]: Reconfiguring daemon mon.a on smithi042 2024-05-27T10:05:07.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.566+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0fbc1a92b0 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.567+0000 7f0fb6a00700 1 -- 172.21.15.42:0/4196379662 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f0fb8012c40 con 0x7f0fbc1a0b30 2024-05-27T10:05:07.568 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:05:07.568 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-05-27T10:05:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:05:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] 2024-05-27T10:05:07.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0fa403e9a0 msgr2=0x7f0fa4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:07.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0fa403e9a0 0x7f0fa4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b5c0 tx=0x7f0fb0005fb0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 msgr2=0x7f0fbc1a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:07.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 0x7f0fbc1a0f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800ba50 tx=0x7f0fb800be10 comp rx=0 tx=0).stop 2024-05-27T10:05:07.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 shutdown_connections 2024-05-27T10:05:07.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fbc1a0b30 0x7f0fbc1a0f40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0fa403e9a0 0x7f0fa4040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc075d20 0x7f0fbc19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 --2- 172.21.15.42:0/4196379662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fbc19c300 0x7f0fbc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.571+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 >> 172.21.15.42:0/4196379662 conn(0x7f0fbc0ff690 msgr2=0x7f0fbc10c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.572+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 shutdown_connections 2024-05-27T10:05:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:07.572+0000 7f0fc4800700 1 -- 172.21.15.42:0/4196379662 wait complete. 2024-05-27T10:05:07.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:07 smithi143 ceph-mon[35545]: Reconfiguring daemon mon.a on smithi042 2024-05-27T10:05:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:07 smithi104 ceph-mon[34812]: Reconfiguring daemon mon.a on smithi042 2024-05-27T10:05:08.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:08 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4196379662' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:08 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4196379662' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:08.809 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-27T10:05:08.810 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:05:08.810 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T10:05:08.889 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:05:08.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:05:08.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:08 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4196379662' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:09.004 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:05:09.004 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T10:05:09.097 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:05:09.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:05:09.225 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:05:09.225 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T10:05:09.324 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:05:09.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:05:09.461 INFO:tasks.cephadm:Adding mgr.x on smithi042 2024-05-27T10:05:09.461 INFO:tasks.cephadm:Adding mgr.y on smithi104 2024-05-27T10:05:09.461 INFO:tasks.cephadm:Adding mgr.z on smithi143 2024-05-27T10:05:09.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi042=x;smithi104=y;smithi143=z' 2024-05-27T10:05:09.576 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:09 smithi143 ceph-mon[35545]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:09.577 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:09.577 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:09.577 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:09.577 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:09.577 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:09 smithi042 ceph-mon[31661]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:09.928 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:05:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:09 smithi104 ceph-mon[34812]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:10.690 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:10 smithi104 ceph-mon[34812]: Reconfiguring mon.b (monmap changed)... 2024-05-27T10:05:10.690 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:10 smithi104 ceph-mon[34812]: Reconfiguring daemon mon.b on smithi104 2024-05-27T10:05:10.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:10 smithi143 ceph-mon[35545]: Reconfiguring mon.b (monmap changed)... 2024-05-27T10:05:10.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:10 smithi143 ceph-mon[35545]: Reconfiguring daemon mon.b on smithi104 2024-05-27T10:05:10.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:10 smithi042 ceph-mon[31661]: Reconfiguring mon.b (monmap changed)... 2024-05-27T10:05:10.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:10 smithi042 ceph-mon[31661]: Reconfiguring daemon mon.b on smithi104 2024-05-27T10:05:11.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:11 smithi104 ceph-mon[34812]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:11.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:11 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:11.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:11 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:11.756 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:11 smithi143 ceph-mon[35545]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:11.756 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:11 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:11.756 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:11 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:11.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:11 smithi042 ceph-mon[31661]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:11.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:11 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:11.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:11 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:11.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 -- 172.21.15.143:0/365864381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8790102be0 msgr2=0x7f8790102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:11.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 --2- 172.21.15.143:0/365864381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8790102be0 0x7f8790102ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8780009910 tx=0x7f8780009c20 comp rx=0 tx=0).stop 2024-05-27T10:05:11.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 -- 172.21.15.143:0/365864381 shutdown_connections 2024-05-27T10:05:11.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 --2- 172.21.15.143:0/365864381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8790102be0 0x7f8790102ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:11.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 -- 172.21.15.143:0/365864381 >> 172.21.15.143:0/365864381 conn(0x7f87900fe240 msgr2=0x7f87901006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:11.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 -- 172.21.15.143:0/365864381 shutdown_connections 2024-05-27T10:05:11.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.993+0000 7f8794c00700 1 -- 172.21.15.143:0/365864381 wait complete. 2024-05-27T10:05:11.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.994+0000 7f8794c00700 1 Processor -- start 2024-05-27T10:05:11.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 -- start start 2024-05-27T10:05:11.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8790102be0 0x7f879019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:11.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f879019c000 0x7f87901a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87901a14d0 0x7f87901a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8790108a10 con 0x7f879019c000 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8790108890 con 0x7f8790102be0 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.995+0000 7f8794c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8790108b90 con 0x7f87901a14d0 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.996+0000 7f878de00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8790102be0 0x7f879019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.996+0000 7f878d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f879019c000 0x7f87901a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.996+0000 7f878de00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8790102be0 0x7f879019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.143:39564/0 (socket says 172.21.15.143:39564) 2024-05-27T10:05:11.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.996+0000 7f878de00700 1 -- 172.21.15.143:0/4142792678 learned_addr learned my addr 172.21.15.143:0/4142792678 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:05:11.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.996+0000 7f878d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f879019c000 0x7f87901a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.143:43564/0 (socket says 172.21.15.143:43564) 2024-05-27T10:05:11.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.996+0000 7f878e800700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87901a14d0 0x7f87901a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f878e800700 1 -- 172.21.15.143:0/4142792678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8790102be0 msgr2=0x7f879019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f878e800700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8790102be0 0x7f879019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f878e800700 1 -- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f879019c000 msgr2=0x7f87901a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f878e800700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f879019c000 0x7f87901a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f878e800700 1 -- 172.21.15.143:0/4142792678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901a3e60 con 0x7f87901a14d0 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f878e800700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87901a14d0 0x7f87901a38f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f878800bd50 tx=0x7f878800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788019070 con 0x7f87901a14d0 2024-05-27T10:05:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f878800cbb0 con 0x7f87901a14d0 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.997+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87901a4150 con 0x7f87901a14d0 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.998+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880125a0 con 0x7f87901a14d0 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.998+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87901a46c0 con 0x7f87901a14d0 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.999+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87901a4980 con 0x7f87901a14d0 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:11.999+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f878801c070 con 0x7f87901a14d0 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.000+0000 7f8786a00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f877403e9a0 0x7f8774040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:12.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.000+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8788052150 con 0x7f87901a14d0 2024-05-27T10:05:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.001+0000 7f878de00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f877403e9a0 0x7f8774040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.002+0000 7f878de00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f877403e9a0 0x7f8774040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8780009750 tx=0x7f878001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:12.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.012+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87880133a0 con 0x7f87901a14d0 2024-05-27T10:05:12.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.266+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi104=y;smithi143=z", "target": ["mon-mgr", ""]}) -- 0x7f879006c3e0 con 0x7f877403e9a0 2024-05-27T10:05:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.282+0000 7f8786a00700 1 -- 172.21.15.143:0/4142792678 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f879006c3e0 con 0x7f877403e9a0 2024-05-27T10:05:12.283 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mgr update... 2024-05-27T10:05:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f877403e9a0 msgr2=0x7f8774040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f877403e9a0 0x7f8774040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8780009750 tx=0x7f878001a040 comp rx=0 tx=0).stop 2024-05-27T10:05:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87901a14d0 msgr2=0x7f87901a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87901a14d0 0x7f87901a38f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f878800bd50 tx=0x7f878800bd80 comp rx=0 tx=0).stop 2024-05-27T10:05:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 shutdown_connections 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87901a14d0 0x7f87901a38f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f877403e9a0 0x7f8774040eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8790102be0 0x7f879019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 --2- 172.21.15.143:0/4142792678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f879019c000 0x7f87901a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 >> 172.21.15.143:0/4142792678 conn(0x7f87900fe240 msgr2=0x7f879006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 shutdown_connections 2024-05-27T10:05:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:05:12.285+0000 7f8794c00700 1 -- 172.21.15.143:0/4142792678 wait complete. 2024-05-27T10:05:12.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:12 smithi143 ceph-mon[35545]: Reconfiguring mon.c (monmap changed)... 2024-05-27T10:05:12.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:12.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:12.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:12.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:12 smithi143 ceph-mon[35545]: Reconfiguring daemon mon.c on smithi143 2024-05-27T10:05:12.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:12.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:12 smithi042 ceph-mon[31661]: Reconfiguring mon.c (monmap changed)... 2024-05-27T10:05:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:12 smithi042 ceph-mon[31661]: Reconfiguring daemon mon.c on smithi143 2024-05-27T10:05:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:12 smithi104 ceph-mon[34812]: Reconfiguring mon.c (monmap changed)... 2024-05-27T10:05:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T10:05:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T10:05:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:12 smithi104 ceph-mon[34812]: Reconfiguring daemon mon.c on smithi143 2024-05-27T10:05:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:12.992 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.y.service 2024-05-27T10:05:12.999 DEBUG:teuthology.orchestra.run.smithi143:mgr.z> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.z.service 2024-05-27T10:05:13.005 INFO:tasks.cephadm:Deploying OSDs... 2024-05-27T10:05:13.005 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:05:13.005 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T10:05:13.067 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T10:05:13.067 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-05-27T10:05:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T10:05:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:13.124 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-27T10:05:13.124 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:13.124 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:13.124 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-27 10:03:57.606461275 +0000 2024-05-27T10:05:13.125 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-27 09:57:31.183652109 +0000 2024-05-27T10:05:13.125 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-27 09:57:31.183652109 +0000 2024-05-27T10:05:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-27 09:57:31.183652109 +0000 2024-05-27T10:05:13.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T10:05:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-27T10:05:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-27T10:05:13.211 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000300732 s, 1.7 MB/s 2024-05-27T10:05:13.215 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T10:05:13.275 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-05-27T10:05:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T10:05:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:13.327 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-27T10:05:13.327 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:13.327 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:13.328 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-27 10:03:57.702460536 +0000 2024-05-27T10:05:13.328 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-27 09:57:31.803647265 +0000 2024-05-27T10:05:13.328 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-27 09:57:31.803647265 +0000 2024-05-27T10:05:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-27 09:57:31.803647265 +0000 2024-05-27T10:05:13.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T10:05:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-27T10:05:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-27T10:05:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000296804 s, 1.7 MB/s 2024-05-27T10:05:13.407 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T10:05:13.466 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-05-27T10:05:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T10:05:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-27 10:03:57.781459928 +0000 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-27 09:57:32.421642436 +0000 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-27 09:57:32.421642436 +0000 2024-05-27T10:05:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-27 09:57:32.421642436 +0000 2024-05-27T10:05:13.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T10:05:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-27T10:05:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-27T10:05:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000272239 s, 1.9 MB/s 2024-05-27T10:05:13.614 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T10:05:13.674 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-05-27T10:05:13.726 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T10:05:13.726 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-27 10:03:57.844459443 +0000 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-27 09:57:32.991637982 +0000 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-27 09:57:32.991637982 +0000 2024-05-27T10:05:13.727 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-27 09:57:32.991637982 +0000 2024-05-27T10:05:13.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T10:05:13.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:13 smithi042 ceph-mon[31661]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:13.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:13 smithi042 ceph-mon[31661]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi104=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:05:13.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:13 smithi042 ceph-mon[31661]: Saving service mgr spec with placement smithi042=x;smithi104=y;smithi143=z;count:3 2024-05-27T10:05:13.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:13 smithi143 ceph-mon[35545]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:13.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:13 smithi143 ceph-mon[35545]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi104=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:05:13.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:13 smithi143 ceph-mon[35545]: Saving service mgr spec with placement smithi042=x;smithi104=y;smithi143=z;count:3 2024-05-27T10:05:13.806 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-27T10:05:13.806 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-27T10:05:13.806 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000291568 s, 1.8 MB/s 2024-05-27T10:05:13.810 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T10:05:13.863 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-27T10:05:13.863 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T10:05:13.927 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T10:05:13.927 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-05-27T10:05:13.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:13 smithi104 ceph-mon[34812]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:13.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:13 smithi104 ceph-mon[34812]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi104=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:05:13.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:13 smithi104 ceph-mon[34812]: Saving service mgr spec with placement smithi042=x;smithi104=y;smithi143=z;count:3 2024-05-27T10:05:13.979 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-27 10:04:55.650278543 +0000 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-27 09:57:41.934307939 +0000 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-27 09:57:41.934307939 +0000 2024-05-27T10:05:13.980 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-27 09:57:41.934307939 +0000 2024-05-27T10:05:13.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T10:05:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-27T10:05:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-27T10:05:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000383971 s, 1.3 MB/s 2024-05-27T10:05:14.111 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T10:05:14.167 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-27 10:04:55.650278543 +0000 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-27 09:57:42.529301086 +0000 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-27 09:57:42.529301086 +0000 2024-05-27T10:05:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-27 09:57:42.529301086 +0000 2024-05-27T10:05:14.221 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T10:05:14.296 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-27T10:05:14.296 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-27T10:05:14.296 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000348462 s, 1.5 MB/s 2024-05-27T10:05:14.299 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T10:05:14.351 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-27 10:04:55.651278531 +0000 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-27 09:57:43.120294280 +0000 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-27 09:57:43.120294280 +0000 2024-05-27T10:05:14.398 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-27 09:57:43.120294280 +0000 2024-05-27T10:05:14.399 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T10:05:14.523 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-27T10:05:14.523 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-27T10:05:14.523 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000367088 s, 1.4 MB/s 2024-05-27T10:05:14.527 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T10:05:14.583 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-05-27T10:05:14.632 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T10:05:14.632 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-27 10:04:55.652278520 +0000 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-27 09:57:43.711287474 +0000 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-27 09:57:43.711287474 +0000 2024-05-27T10:05:14.633 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-27 09:57:43.711287474 +0000 2024-05-27T10:05:14.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T10:05:14.710 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-27T10:05:14.710 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-27T10:05:14.710 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000316504 s, 1.6 MB/s 2024-05-27T10:05:14.713 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T10:05:14.769 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-27T10:05:14.769 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T10:05:14.822 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T10:05:14.822 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-05-27T10:05:14.866 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T10:05:14.866 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-27 10:04:44.760518236 +0000 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-27 09:57:38.154182333 +0000 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-27 09:57:38.154182333 +0000 2024-05-27T10:05:14.867 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-27 09:57:38.154182333 +0000 2024-05-27T10:05:14.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T10:05:14.970 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-27T10:05:14.970 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-27T10:05:14.970 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000311115 s, 1.6 MB/s 2024-05-27T10:05:14.973 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T10:05:15.016 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-27 10:04:44.761518210 +0000 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-27 09:57:38.720167331 +0000 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-27 09:57:38.720167331 +0000 2024-05-27T10:05:15.105 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-27 09:57:38.720167331 +0000 2024-05-27T10:05:15.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T10:05:15.201 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-27T10:05:15.201 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-27T10:05:15.201 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000275271 s, 1.9 MB/s 2024-05-27T10:05:15.204 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T10:05:15.243 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-05-27T10:05:15.258 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:15.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:15 smithi143 ceph-mon[35545]: Deploying daemon mgr.z on smithi143 2024-05-27T10:05:15.301 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T10:05:15.301 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-27 10:04:44.761518210 +0000 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-27 09:57:39.312151640 +0000 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-27 09:57:39.313151613 +0000 2024-05-27T10:05:15.302 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-27 09:57:39.312151640 +0000 2024-05-27T10:05:15.302 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T10:05:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-27T10:05:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-27T10:05:15.434 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000385043 s, 1.3 MB/s 2024-05-27T10:05:15.438 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T10:05:15.483 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-05-27T10:05:15.572 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T10:05:15.572 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T10:05:15.572 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-27T10:05:15.572 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:05:15.573 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T10:05:15.573 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-27 10:04:44.762518184 +0000 2024-05-27T10:05:15.573 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-27 09:57:39.890136320 +0000 2024-05-27T10:05:15.573 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-27 09:57:39.890136320 +0000 2024-05-27T10:05:15.573 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-27 09:57:39.890136320 +0000 2024-05-27T10:05:15.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:15.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T10:05:15.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:15.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:15.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:15 smithi042 ceph-mon[31661]: Deploying daemon mgr.z on smithi143 2024-05-27T10:05:15.637 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-27T10:05:15.637 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-27T10:05:15.637 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000308423 s, 1.7 MB/s 2024-05-27T10:05:15.640 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T10:05:15.680 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-05-27T10:05:15.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T10:05:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:15 smithi104 ceph-mon[34812]: Deploying daemon mgr.z on smithi143 2024-05-27T10:05:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:05:17.272 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:16 smithi143 systemd[1]: Starting Ceph mgr.z for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:05:17.469 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:17 smithi042 ceph-mon[31661]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:17.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:17 smithi143 ceph-mon[35545]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:17.558 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 podman[37467]: 2024-05-27 10:05:17.254684287 +0000 UTC m=+0.031310685 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:05:17.558 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 podman[37467]: 2024-05-27 10:05:17.373138217 +0000 UTC m=+0.149764651 container create aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T10:05:17.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:17 smithi104 ceph-mon[34812]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:17.914 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 podman[37467]: 2024-05-27 10:05:17.585073959 +0000 UTC m=+0.361700463 container init aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T10:05:17.915 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 podman[37467]: 2024-05-27 10:05:17.596101355 +0000 UTC m=+0.372727761 container start aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T10:05:17.915 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 bash[37467]: aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 2024-05-27T10:05:17.915 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 systemd[1]: Started Ceph mgr.z for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:05:17.915 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 ceph-mgr[37486]: -- 172.21.15.143:0/1476779065 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613c7dd65a0 con 0x5613ccb6a000 2024-05-27T10:05:18.172 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:17.912+0000 7f31535f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:05:18.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:18.170+0000 7f31535f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:05:18.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:18.302+0000 7f31535f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:05:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T10:05:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T10:05:19.050 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-05-27T10:05:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-05-27T10:05:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0358166 s, 293 MB/s 2024-05-27T10:05:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: Deploying daemon mgr.y on smithi104 2024-05-27T10:05:19.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:18 smithi042 ceph-mon[31661]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T10:05:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:19.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:19.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: Deploying daemon mgr.y on smithi104 2024-05-27T10:05:19.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:18 smithi143 ceph-mon[35545]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:19.058 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:18.705+0000 7f31535f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:05:19.058 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:18.819+0000 7f31535f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: Deploying daemon mgr.y on smithi104 2024-05-27T10:05:19.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:18 smithi104 ceph-mon[34812]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:19.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-05-27T10:05:20.319 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:05:20.807 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:20.395+0000 7f31535f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:05:20.807 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:20.623+0000 7f31535f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:05:21.126 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:20 smithi104 systemd[1]: Starting Ceph mgr.y for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:05:21.229 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:20.848+0000 7f31535f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:05:21.229 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:21.227+0000 7f31535f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:05:21.385 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:21 smithi104 ceph-mon[34812]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:21.386 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 podman[36167]: 2024-05-27 10:05:21.027355779 +0000 UTC m=+0.034047369 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:05:21.386 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 podman[36167]: 2024-05-27 10:05:21.12331553 +0000 UTC m=+0.130007127 container create 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-27T10:05:21.386 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 podman[36167]: 2024-05-27 10:05:21.385484951 +0000 UTC m=+0.392176605 container init 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5) 2024-05-27T10:05:21.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:21 smithi042 ceph-mon[31661]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:21.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:21 smithi143 ceph-mon[35545]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:21.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:21.352+0000 7f31535f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:05:21.698 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 podman[36167]: 2024-05-27 10:05:21.397063106 +0000 UTC m=+0.403754729 container start 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T10:05:21.699 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 bash[36167]: 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 2024-05-27T10:05:21.699 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 systemd[1]: Started Ceph mgr.y for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:05:21.699 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 ceph-mgr[36186]: -- 172.21.15.104:0/3516087036 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600f6a465a0 con 0x5600fb7d0000 2024-05-27T10:05:21.869 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:21.609+0000 7f31535f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:05:21.869 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:21.868+0000 7f31535f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:05:21.970 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:21.707+0000 7fd875ac3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:05:21.971 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:21.969+0000 7fd875ac3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:05:22.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 -- 172.21.15.42:0/2861560238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c410a0f0 msgr2=0x7fe5c410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:22.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 --2- 172.21.15.42:0/2861560238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c410a0f0 0x7fe5c410c510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00b0a0 tx=0x7fe5bc00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:05:22.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 -- 172.21.15.42:0/2861560238 shutdown_connections 2024-05-27T10:05:22.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 --2- 172.21.15.42:0/2861560238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c410a0f0 0x7fe5c410c510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:22.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 --2- 172.21.15.42:0/2861560238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4104e30 0x7fe5c4107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:22.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 --2- 172.21.15.42:0/2861560238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4107790 0x7fe5c4109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:22.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 -- 172.21.15.42:0/2861560238 >> 172.21.15.42:0/2861560238 conn(0x7fe5c40fe870 msgr2=0x7fe5c4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:22.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 -- 172.21.15.42:0/2861560238 shutdown_connections 2024-05-27T10:05:22.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.395+0000 7fe5c9000700 1 -- 172.21.15.42:0/2861560238 wait complete. 2024-05-27T10:05:22.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.396+0000 7fe5c9000700 1 Processor -- start 2024-05-27T10:05:22.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 -- start start 2024-05-27T10:05:22.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4104e30 0x7fe5c4104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:22.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 0x7fe5c4102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:22.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c410a0f0 0x7fe5c4103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:22.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410fd00 con 0x7fe5c4107790 2024-05-27T10:05:22.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410fb80 con 0x7fe5c410a0f0 2024-05-27T10:05:22.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.397+0000 7fe5c9000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410fe80 con 0x7fe5c4104e30 2024-05-27T10:05:22.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 0x7fe5c4102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 0x7fe5c4102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60732/0 (socket says 172.21.15.42:60732) 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4104e30 0x7fe5c4104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 -- 172.21.15.42:0/650316938 learned_addr learned my addr 172.21.15.42:0/650316938 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c2800700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c410a0f0 0x7fe5c4103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 -- 172.21.15.42:0/650316938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4104e30 msgr2=0x7fe5c4104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4104e30 0x7fe5c4104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 -- 172.21.15.42:0/650316938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c410a0f0 msgr2=0x7fe5c4103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c410a0f0 0x7fe5c4103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c1400700 1 -- 172.21.15.42:0/650316938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41039e0 con 0x7fe5c4107790 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.398+0000 7fe5c2800700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c410a0f0 0x7fe5c4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:05:22.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.399+0000 7fe5c1400700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 0x7fe5c4102d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000d950 tx=0x7fe5b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:22.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.399+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0016070 con 0x7fe5c4107790 2024-05-27T10:05:22.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.399+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b0009dd0 con 0x7fe5c4107790 2024-05-27T10:05:22.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.399+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000f3c0 con 0x7fe5c4107790 2024-05-27T10:05:22.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.399+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c41a4920 con 0x7fe5c4107790 2024-05-27T10:05:22.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.399+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a4e80 con 0x7fe5c4107790 2024-05-27T10:05:22.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.400+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c402e5e0 con 0x7fe5c4107790 2024-05-27T10:05:22.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.402+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe5b0019070 con 0x7fe5c4107790 2024-05-27T10:05:22.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.402+0000 7fe5baa00700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe5a803e750 0x7fe5a8040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:05:22.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.402+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5b00527e0 con 0x7fe5c4107790 2024-05-27T10:05:22.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.402+0000 7fe5c1e00700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe5a803e750 0x7fe5a8040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:05:22.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.403+0000 7fe5c1e00700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe5a803e750 0x7fe5a8040c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5c406c020 tx=0x7fe5b4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:05:22.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.408+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5b00106a0 con 0x7fe5c4107790 2024-05-27T10:05:22.447 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:22.110+0000 7fd875ac3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:05:22.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:22.669+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe5c4063bf0 con 0x7fe5a803e750 2024-05-27T10:05:22.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:22 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:22 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:22 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.670 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:22 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.671 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:22 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:22.671 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:22 smithi042 ceph-mon[31661]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:22.947 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:22.528+0000 7fd875ac3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:22.648+0000 7fd875ac3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:22 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:22 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:22 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:22 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:22 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:22 smithi104 ceph-mon[34812]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:23.030 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:22.712+0000 7f31535f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:05:23.030 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:23.030+0000 7f31535f2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:05:23.030 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:22 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:23.031 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:22 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:23.031 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:22 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:23.031 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:22 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:23.031 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:22 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:23.031 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:22 smithi143 ceph-mon[35545]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:23 smithi143 ceph-mon[35545]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:05:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:05:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:05:23.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:23.745 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:23.613+0000 7f31535f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:05:23.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:23 smithi042 ceph-mon[31661]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:05:23.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:05:23.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:05:23.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:23.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:23 smithi104 ceph-mon[34812]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:05:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:05:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:05:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:24.057 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:23.744+0000 7f31535f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:05:24.559 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:24.262+0000 7f31535f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:05:24.698 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:24 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:24.312+0000 7fd875ac3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:05:24.698 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:24 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:24.549+0000 7fd875ac3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:05:24.846 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:24.717+0000 7f31535f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:05:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:24 smithi042 ceph-mon[31661]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:25.161 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:24 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:24.772+0000 7fd875ac3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:05:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:24 smithi104 ceph-mon[34812]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:25.180 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:24.846+0000 7f31535f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:05:25.180 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:25.179+0000 7f31535f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:05:25.180 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.180 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.180 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.181 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:25.181 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:24 smithi143 ceph-mon[35545]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:25.447 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:25.160+0000 7fd875ac3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:05:25.448 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:25.284+0000 7fd875ac3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:05:25.500 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:25.368+0000 7f31535f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:05:25.500 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:25.500+0000 7f31535f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:05:25.798 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:25.543+0000 7fd875ac3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:05:25.798 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:25.796+0000 7fd875ac3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:26.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:25 smithi104 ceph-mon[34812]: Reconfiguring daemon mgr.x on smithi042 2024-05-27T10:05:26.307 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:25.925+0000 7f31535f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:26.053+0000 7f31535f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:26.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:25 smithi143 ceph-mon[35545]: Reconfiguring daemon mgr.x on smithi042 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:26.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:25 smithi042 ceph-mon[31661]: Reconfiguring daemon mgr.x on smithi042 2024-05-27T10:05:26.912 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:26.621+0000 7fd875ac3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:05:26.912 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:26.910+0000 7fd875ac3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:05:27.101 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:27 smithi143 ceph-mon[35545]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:27.101 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:26.814+0000 7f31535f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:05:27.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:27 smithi104 ceph-mon[34812]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:27.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:27 smithi042 ceph-mon[31661]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:27.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:05:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:05:27.100+0000 7f31535f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:05:27.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:27.471+0000 7fd875ac3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:05:27.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:27.600+0000 7fd875ac3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:05:28.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:28.081+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fe5b000ad10 con 0x7fe5c4107790 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3261527423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f20f91e-7aa0-41e1-bff3-c8a8c7e24321"}]: dispatch 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3261527423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f20f91e-7aa0-41e1-bff3-c8a8c7e24321"}]': finished 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: Standby manager daemon z started 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T10:05:28.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:28 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:05:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3261527423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f20f91e-7aa0-41e1-bff3-c8a8c7e24321"}]: dispatch 2024-05-27T10:05:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3261527423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f20f91e-7aa0-41e1-bff3-c8a8c7e24321"}]': finished 2024-05-27T10:05:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T10:05:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: Standby manager daemon z started 2024-05-27T10:05:28.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T10:05:28.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:05:28.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T10:05:28.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:28 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:28.064+0000 7fd875ac3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3261527423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f20f91e-7aa0-41e1-bff3-c8a8c7e24321"}]: dispatch 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3261527423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f20f91e-7aa0-41e1-bff3-c8a8c7e24321"}]': finished 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: Standby manager daemon z started 2024-05-27T10:05:28.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T10:05:28.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:05:28.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T10:05:28.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:28 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/4146721479' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:05:28.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:28.515+0000 7fd875ac3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:05:28.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:28.638+0000 7fd875ac3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:05:29.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:29 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2626841539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:05:29.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:29 smithi104 ceph-mon[34812]: mgrmap e14: x(active, since 107s), standbys: z 2024-05-27T10:05:29.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T10:05:29.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:29 smithi104 ceph-mon[34812]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:29.332 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:28.957+0000 7fd875ac3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:05:29.332 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:29.136+0000 7fd875ac3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:05:29.332 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:29.265+0000 7fd875ac3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:05:29.332 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:29 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2626841539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:05:29.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:29 smithi143 ceph-mon[35545]: mgrmap e14: x(active, since 107s), standbys: z 2024-05-27T10:05:29.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T10:05:29.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:29 smithi143 ceph-mon[35545]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:29 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2626841539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:05:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:29 smithi042 ceph-mon[31661]: mgrmap e14: x(active, since 107s), standbys: z 2024-05-27T10:05:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T10:05:29.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:29 smithi042 ceph-mon[31661]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:29.663 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:29.662+0000 7fd875ac3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:05:29.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:29.786+0000 7fd875ac3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:05:30.835 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:30.548+0000 7fd875ac3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:05:30.836 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:05:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:05:30.834+0000 7fd875ac3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:05:31.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:31.308+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe5b0023c50 con 0x7fe5c4107790 2024-05-27T10:05:31.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:31 smithi143 ceph-mon[35545]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:31.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:31 smithi143 ceph-mon[35545]: Standby manager daemon y started 2024-05-27T10:05:31.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:31 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T10:05:31.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:31 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:05:31.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:31 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T10:05:31.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:31 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:05:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:31 smithi042 ceph-mon[31661]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:31 smithi042 ceph-mon[31661]: Standby manager daemon y started 2024-05-27T10:05:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:31 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T10:05:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:31 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:05:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:31 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T10:05:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:31 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:05:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:31 smithi104 ceph-mon[34812]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:31 smithi104 ceph-mon[34812]: Standby manager daemon y started 2024-05-27T10:05:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:31 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T10:05:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:31 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:05:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:31 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T10:05:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:31 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/3463621055' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:05:32.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:32 smithi042 ceph-mon[31661]: mgrmap e15: x(active, since 111s), standbys: z, y 2024-05-27T10:05:32.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:32 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T10:05:32.601 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:32 smithi104 ceph-mon[34812]: mgrmap e15: x(active, since 111s), standbys: z, y 2024-05-27T10:05:32.601 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:32 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T10:05:32.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:32 smithi143 ceph-mon[35545]: mgrmap e15: x(active, since 111s), standbys: z, y 2024-05-27T10:05:32.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:32 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T10:05:33.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:33 smithi042 ceph-mon[31661]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:33.629 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:33 smithi104 ceph-mon[34812]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:33 smithi143 ceph-mon[35545]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:35.145 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:35.145 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:35.145 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:35.145 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:34 smithi042 ceph-mon[31661]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:35.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:34 smithi143 ceph-mon[35545]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:35.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:35.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:35.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:35.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:34 smithi104 ceph-mon[34812]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:36.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:36.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:36.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:36.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:36.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:36.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:36.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:36.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:36.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:37.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:37 smithi042 ceph-mon[31661]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:37.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:37 smithi143 ceph-mon[35545]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:37.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:37 smithi104 ceph-mon[34812]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:38.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:38 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T10:05:38.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:38 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:38.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:38 smithi042 ceph-mon[31661]: Deploying daemon osd.0 on smithi042 2024-05-27T10:05:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:38 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T10:05:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:38 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:38 smithi143 ceph-mon[35545]: Deploying daemon osd.0 on smithi042 2024-05-27T10:05:38.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:38 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T10:05:38.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:38 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:38.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:38 smithi104 ceph-mon[34812]: Deploying daemon osd.0 on smithi042 2024-05-27T10:05:39.286 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:39 smithi042 ceph-mon[31661]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:39.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:39 smithi143 ceph-mon[35545]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:39.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:39 smithi104 ceph-mon[34812]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:41.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:41 smithi143 ceph-mon[35545]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:41.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:41 smithi042 ceph-mon[31661]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:41 smithi104 ceph-mon[34812]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:43.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:43 smithi143 ceph-mon[35545]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:43 smithi042 ceph-mon[31661]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:43.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:43 smithi104 ceph-mon[34812]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:45 smithi143 ceph-mon[35545]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:45.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:45 smithi042 ceph-mon[31661]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:45 smithi104 ceph-mon[34812]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:47.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:47 smithi042 ceph-mon[31661]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:47.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:47 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:47.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:47 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:47.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:47 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:47 smithi104 ceph-mon[34812]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:47 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:47 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:47 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:47 smithi143 ceph-mon[35545]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:47 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:48.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:47 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:48.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:47 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:49.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:49 smithi042 ceph-mon[31661]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:49 smithi104 ceph-mon[34812]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:50.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:49 smithi143 ceph-mon[35545]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:51.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:51.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:51.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:51.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:51 smithi042 ceph-mon[31661]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T10:05:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:51.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:51 smithi104 ceph-mon[34812]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:51.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:51 smithi143 ceph-mon[35545]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T10:05:52.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.436+0000 7fe5baa00700 1 -- 172.21.15.42:0/650316938 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5c4063bf0 con 0x7fe5a803e750 2024-05-27T10:05:52.437 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-05-27T10:05:52.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.439+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe5a803e750 msgr2=0x7fe5a8040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:52.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.439+0000 7fe5c9000700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe5a803e750 0x7fe5a8040c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5c406c020 tx=0x7fe5b4009190 comp rx=0 tx=0).stop 2024-05-27T10:05:52.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.439+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 msgr2=0x7fe5c4102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:05:52.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.439+0000 7fe5c9000700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 0x7fe5c4102d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000d950 tx=0x7fe5b000dc60 comp rx=0 tx=0).stop 2024-05-27T10:05:52.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 shutdown_connections 2024-05-27T10:05:52.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe5a803e750 0x7fe5a8040c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:52.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c410a0f0 0x7fe5c4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:52.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4104e30 0x7fe5c4104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:52.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 --2- 172.21.15.42:0/650316938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5c4107790 0x7fe5c4102d40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:05:52.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 >> 172.21.15.42:0/650316938 conn(0x7fe5c40fe870 msgr2=0x7fe5c41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:05:52.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 shutdown_connections 2024-05-27T10:05:52.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:05:52.440+0000 7fe5c9000700 1 -- 172.21.15.42:0/650316938 wait complete. 2024-05-27T10:05:52.887 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T10:05:52.888 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T10:05:52.888 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:52.888 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:05:52.888 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:52.888 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:52.888 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T10:05:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T10:05:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:05:52.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:52.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:52.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:53.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T10:05:53.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T10:05:53.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:53.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:05:53.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:05:53.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:53.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:53.438 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.0.service 2024-05-27T10:05:53.443 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2024-05-27T10:05:53.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T10:05:53.684 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:53 smithi042 ceph-mon[31661]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:53.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:53 smithi042 ceph-mon[31661]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:05:53.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:53 smithi042 ceph-mon[31661]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T10:05:53.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:53.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:53.884 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:05:53.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:53 smithi104 ceph-mon[34812]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:53 smithi104 ceph-mon[34812]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:05:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:53 smithi104 ceph-mon[34812]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T10:05:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:53 smithi143 ceph-mon[35545]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T10:05:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:53 smithi143 ceph-mon[35545]: from='osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:05:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:53 smithi143 ceph-mon[35545]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T10:05:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:54.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:54 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:05:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:54 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:05:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:54 smithi042 ceph-mon[31661]: osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] boot 2024-05-27T10:05:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:54 smithi042 ceph-mon[31661]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T10:05:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:54 smithi042 ceph-mon[31661]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:54 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:05:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:54 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:05:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:54 smithi104 ceph-mon[34812]: osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] boot 2024-05-27T10:05:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:54 smithi104 ceph-mon[34812]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T10:05:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:54 smithi104 ceph-mon[34812]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:54 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:05:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:54 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:05:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:54 smithi143 ceph-mon[35545]: osd.0 [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] boot 2024-05-27T10:05:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:54 smithi143 ceph-mon[35545]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T10:05:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:05:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:54 smithi143 ceph-mon[35545]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:55.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:55 smithi042 ceph-mon[31661]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T10:05:55.792 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:55 smithi104 ceph-mon[34812]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T10:05:56.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:55 smithi143 ceph-mon[35545]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T10:05:56.718 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T10:05:56.718 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T10:05:56.718 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-27T10:05:56.718 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-27T10:05:56.718 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0353998 s, 296 MB/s 2024-05-27T10:05:56.718 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-27T10:05:56.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:56 smithi104 ceph-mon[34812]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:57.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:56 smithi143 ceph-mon[35545]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:57.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:56 smithi042 ceph-mon[31661]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:57.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-05-27T10:05:57.934 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:05:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:05:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:05:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:05:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:05:59.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:05:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 -- 172.21.15.104:0/3046736710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc107470 msgr2=0x7f38fc109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 --2- 172.21.15.104:0/3046736710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc107470 0x7f38fc109950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009990 tx=0x7f38e8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 -- 172.21.15.104:0/3046736710 shutdown_connections 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 --2- 172.21.15.104:0/3046736710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38fc109fe0 0x7f38fc10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 --2- 172.21.15.104:0/3046736710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc06c2c0 0x7f38fc106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 --2- 172.21.15.104:0/3046736710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc107470 0x7f38fc109950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.158+0000 7f3904400700 1 -- 172.21.15.104:0/3046736710 >> 172.21.15.104:0/3046736710 conn(0x7f38fc0fe870 msgr2=0x7f38fc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.159+0000 7f3904400700 1 -- 172.21.15.104:0/3046736710 shutdown_connections 2024-05-27T10:06:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.159+0000 7f3904400700 1 -- 172.21.15.104:0/3046736710 wait complete. 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 Processor -- start 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 -- start start 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 0x7f38fc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38fc107470 0x7f38fc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc109fe0 0x7f38fc1a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc10fc70 con 0x7f38fc109fe0 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc10faf0 con 0x7f38fc06c2c0 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3904400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc10fdf0 con 0x7f38fc107470 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.160+0000 7f3901a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 0x7f38fc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 0x7f38fc19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58222/0 (socket says 172.21.15.104:58222) 2024-05-27T10:06:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 -- 172.21.15.104:0/2295457482 learned_addr learned my addr 172.21.15.104:0/2295457482 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901000700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38fc107470 0x7f38fc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3902400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc109fe0 0x7f38fc1a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 -- 172.21.15.104:0/2295457482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38fc107470 msgr2=0x7f38fc19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38fc107470 0x7f38fc19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 -- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc109fe0 msgr2=0x7f38fc1a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc109fe0 0x7f38fc1a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 -- 172.21.15.104:0/2295457482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc1a45d0 con 0x7f38fc06c2c0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3902400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc109fe0 0x7f38fc1a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.161+0000 7f3901a00700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 0x7f38fc19ca70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38f000d950 tx=0x7f38f000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.162+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0016070 con 0x7f38fc06c2c0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.162+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38f0010460 con 0x7f38fc06c2c0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.162+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc1a48c0 con 0x7f38fc06c2c0 2024-05-27T10:06:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.162+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f000f620 con 0x7f38fc06c2c0 2024-05-27T10:06:00.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.162+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a4e50 con 0x7f38fc06c2c0 2024-05-27T10:06:00.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.163+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc0512b0 con 0x7f38fc06c2c0 2024-05-27T10:06:00.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.166+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f38f0019070 con 0x7f38fc06c2c0 2024-05-27T10:06:00.168 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.167+0000 7f38fa000700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f38e40b2870 0x7f38e40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:00.168 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.168+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f38f00e0390 con 0x7f38fc06c2c0 2024-05-27T10:06:00.169 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.168+0000 7f3901000700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f38e40b2870 0x7f38e40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:00.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.169+0000 7f3901000700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f38e40b2870 0x7f38e40b4d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006010 tx=0x7f38e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:06:00.172 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.171+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38f00a9a80 con 0x7f38fc06c2c0 2024-05-27T10:06:00.418 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:00.417+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f38fc1a5080 con 0x7f38e40b2870 2024-05-27T10:06:00.419 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:00 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi042 2024-05-27T10:06:00.419 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:00 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi042 to 17094M 2024-05-27T10:06:00.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:00 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi042 2024-05-27T10:06:00.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:00 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi042 to 17094M 2024-05-27T10:06:00.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:00 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi042 2024-05-27T10:06:00.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:00 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi042 to 17094M 2024-05-27T10:06:01.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:01 smithi143 ceph-mon[35545]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:01.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:01 smithi143 ceph-mon[35545]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:06:01.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:06:01.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:06:01.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:01.583 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:01 smithi104 ceph-mon[34812]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:01.583 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:01 smithi104 ceph-mon[34812]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:06:01.583 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:06:01.584 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:06:01.584 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:01.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:01 smithi042 ceph-mon[31661]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:01.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:01 smithi042 ceph-mon[31661]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:06:01.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:06:01.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:06:01.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:03.483 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:03 smithi104 ceph-mon[34812]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:03.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:03 smithi143 ceph-mon[35545]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:03.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:03 smithi042 ceph-mon[31661]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:05.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:05 smithi042 ceph-mon[31661]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:05.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:05 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/4155317550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]: dispatch 2024-05-27T10:06:05.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:05 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]: dispatch 2024-05-27T10:06:05.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:05 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]': finished 2024-05-27T10:06:05.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:05 smithi042 ceph-mon[31661]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T10:06:05.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:05 smithi104 ceph-mon[34812]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:05 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/4155317550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]: dispatch 2024-05-27T10:06:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:05 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]: dispatch 2024-05-27T10:06:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:05 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]': finished 2024-05-27T10:06:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:05 smithi104 ceph-mon[34812]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T10:06:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:05 smithi143 ceph-mon[35545]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:05 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/4155317550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]: dispatch 2024-05-27T10:06:05.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:05 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]: dispatch 2024-05-27T10:06:05.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:05 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3ff39d-4276-43ec-8f9f-41a4b970afaf"}]': finished 2024-05-27T10:06:05.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:05 smithi143 ceph-mon[35545]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T10:06:05.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:06.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:06 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/4223256720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:06:06.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:06 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/4223256720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:06:06.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:06 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/4223256720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:06:07.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:07 smithi042 ceph-mon[31661]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:07.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:07 smithi104 ceph-mon[34812]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:07.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:07 smithi143 ceph-mon[35545]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:09.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:09 smithi042 ceph-mon[31661]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:09.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:09 smithi104 ceph-mon[34812]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:09 smithi143 ceph-mon[35545]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:11.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:11 smithi042 ceph-mon[31661]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:11.637 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:11 smithi104 ceph-mon[34812]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:11.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:11 smithi143 ceph-mon[35545]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:13.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:13 smithi042 ceph-mon[31661]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:13.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:13 smithi104 ceph-mon[34812]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:13.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:13 smithi143 ceph-mon[35545]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:15 smithi104 ceph-mon[34812]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T10:06:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:15 smithi104 ceph-mon[34812]: Deploying daemon osd.1 on smithi104 2024-05-27T10:06:15.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:15 smithi143 ceph-mon[35545]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:15.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T10:06:15.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:15.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:15 smithi143 ceph-mon[35545]: Deploying daemon osd.1 on smithi104 2024-05-27T10:06:15.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:15 smithi042 ceph-mon[31661]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:15.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T10:06:15.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:15.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:15 smithi042 ceph-mon[31661]: Deploying daemon osd.1 on smithi104 2024-05-27T10:06:17.397 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:17 smithi104 ceph-mon[34812]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:17.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:17 smithi143 ceph-mon[35545]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:17.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:17 smithi042 ceph-mon[31661]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:19.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:19 smithi104 ceph-mon[34812]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:19.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:19 smithi143 ceph-mon[35545]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:19.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:19 smithi042 ceph-mon[31661]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:21.646 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:21 smithi042 ceph-mon[31661]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:21.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:21 smithi104 ceph-mon[34812]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:21.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:21 smithi143 ceph-mon[35545]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:23.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:23 smithi042 ceph-mon[31661]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:23.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:23 smithi104 ceph-mon[34812]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:23 smithi143 ceph-mon[35545]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:24.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:24.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:24.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:25.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:25.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:25.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:25.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:25.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:25 smithi104 ceph-mon[34812]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:26.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:25 smithi143 ceph-mon[35545]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:26.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:25 smithi042 ceph-mon[31661]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:27.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:27 smithi104 ceph-mon[34812]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:27.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:27.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:27 smithi143 ceph-mon[35545]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:28.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:27 smithi042 ceph-mon[31661]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:28.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:28.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:28.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:28 smithi104 ceph-mon[34812]: from='osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T10:06:28.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:28 smithi104 ceph-mon[34812]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T10:06:28.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:28.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:06:28.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:28.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:28 smithi104 ceph-mon[34812]: pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:28.899 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2024-05-27T10:06:28.900 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.898+0000 7f38fa000700 1 -- 172.21.15.104:0/2295457482 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38fc1a5080 con 0x7f38e40b2870 2024-05-27T10:06:28.902 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f38e40b2870 msgr2=0x7f38e40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:28.903 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f38e40b2870 0x7f38e40b4d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006010 tx=0x7f38e801a040 comp rx=0 tx=0).stop 2024-05-27T10:06:28.903 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 msgr2=0x7f38fc19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:28.903 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 0x7f38fc19ca70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38f000d950 tx=0x7f38f000dd10 comp rx=0 tx=0).stop 2024-05-27T10:06:28.903 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 shutdown_connections 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38fc109fe0 0x7f38fc1a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f38e40b2870 0x7f38e40b4d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38fc06c2c0 0x7f38fc19ca70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 --2- 172.21.15.104:0/2295457482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38fc107470 0x7f38fc19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 >> 172.21.15.104:0/2295457482 conn(0x7f38fc0fe870 msgr2=0x7f38fc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.903+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 shutdown_connections 2024-05-27T10:06:28.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:06:28.904+0000 7f3904400700 1 -- 172.21.15.104:0/2295457482 wait complete. 2024-05-27T10:06:29.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:28 smithi143 ceph-mon[35545]: from='osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T10:06:29.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:28 smithi143 ceph-mon[35545]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T10:06:29.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:29.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:06:29.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:29.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:28 smithi143 ceph-mon[35545]: pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:29.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:28 smithi042 ceph-mon[31661]: from='osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T10:06:29.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:28 smithi042 ceph-mon[31661]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T10:06:29.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:29.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:06:29.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:29.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:28 smithi042 ceph-mon[31661]: pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:29.741 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.1.service 2024-05-27T10:06:29.746 INFO:tasks.cephadm:Deploying osd.2 on smithi143 with /dev/vg_nvme/lv_4... 2024-05-27T10:06:29.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T10:06:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T10:06:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:06:29.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T10:06:29.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:29.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:06:29.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:29.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:29.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:30.194 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:06:30.448 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:06:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1[39369]: 2024-05-27T10:06:30.293+0000 7fb525000700 -1 osd.1 0 waiting for initial osdmap 2024-05-27T10:06:30.449 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:06:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1[39369]: 2024-05-27T10:06:30.314+0000 7fb51ae00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T10:06:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:30 smithi104 ceph-mon[34812]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:06:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:30 smithi104 ceph-mon[34812]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T10:06:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:30 smithi104 ceph-mon[34812]: pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:30 smithi104 ceph-mon[34812]: from='osd.1 ' entity='osd.1' 2024-05-27T10:06:31.015 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:30 smithi143 ceph-mon[35545]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:06:31.015 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:30 smithi143 ceph-mon[35545]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T10:06:31.015 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.015 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.015 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:30 smithi143 ceph-mon[35545]: pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:31.015 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:30 smithi143 ceph-mon[35545]: from='osd.1 ' entity='osd.1' 2024-05-27T10:06:31.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:30 smithi042 ceph-mon[31661]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:06:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:30 smithi042 ceph-mon[31661]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T10:06:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:30 smithi042 ceph-mon[31661]: pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T10:06:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:30 smithi042 ceph-mon[31661]: from='osd.1 ' entity='osd.1' 2024-05-27T10:06:31.922 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:31 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:06:31.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:31 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:06:31.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:31 smithi143 ceph-mon[35545]: osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] boot 2024-05-27T10:06:31.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:31 smithi143 ceph-mon[35545]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T10:06:31.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:31 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:06:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:31 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:06:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:31 smithi104 ceph-mon[34812]: osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] boot 2024-05-27T10:06:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:31 smithi104 ceph-mon[34812]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T10:06:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:31 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:06:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:31 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:06:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:31 smithi042 ceph-mon[31661]: osd.1 [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] boot 2024-05-27T10:06:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:31 smithi042 ceph-mon[31661]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T10:06:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:06:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T10:06:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T10:06:32.827 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-27T10:06:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-27T10:06:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0395717 s, 265 MB/s 2024-05-27T10:06:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-27T10:06:33.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:33 smithi143 ceph-mon[35545]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:33.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:33 smithi143 ceph-mon[35545]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T10:06:33.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_4 2024-05-27T10:06:33.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:33 smithi042 ceph-mon[31661]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:33.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:33 smithi042 ceph-mon[31661]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T10:06:33.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:33 smithi104 ceph-mon[34812]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:33.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:33 smithi104 ceph-mon[34812]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T10:06:34.005 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:06:35.447 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:35.447 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.447 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:06:35.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:35.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:06:35.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:06:35.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:06:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 -- 172.21.15.143:0/2414950165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8105e00 msgr2=0x7fd0f810c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/2414950165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8105e00 0x7fd0f810c6b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b0a0 tx=0x7fd0f000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 -- 172.21.15.143:0/2414950165 shutdown_connections 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/2414950165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8105e00 0x7fd0f810c6b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/2414950165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0f8104330 0x7fd0f8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/2414950165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0f8105460 0x7fd0f8105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 -- 172.21.15.143:0/2414950165 >> 172.21.15.143:0/2414950165 conn(0x7fd0f80ff9b0 msgr2=0x7fd0f8101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 -- 172.21.15.143:0/2414950165 shutdown_connections 2024-05-27T10:06:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.984+0000 7fd0ffc00700 1 -- 172.21.15.143:0/2414950165 wait complete. 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.985+0000 7fd0ffc00700 1 Processor -- start 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.985+0000 7fd0ffc00700 1 -- start start 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0ffc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 0x7fd0f819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0ffc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0f8105460 0x7fd0f819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0ffc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0f8105e00 0x7fd0f81a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0ffc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f810fca0 con 0x7fd0f8105e00 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0ffc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f810fb20 con 0x7fd0f8105460 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0ffc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f810fe20 con 0x7fd0f8104330 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0fd200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 0x7fd0f819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:35.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0fd200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 0x7fd0f819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41736/0 (socket says 172.21.15.143:41736) 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0fd200700 1 -- 172.21.15.143:0/4243732033 learned_addr learned my addr 172.21.15.143:0/4243732033 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0efe00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0f8105460 0x7fd0f819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.986+0000 7fd0fdc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0f8105e00 0x7fd0f81a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0fd200700 1 -- 172.21.15.143:0/4243732033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0f8105460 msgr2=0x7fd0f819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0fd200700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0f8105460 0x7fd0f819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0fd200700 1 -- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0f8105e00 msgr2=0x7fd0f81a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0fd200700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0f8105e00 0x7fd0f81a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0fd200700 1 -- 172.21.15.143:0/4243732033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f81a22c0 con 0x7fd0f8104330 2024-05-27T10:06:35.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0fd200700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 0x7fd0f819a7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800d950 tx=0x7fd0e800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:06:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8017070 con 0x7fd0f8104330 2024-05-27T10:06:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e8009730 con 0x7fd0f8104330 2024-05-27T10:06:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.987+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f8075a40 con 0x7fd0f8104330 2024-05-27T10:06:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.988+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8003a00 con 0x7fd0f8104330 2024-05-27T10:06:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.988+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f8075f80 con 0x7fd0f8104330 2024-05-27T10:06:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.989+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f80512b0 con 0x7fd0f8104330 2024-05-27T10:06:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.991+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fd0e8003ba0 con 0x7fd0f8104330 2024-05-27T10:06:35.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.993+0000 7fd0f6000700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd0dc0b27f0 0x7fd0dc0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:06:35.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.994+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd0e80e0d30 con 0x7fd0f8104330 2024-05-27T10:06:35.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.994+0000 7fd0efe00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd0dc0b27f0 0x7fd0dc0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:06:35.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.995+0000 7fd0efe00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd0dc0b27f0 0x7fd0dc0b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0f819b8a0 tx=0x7fd0e000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:06:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:35.996+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0e8020030 con 0x7fd0f8104330 2024-05-27T10:06:36.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:06:36.239+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd0f8063cb0 con 0x7fd0dc0b27f0 2024-05-27T10:06:36.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:36 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi104 2024-05-27T10:06:36.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:36 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi104 to 17089M 2024-05-27T10:06:36.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:06:36.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:06:36.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:36.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:36 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi104 2024-05-27T10:06:36.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:36 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi104 to 17089M 2024-05-27T10:06:36.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:06:36.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:06:36.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:36 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi104 2024-05-27T10:06:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:36 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi104 to 17089M 2024-05-27T10:06:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:06:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:06:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:37.481 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:37 smithi143 ceph-mon[35545]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:37.482 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:37 smithi143 ceph-mon[35545]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:06:37.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:37 smithi042 ceph-mon[31661]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:37.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:37 smithi042 ceph-mon[31661]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:06:37.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:37 smithi104 ceph-mon[34812]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:37.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:37 smithi104 ceph-mon[34812]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:06:39.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:39 smithi042 ceph-mon[31661]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:39.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:39 smithi104 ceph-mon[34812]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:39.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:39 smithi143 ceph-mon[35545]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:40.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:40 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/955681886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]: dispatch 2024-05-27T10:06:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:40 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]: dispatch 2024-05-27T10:06:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:40 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]': finished 2024-05-27T10:06:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:40 smithi042 ceph-mon[31661]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T10:06:40.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:06:40.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:40 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/955681886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]: dispatch 2024-05-27T10:06:40.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:40 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]: dispatch 2024-05-27T10:06:40.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:40 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]': finished 2024-05-27T10:06:40.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:40 smithi104 ceph-mon[34812]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T10:06:40.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:06:40.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:40 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/955681886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]: dispatch 2024-05-27T10:06:40.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:40 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]: dispatch 2024-05-27T10:06:40.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:40 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72adfe76-c970-4f68-baea-1d04215301bf"}]': finished 2024-05-27T10:06:40.752 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:40 smithi143 ceph-mon[35545]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T10:06:40.752 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:06:41.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:41 smithi042 ceph-mon[31661]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:41.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:41 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/904321106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:06:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:41 smithi104 ceph-mon[34812]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:41 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/904321106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:06:41.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:41 smithi143 ceph-mon[35545]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:41.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:41 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/904321106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:06:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:43 smithi104 ceph-mon[34812]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:43.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:43 smithi143 ceph-mon[35545]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:43.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:43 smithi042 ceph-mon[31661]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:45 smithi104 ceph-mon[34812]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:45.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:45 smithi143 ceph-mon[35545]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:45 smithi042 ceph-mon[31661]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:47.674 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:47 smithi143 ceph-mon[35545]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:47 smithi104 ceph-mon[34812]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:47 smithi042 ceph-mon[31661]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:49 smithi104 ceph-mon[34812]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T10:06:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:49 smithi143 ceph-mon[35545]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T10:06:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:49.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:49 smithi042 ceph-mon[31661]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:49.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T10:06:49.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:06:50.415 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:50 smithi143 ceph-mon[35545]: Deploying daemon osd.2 on smithi143 2024-05-27T10:06:50.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:50 smithi104 ceph-mon[34812]: Deploying daemon osd.2 on smithi143 2024-05-27T10:06:50.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:50 smithi042 ceph-mon[31661]: Deploying daemon osd.2 on smithi143 2024-05-27T10:06:51.635 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:51 smithi143 ceph-mon[35545]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:51 smithi104 ceph-mon[34812]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:51.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:51 smithi042 ceph-mon[31661]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:53 smithi104 ceph-mon[34812]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:53.752 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:53 smithi143 ceph-mon[35545]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:53.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:53 smithi042 ceph-mon[31661]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:55.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:55 smithi143 ceph-mon[35545]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:55.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:55 smithi104 ceph-mon[34812]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:55.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:55 smithi042 ceph-mon[31661]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:57.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:57 smithi104 ceph-mon[34812]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:57.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:57 smithi143 ceph-mon[35545]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:57.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:57 smithi042 ceph-mon[31661]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:58.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:58 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:58.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:58 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:58.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:58 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:58.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:58 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:58.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:58 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:58.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:58 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:58.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:58 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:58.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:58 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:06:58.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:58 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:06:59.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:06:59 smithi143 ceph-mon[35545]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:59.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:06:59 smithi042 ceph-mon[31661]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:06:59.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:06:59 smithi104 ceph-mon[34812]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:01.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:01 smithi143 ceph-mon[35545]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:02.075 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.075 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:01 smithi104 ceph-mon[34812]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:02.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.076 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.076 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:01 smithi042 ceph-mon[31661]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:02.076 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.076 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.606+0000 7fd0f6000700 1 -- 172.21.15.143:0/4243732033 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd0f8063cb0 con 0x7fd0dc0b27f0 2024-05-27T10:07:02.608 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 2 on host 'smithi143' 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd0dc0b27f0 msgr2=0x7fd0dc0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd0dc0b27f0 0x7fd0dc0b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0f819b8a0 tx=0x7fd0e000a380 comp rx=0 tx=0).stop 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 msgr2=0x7fd0f819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 0x7fd0f819a7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800d950 tx=0x7fd0e800dc60 comp rx=0 tx=0).stop 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 shutdown_connections 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0f8105e00 0x7fd0f81a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f8104330 0x7fd0f819a7f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:02.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd0dc0b27f0 0x7fd0dc0b4d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:02.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.610+0000 7fd0ffc00700 1 --2- 172.21.15.143:0/4243732033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0f8105460 0x7fd0f819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:02.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.611+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 >> 172.21.15.143:0/4243732033 conn(0x7fd0f80ff9b0 msgr2=0x7fd0f8108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:07:02.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.611+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 shutdown_connections 2024-05-27T10:07:02.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:07:02.611+0000 7fd0ffc00700 1 -- 172.21.15.143:0/4243732033 wait complete. 2024-05-27T10:07:02.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:02 smithi143 ceph-mon[35545]: from='osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T10:07:02.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:02 smithi143 ceph-mon[35545]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T10:07:02.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:02.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:02.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:02 smithi042 ceph-mon[31661]: from='osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T10:07:02.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:02 smithi042 ceph-mon[31661]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T10:07:02.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:02.847 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:02.847 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:02.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:02 smithi104 ceph-mon[34812]: from='osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T10:07:02.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:02 smithi104 ceph-mon[34812]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T10:07:02.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:02.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:02.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.344 DEBUG:teuthology.orchestra.run.smithi143:osd.2> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.2.service 2024-05-27T10:07:03.349 INFO:tasks.cephadm:Deploying osd.3 on smithi042 with /dev/vg_nvme/lv_3... 2024-05-27T10:07:03.349 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T10:07:03.814 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:07:03.839 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:07:03.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:03.841 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:03 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2[40751]: 2024-05-27T10:07:03.606+0000 7f84fce00700 -1 osd.2 0 waiting for initial osdmap 2024-05-27T10:07:03.841 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:03 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2[40751]: 2024-05-27T10:07:03.625+0000 7f84f7c00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:07:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:07:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:07:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:03.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:07:03.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:04 smithi042 ceph-mon[31661]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:07:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:04 smithi042 ceph-mon[31661]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T10:07:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:04.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:04 smithi104 ceph-mon[34812]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:07:04.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:04 smithi104 ceph-mon[34812]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T10:07:04.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:04 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:04.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:04 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:04.948 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:04 smithi143 ceph-mon[35545]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:07:04.949 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:04 smithi143 ceph-mon[35545]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T10:07:04.949 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:04 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:04.949 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:04 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:05.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:05 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:07:05.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:05 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:07:05.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:05 smithi042 ceph-mon[31661]: pgmap v109: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:05.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:05 smithi042 ceph-mon[31661]: osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] boot 2024-05-27T10:07:05.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:05 smithi042 ceph-mon[31661]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T10:07:05.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:05.952 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:05 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:07:05.952 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:05 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:07:05.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:05 smithi104 ceph-mon[34812]: pgmap v109: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:05.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:05 smithi104 ceph-mon[34812]: osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] boot 2024-05-27T10:07:05.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:05 smithi104 ceph-mon[34812]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T10:07:05.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:05 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:07:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:05 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:07:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:05 smithi143 ceph-mon[35545]: pgmap v109: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T10:07:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:05 smithi143 ceph-mon[35545]: osd.2 [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] boot 2024-05-27T10:07:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:05 smithi143 ceph-mon[35545]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T10:07:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:07:06.948 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:06 smithi042 ceph-mon[31661]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T10:07:06.949 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T10:07:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T10:07:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T10:07:06.949 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-05-27T10:07:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-05-27T10:07:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0390017 s, 269 MB/s 2024-05-27T10:07:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-05-27T10:07:06.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:06 smithi104 ceph-mon[34812]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T10:07:06.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T10:07:07.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:06 smithi143 ceph-mon[35545]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T10:07:07.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: pgmap v112: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:07.760 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:07.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: pgmap v112: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T10:07:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T10:07:07.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:07.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-05-27T10:07:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: pgmap v112: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:07.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.094 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T10:07:08.094 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.094 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50934]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.095 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50934]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.095 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50938]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T10:07:08.095 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.095 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50938]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.095 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:07:07 smithi042 sudo[50938]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:07:08.448 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:07 smithi104 sudo[43261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T10:07:08.448 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:07 smithi104 sudo[43261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.448 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:07 smithi104 sudo[43261]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.448 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43261]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.449 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43264]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T10:07:08.449 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43264]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.449 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43264]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.449 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43264]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44424]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44424]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44424]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44424]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44427]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.460 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44427]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.461 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44427]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51009]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T10:07:08.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51009]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51009]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51014]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T10:07:08.619 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.619 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51014]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.619 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 sudo[51014]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44430]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T10:07:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44430]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44430]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44430]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44433]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44433]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 sudo[44433]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi143 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi143 to 17089M 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi143 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi143 to 17089M 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43267]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43267]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43267]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43267]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43270]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 sudo[43270]: pam_unix(sudo:session): session closed for user root 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi143 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi143 to 17089M 2024-05-27T10:07:08.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:08.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:08.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:07:08.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:07:08.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:07:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:09 smithi104 ceph-mon[34812]: pgmap v115: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:09 smithi104 ceph-mon[34812]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:09 smithi104 ceph-mon[34812]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:09 smithi104 ceph-mon[34812]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T10:07:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:09 smithi104 ceph-mon[34812]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T10:07:10.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:09 smithi042 ceph-mon[31661]: pgmap v115: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:10.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:09 smithi042 ceph-mon[31661]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:10.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:09 smithi042 ceph-mon[31661]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:10.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:09 smithi042 ceph-mon[31661]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T10:07:10.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:09 smithi042 ceph-mon[31661]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T10:07:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:09 smithi143 ceph-mon[35545]: pgmap v115: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:09 smithi143 ceph-mon[35545]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T10:07:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:09 smithi143 ceph-mon[35545]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T10:07:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:09 smithi143 ceph-mon[35545]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T10:07:10.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:09 smithi143 ceph-mon[35545]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T10:07:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 -- 172.21.15.42:0/2881443250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9201072c0 msgr2=0x7fe9201097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 --2- 172.21.15.42:0/2881443250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9201072c0 0x7fe9201097a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009990 tx=0x7fe90c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 -- 172.21.15.42:0/2881443250 shutdown_connections 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 --2- 172.21.15.42:0/2881443250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe920109e30 0x7fe92010c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 --2- 172.21.15.42:0/2881443250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe92006cd00 0x7fe920106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 --2- 172.21.15.42:0/2881443250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9201072c0 0x7fe9201097a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 -- 172.21.15.42:0/2881443250 >> 172.21.15.42:0/2881443250 conn(0x7fe9200fe6c0 msgr2=0x7fe920100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 -- 172.21.15.42:0/2881443250 shutdown_connections 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.632+0000 7fe925600700 1 -- 172.21.15.42:0/2881443250 wait complete. 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.633+0000 7fe925600700 1 Processor -- start 2024-05-27T10:07:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.633+0000 7fe925600700 1 -- start start 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.633+0000 7fe925600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe92006cd00 0x7fe92019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.633+0000 7fe925600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9201072c0 0x7fe92019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe925600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 0x7fe9201a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe925600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe92010fa80 con 0x7fe920109e30 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe925600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe92010f900 con 0x7fe9201072c0 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe925600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe92010fc00 con 0x7fe92006cd00 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe924c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 0x7fe9201a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe924c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 0x7fe9201a3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43890/0 (socket says 172.21.15.42:43890) 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe924c00700 1 -- 172.21.15.42:0/2080640748 learned_addr learned my addr 172.21.15.42:0/2080640748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe91f400700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9201072c0 0x7fe92019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.634+0000 7fe91fe00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe92006cd00 0x7fe92019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe924c00700 1 -- 172.21.15.42:0/2080640748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe92006cd00 msgr2=0x7fe92019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe924c00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe92006cd00 0x7fe92019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe924c00700 1 -- 172.21.15.42:0/2080640748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9201072c0 msgr2=0x7fe92019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe924c00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9201072c0 0x7fe92019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe924c00700 1 -- 172.21.15.42:0/2080640748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9201a4340 con 0x7fe920109e30 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe91f400700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9201072c0 0x7fe92019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe924c00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 0x7fe9201a3cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe918007ee0 tx=0x7fe91800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91800dbd0 con 0x7fe920109e30 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe91800dd70 con 0x7fe920109e30 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9201a4630 con 0x7fe920109e30 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.635+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180058d0 con 0x7fe920109e30 2024-05-27T10:07:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.636+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201a4b70 con 0x7fe920109e30 2024-05-27T10:07:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.637+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9200512b0 con 0x7fe920109e30 2024-05-27T10:07:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.639+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe91801a020 con 0x7fe920109e30 2024-05-27T10:07:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.641+0000 7fe91cc00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe9040b2900 0x7fe9040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:10.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.641+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7fe9180e0c60 con 0x7fe920109e30 2024-05-27T10:07:10.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.641+0000 7fe91fe00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe9040b2900 0x7fe9040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:10.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.642+0000 7fe91fe00700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe9040b2900 0x7fe9040b4e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe910005950 tx=0x7fe91000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:07:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.644+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe918010070 con 0x7fe920109e30 2024-05-27T10:07:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:10.893+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe92010aaf0 con 0x7fe9040b2900 2024-05-27T10:07:10.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:10 smithi042 ceph-mon[31661]: pgmap v117: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:10.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:10 smithi104 ceph-mon[34812]: pgmap v117: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:11.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:10 smithi143 ceph-mon[35545]: pgmap v117: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:11.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:11 smithi104 ceph-mon[34812]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:07:11.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:11 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:07:11.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:11 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:07:11.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:11 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:11 smithi143 ceph-mon[35545]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:07:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:11 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:07:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:11 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:07:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:11 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:12.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:11 smithi042 ceph-mon[31661]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:07:12.069 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:11 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:07:12.069 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:11 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:07:12.069 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:11 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:12 smithi104 ceph-mon[34812]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:13.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:12 smithi143 ceph-mon[35545]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:12 smithi042 ceph-mon[31661]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:15.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:15 smithi143 ceph-mon[35545]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:15.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:15 smithi042 ceph-mon[31661]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:15 smithi104 ceph-mon[34812]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:16.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:16 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/660393876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11c122a0-b307-4f56-8109-52475745da14"}]: dispatch 2024-05-27T10:07:16.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:16 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/660393876' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11c122a0-b307-4f56-8109-52475745da14"}]': finished 2024-05-27T10:07:16.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:16 smithi143 ceph-mon[35545]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T10:07:16.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:16 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:16.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:16 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/660393876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11c122a0-b307-4f56-8109-52475745da14"}]: dispatch 2024-05-27T10:07:16.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:16 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/660393876' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11c122a0-b307-4f56-8109-52475745da14"}]': finished 2024-05-27T10:07:16.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:16 smithi042 ceph-mon[31661]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T10:07:16.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:16 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:16.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:16 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/660393876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11c122a0-b307-4f56-8109-52475745da14"}]: dispatch 2024-05-27T10:07:16.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:16 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/660393876' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11c122a0-b307-4f56-8109-52475745da14"}]': finished 2024-05-27T10:07:16.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:16 smithi104 ceph-mon[34812]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T10:07:16.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:16 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:17.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:17 smithi042 ceph-mon[31661]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:17.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:17 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2887372970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:07:17.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:17 smithi104 ceph-mon[34812]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:17.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:17 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2887372970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:07:17.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:17 smithi143 ceph-mon[35545]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:17.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:17 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2887372970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:07:19.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:19 smithi042 ceph-mon[31661]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:19.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:19 smithi104 ceph-mon[34812]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:19.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:19 smithi143 ceph-mon[35545]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:21.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:21 smithi042 ceph-mon[31661]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:21.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:21 smithi104 ceph-mon[34812]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:21.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:21 smithi143 ceph-mon[35545]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:23.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:23 smithi042 ceph-mon[31661]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:23.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:23 smithi104 ceph-mon[34812]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:23 smithi143 ceph-mon[35545]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:25.588 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:25 smithi042 ceph-mon[31661]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:25.610 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:25 smithi143 ceph-mon[35545]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:25.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:25 smithi104 ceph-mon[34812]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:26.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T10:07:26.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:26.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T10:07:26.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:26.724 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T10:07:26.724 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:27.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:27 smithi143 ceph-mon[35545]: Deploying daemon osd.3 on smithi042 2024-05-27T10:07:27.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:27 smithi143 ceph-mon[35545]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:27.686 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:27 smithi042 ceph-mon[31661]: Deploying daemon osd.3 on smithi042 2024-05-27T10:07:27.686 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:27 smithi042 ceph-mon[31661]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:27.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:27 smithi104 ceph-mon[34812]: Deploying daemon osd.3 on smithi042 2024-05-27T10:07:27.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:27 smithi104 ceph-mon[34812]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:29.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:29 smithi042 ceph-mon[31661]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:29.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:29 smithi104 ceph-mon[34812]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:29 smithi143 ceph-mon[35545]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:31.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:31 smithi042 ceph-mon[31661]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:31 smithi104 ceph-mon[34812]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:31.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:31 smithi143 ceph-mon[35545]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:33.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:33 smithi042 ceph-mon[31661]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:33.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:33 smithi104 ceph-mon[34812]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:33 smithi143 ceph-mon[35545]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:35.514 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:35 smithi042 ceph-mon[31661]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:35 smithi104 ceph-mon[34812]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:35 smithi143 ceph-mon[35545]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:36.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:36.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:36.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:36.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:36 smithi042 ceph-mon[31661]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:37.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:37.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:37.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:37.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:36 smithi143 ceph-mon[35545]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:36 smithi104 ceph-mon[34812]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:39.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:39 smithi042 ceph-mon[31661]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:39.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:39 smithi104 ceph-mon[34812]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:39.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:39 smithi143 ceph-mon[35545]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:40 smithi143 ceph-mon[35545]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T10:07:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:40 smithi143 ceph-mon[35545]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:41.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:41.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:41.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:40 smithi042 ceph-mon[31661]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T10:07:41.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:40 smithi042 ceph-mon[31661]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:41.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:41.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:41.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:40 smithi104 ceph-mon[34812]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T10:07:41.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:40 smithi104 ceph-mon[34812]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:41.769 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T10:07:41.769 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T10:07:41.769 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:07:42.054 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 3 on host 'smithi042' 2024-05-27T10:07:42.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.051+0000 7fe91cc00700 1 -- 172.21.15.42:0/2080640748 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe92010aaf0 con 0x7fe9040b2900 2024-05-27T10:07:42.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:42.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:42.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:42.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:42.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe9040b2900 msgr2=0x7fe9040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:42.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe9040b2900 0x7fe9040b4e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe910005950 tx=0x7fe91000b410 comp rx=0 tx=0).stop 2024-05-27T10:07:42.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 msgr2=0x7fe9201a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:42.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 0x7fe9201a3cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe918007ee0 tx=0x7fe91800d3b0 comp rx=0 tx=0).stop 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 shutdown_connections 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe920109e30 0x7fe9201a3cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe92006cd00 0x7fe92019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe9040b2900 0x7fe9040b4e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 --2- 172.21.15.42:0/2080640748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9201072c0 0x7fe92019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.056+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 >> 172.21.15.42:0/2080640748 conn(0x7fe9200fe6c0 msgr2=0x7fe920100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.057+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 shutdown_connections 2024-05-27T10:07:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:07:42.057+0000 7fe925600700 1 -- 172.21.15.42:0/2080640748 wait complete. 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.049 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:07:43.049 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T10:07:43.049 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.049 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.050 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.050 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.050 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:43.050 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:42 smithi042 ceph-mon[31661]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:43.053 DEBUG:teuthology.orchestra.run.smithi042:osd.3> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.3.service 2024-05-27T10:07:43.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:07:43.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T10:07:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:42 smithi143 ceph-mon[35545]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:43.060 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-05-27T10:07:43.060 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T10:07:43.133 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: from='osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:07:43.133 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T10:07:43.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:43.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:07:43.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:42 smithi104 ceph-mon[34812]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T10:07:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:07:43.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:43 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:07:43.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:43 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:07:43.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:43.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:43 smithi104 ceph-mon[34812]: osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] boot 2024-05-27T10:07:43.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:43 smithi104 ceph-mon[34812]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T10:07:43.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:43 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:07:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:43 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:07:44.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:44.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:43 smithi143 ceph-mon[35545]: osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] boot 2024-05-27T10:07:44.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:43 smithi143 ceph-mon[35545]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T10:07:44.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:44.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:43 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:07:44.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:43 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:07:44.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:44.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:43 smithi042 ceph-mon[31661]: osd.3 [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] boot 2024-05-27T10:07:44.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:43 smithi042 ceph-mon[31661]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T10:07:44.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:07:45.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:44 smithi042 ceph-mon[31661]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T10:07:45.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:44 smithi042 ceph-mon[31661]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:45.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:44 smithi143 ceph-mon[35545]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T10:07:45.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:44 smithi143 ceph-mon[35545]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:45.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:44 smithi104 ceph-mon[34812]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T10:07:45.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:44 smithi104 ceph-mon[34812]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:45 smithi143 ceph-mon[35545]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T10:07:46.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:45 smithi042 ceph-mon[31661]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T10:07:46.161 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:45 smithi104 ceph-mon[34812]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T10:07:46.477 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T10:07:46.478 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T10:07:46.478 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-27T10:07:46.478 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-27T10:07:46.478 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0351678 s, 298 MB/s 2024-05-27T10:07:46.478 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-27T10:07:47.077 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:46 smithi104 ceph-mon[34812]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:47.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:46 smithi042 ceph-mon[31661]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:47.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:46 smithi143 ceph-mon[35545]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:47.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-05-27T10:07:47.784 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:07:49.640 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi042 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi042 to 8547M 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:49.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi042 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi042 to 8547M 2024-05-27T10:07:49.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:49.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:49.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi042 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi042 to 8547M 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:07:49.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:07:49.974 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.973+0000 7f9e2e000700 1 -- 172.21.15.104:0/1365010795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e281074b0 msgr2=0x7f9e281078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:49.974 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.973+0000 7f9e2e000700 1 --2- 172.21.15.104:0/1365010795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e281074b0 0x7f9e281078c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009990 tx=0x7f9e14009ca0 comp rx=0 tx=0).stop 2024-05-27T10:07:49.974 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.973+0000 7f9e2e000700 1 -- 172.21.15.104:0/1365010795 shutdown_connections 2024-05-27T10:07:49.974 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.973+0000 7f9e2e000700 1 --2- 172.21.15.104:0/1365010795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28107e50 0x7f9e2810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:49.975 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.974+0000 7f9e2e000700 1 --2- 172.21.15.104:0/1365010795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e28106380 0x7f9e28106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:49.975 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.974+0000 7f9e2e000700 1 --2- 172.21.15.104:0/1365010795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e281074b0 0x7f9e281078c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:49.975 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.974+0000 7f9e2e000700 1 -- 172.21.15.104:0/1365010795 >> 172.21.15.104:0/1365010795 conn(0x7f9e28101a00 msgr2=0x7f9e28103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:07:49.975 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.974+0000 7f9e2e000700 1 -- 172.21.15.104:0/1365010795 shutdown_connections 2024-05-27T10:07:49.975 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.974+0000 7f9e2e000700 1 -- 172.21.15.104:0/1365010795 wait complete. 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 Processor -- start 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 -- start start 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28106380 0x7f9e2819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e281074b0 0x7f9e2819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 0x7f9e281a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28111cf0 con 0x7f9e28106380 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28111b70 con 0x7f9e28107e50 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.975+0000 7f9e2e000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28111e70 con 0x7f9e281074b0 2024-05-27T10:07:49.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 0x7f9e281a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 0x7f9e281a3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50596/0 (socket says 172.21.15.104:50596) 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 -- 172.21.15.104:0/200436307 learned_addr learned my addr 172.21.15.104:0/200436307 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2cc00700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28106380 0x7f9e2819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e27e00700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e281074b0 0x7f9e2819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 -- 172.21.15.104:0/200436307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e281074b0 msgr2=0x7f9e2819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e281074b0 0x7f9e2819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 -- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28106380 msgr2=0x7f9e2819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28106380 0x7f9e2819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2d600700 1 -- 172.21.15.104:0/200436307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e281a43c0 con 0x7f9e28107e50 2024-05-27T10:07:49.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e27e00700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e281074b0 0x7f9e2819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.976+0000 7f9e2cc00700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28106380 0x7f9e2819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.977+0000 7f9e2d600700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 0x7f9e281a3d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e20007ee0 tx=0x7f9e2000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.977+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000dbd0 con 0x7f9e28107e50 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.977+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e2000dd70 con 0x7f9e28107e50 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.977+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e281a46b0 con 0x7f9e28107e50 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.977+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200058d0 con 0x7f9e28107e50 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.977+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e281a4ba0 con 0x7f9e28107e50 2024-05-27T10:07:49.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.979+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e280512b0 con 0x7f9e28107e50 2024-05-27T10:07:49.982 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.983+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9e2001a020 con 0x7f9e28107e50 2024-05-27T10:07:49.984 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.984+0000 7f9e25600700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f9e100b2880 0x7f9e100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:07:49.984 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.985+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f9e200e0f90 con 0x7f9e28107e50 2024-05-27T10:07:49.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.985+0000 7f9e2cc00700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f9e100b2880 0x7f9e100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:07:49.986 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.986+0000 7f9e2cc00700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f9e100b2880 0x7f9e100b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005950 tx=0x7f9e1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:07:49.987 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:49.988+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e20010070 con 0x7f9e28107e50 2024-05-27T10:07:50.228 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:07:50.228+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9e28063bf0 con 0x7f9e100b2880 2024-05-27T10:07:50.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:07:50.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:07:50.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:07:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:07:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:50.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:07:50.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:07:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:07:51.738 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:51 smithi042 ceph-mon[31661]: from='client.34217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:07:51.738 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:51 smithi042 ceph-mon[31661]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-27T10:07:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:51 smithi143 ceph-mon[35545]: from='client.34217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:07:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:51 smithi143 ceph-mon[35545]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-27T10:07:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:51 smithi104 ceph-mon[34812]: from='client.34217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:07:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:51 smithi104 ceph-mon[34812]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-27T10:07:53.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:53 smithi143 ceph-mon[35545]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-27T10:07:53.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:53 smithi042 ceph-mon[31661]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-27T10:07:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:53 smithi104 ceph-mon[34812]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-27T10:07:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:55 smithi143 ceph-mon[35545]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:07:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:55 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/3682474508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]: dispatch 2024-05-27T10:07:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:55 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]: dispatch 2024-05-27T10:07:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:55 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]': finished 2024-05-27T10:07:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:55 smithi143 ceph-mon[35545]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T10:07:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:07:55.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:55 smithi042 ceph-mon[31661]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:07:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:55 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/3682474508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]: dispatch 2024-05-27T10:07:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:55 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]: dispatch 2024-05-27T10:07:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:55 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]': finished 2024-05-27T10:07:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:55 smithi042 ceph-mon[31661]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T10:07:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:07:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:55 smithi104 ceph-mon[34812]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:07:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:55 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/3682474508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]: dispatch 2024-05-27T10:07:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:55 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]: dispatch 2024-05-27T10:07:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:55 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78330de0-74fb-49b7-8de6-d694077e8ea8"}]': finished 2024-05-27T10:07:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:55 smithi104 ceph-mon[34812]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T10:07:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:07:56.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:56 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/189732693' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:07:56.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:56 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/189732693' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:07:56.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:56 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/189732693' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:07:57.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:57 smithi143 ceph-mon[35545]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:07:57.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:57 smithi042 ceph-mon[31661]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:07:57.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:57 smithi104 ceph-mon[34812]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:07:59.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:07:59 smithi143 ceph-mon[35545]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:59.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:07:59 smithi042 ceph-mon[31661]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:07:59.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:07:59 smithi104 ceph-mon[34812]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:01.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:01 smithi104 ceph-mon[34812]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:01.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:01 smithi143 ceph-mon[35545]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:01 smithi042 ceph-mon[31661]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:03.773 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:03 smithi104 ceph-mon[34812]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:03.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:03 smithi143 ceph-mon[35545]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:03 smithi042 ceph-mon[31661]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:04.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:04 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T10:08:04.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:04 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:04.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:04 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T10:08:04.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:04 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:04.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T10:08:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:05 smithi143 ceph-mon[35545]: Deploying daemon osd.4 on smithi104 2024-05-27T10:08:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:05 smithi143 ceph-mon[35545]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:05.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:05 smithi042 ceph-mon[31661]: Deploying daemon osd.4 on smithi104 2024-05-27T10:08:05.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:05 smithi042 ceph-mon[31661]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:05 smithi104 ceph-mon[34812]: Deploying daemon osd.4 on smithi104 2024-05-27T10:08:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:05 smithi104 ceph-mon[34812]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:07.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:07 smithi143 ceph-mon[35545]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:07.823 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:07 smithi104 ceph-mon[34812]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:07.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:07 smithi042 ceph-mon[31661]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:09 smithi143 ceph-mon[35545]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:09.833 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:09 smithi104 ceph-mon[34812]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:09.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:09 smithi042 ceph-mon[31661]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:11.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:11 smithi143 ceph-mon[35545]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:11.824 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:11 smithi104 ceph-mon[34812]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:11.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:11 smithi042 ceph-mon[31661]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:13.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:13 smithi143 ceph-mon[35545]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:13.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:13.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:13.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:13.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:13 smithi042 ceph-mon[31661]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:13.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:13.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:13.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:13.922 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:13 smithi104 ceph-mon[34812]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:13.923 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:13.923 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:13.923 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:15.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:15 smithi042 ceph-mon[31661]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:15.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:15 smithi104 ceph-mon[34812]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:16.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:15 smithi143 ceph-mon[35545]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:17.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:17 smithi104 ceph-mon[34812]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:17.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:17 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:17.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:17 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:17.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:17 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:17.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:17 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:17.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:17 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:17.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:17 smithi042 ceph-mon[31661]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:17.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:17 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:17.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:17 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:17.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:17 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:17.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:17 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:17.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:17 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:17 smithi143 ceph-mon[35545]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:17 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:18.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:17 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:18.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:17 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:18.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:17 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:18.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:17 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:18.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:18 smithi042 ceph-mon[31661]: from='osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T10:08:18.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:18 smithi042 ceph-mon[31661]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T10:08:18.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:18 smithi104 ceph-mon[34812]: from='osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T10:08:18.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:18 smithi104 ceph-mon[34812]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T10:08:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:18 smithi143 ceph-mon[35545]: from='osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T10:08:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:18 smithi143 ceph-mon[35545]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T10:08:19.309 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.306+0000 7f9e25600700 1 -- 172.21.15.104:0/200436307 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e28063bf0 con 0x7f9e100b2880 2024-05-27T10:08:19.482 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.310+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f9e100b2880 msgr2=0x7f9e100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.310+0000 7f9e2e000700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f9e100b2880 0x7f9e100b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005950 tx=0x7f9e1c0058e0 comp rx=0 tx=0).stop 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.310+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 msgr2=0x7f9e281a3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 0x7f9e281a3d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e20007ee0 tx=0x7f9e2000d3b0 comp rx=0 tx=0).stop 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 shutdown_connections 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f9e100b2880 0x7f9e100b4d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e28107e50 0x7f9e281a3d40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e28106380 0x7f9e2819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 --2- 172.21.15.104:0/200436307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e281074b0 0x7f9e2819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 >> 172.21.15.104:0/200436307 conn(0x7f9e28101a00 msgr2=0x7f9e2810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:08:19.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 shutdown_connections 2024-05-27T10:08:19.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:08:19.311+0000 7f9e2e000700 1 -- 172.21.15.104:0/200436307 wait complete. 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:19.861 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:20.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:20.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:20.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:20.425 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.4.service 2024-05-27T10:08:20.433 INFO:tasks.cephadm:Deploying osd.5 on smithi143 with /dev/vg_nvme/lv_3... 2024-05-27T10:08:20.433 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T10:08:20.795 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:20 smithi104 ceph-mon[34812]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:08:20.796 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:20 smithi104 ceph-mon[34812]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T10:08:20.796 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:20.796 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:20.796 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:20 smithi104 ceph-mon[34812]: from='osd.4 ' entity='osd.4' 2024-05-27T10:08:20.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:20 smithi143 ceph-mon[35545]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:08:20.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:20 smithi143 ceph-mon[35545]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T10:08:20.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:20.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:20.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:20 smithi143 ceph-mon[35545]: from='osd.4 ' entity='osd.4' 2024-05-27T10:08:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:08:21.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:20 smithi042 ceph-mon[31661]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:08:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:20 smithi042 ceph-mon[31661]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T10:08:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:20 smithi042 ceph-mon[31661]: from='osd.4 ' entity='osd.4' 2024-05-27T10:08:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:08:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:08:21.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:21.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:21.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] boot 2024-05-27T10:08:21.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T10:08:21.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:21 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:21.996 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:08:21.996 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:08:21.996 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:21.996 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:21.996 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] boot 2024-05-27T10:08:21.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T10:08:21.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:22.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:08:22.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:08:22.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-27T10:08:22.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:22.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: osd.4 [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] boot 2024-05-27T10:08:22.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T10:08:22.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:21 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:08:23.680 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T10:08:23.680 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T10:08:23.680 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-27T10:08:23.680 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-27T10:08:23.680 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0330434 s, 317 MB/s 2024-05-27T10:08:23.680 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-27T10:08:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:23 smithi104 ceph-mon[34812]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:23 smithi104 ceph-mon[34812]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T10:08:24.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:23 smithi143 ceph-mon[35545]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:24.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:23 smithi143 ceph-mon[35545]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T10:08:24.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:23 smithi042 ceph-mon[31661]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:24.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:23 smithi042 ceph-mon[31661]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T10:08:24.538 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_3 2024-05-27T10:08:24.991 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:08:25.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:25 smithi143 ceph-mon[35545]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:25.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:25 smithi104 ceph-mon[34812]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:26.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:25 smithi042 ceph-mon[31661]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:26.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi104 2024-05-27T10:08:26.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:08:26.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:08:26.779 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.780 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:26.780 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:26.780 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi104 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:26.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.013+0000 7f1074600700 1 -- 172.21.15.143:0/1800731805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 msgr2=0x7f106c10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.013+0000 7f1074600700 1 --2- 172.21.15.143:0/1800731805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c10c4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f106400b0a0 tx=0x7f106400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 -- 172.21.15.143:0/1800731805 shutdown_connections 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 --2- 172.21.15.143:0/1800731805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c10c4e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 --2- 172.21.15.143:0/1800731805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f106c06ccb0 0x7f106c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 --2- 172.21.15.143:0/1800731805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f106c1074a0 0x7f106c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 -- 172.21.15.143:0/1800731805 >> 172.21.15.143:0/1800731805 conn(0x7f106c0fe860 msgr2=0x7f106c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 -- 172.21.15.143:0/1800731805 shutdown_connections 2024-05-27T10:08:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.014+0000 7f1074600700 1 -- 172.21.15.143:0/1800731805 wait complete. 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.015+0000 7f1074600700 1 Processor -- start 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.015+0000 7f1074600700 1 -- start start 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1074600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f106c06ccb0 0x7f106c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1074600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f106c1074a0 0x7f106c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1074600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c1a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1074600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f106c10fc80 con 0x7f106c1074a0 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1074600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f106c10fb00 con 0x7f106c06ccb0 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1074600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f106c10fe00 con 0x7f106c109fd0 2024-05-27T10:08:27.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1072600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c1a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1072600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c1a3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55538/0 (socket says 172.21.15.143:55538) 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1072600700 1 -- 172.21.15.143:0/842265007 learned_addr learned my addr 172.21.15.143:0/842265007 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1071c00700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f106c06ccb0 0x7f106c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.016+0000 7f1071200700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f106c1074a0 0x7f106c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1072600700 1 -- 172.21.15.143:0/842265007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f106c06ccb0 msgr2=0x7f106c19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1072600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f106c06ccb0 0x7f106c19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1072600700 1 -- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f106c1074a0 msgr2=0x7f106c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1072600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f106c1074a0 0x7f106c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1072600700 1 -- 172.21.15.143:0/842265007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c1a4520 con 0x7f106c109fd0 2024-05-27T10:08:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1072600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c1a3ea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f10640062a0 tx=0x7f1064004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:08:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106401f030 con 0x7f106c109fd0 2024-05-27T10:08:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1064007ac0 con 0x7f106c109fd0 2024-05-27T10:08:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640047d0 con 0x7f106c109fd0 2024-05-27T10:08:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.017+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c1a47b0 con 0x7f106c109fd0 2024-05-27T10:08:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.018+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c1a4c10 con 0x7f106c109fd0 2024-05-27T10:08:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.019+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c0512b0 con 0x7f106c109fd0 2024-05-27T10:08:27.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.023+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1064003a00 con 0x7f106c109fd0 2024-05-27T10:08:27.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.024+0000 7f106a000700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10540b2880 0x7f10540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:08:27.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.025+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f106400f020 con 0x7f106c109fd0 2024-05-27T10:08:27.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.025+0000 7f1071c00700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10540b2880 0x7f10540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:08:27.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.026+0000 7f1071c00700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10540b2880 0x7f10540b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1060005950 tx=0x7f106000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:08:27.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.028+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1064029020 con 0x7f106c109fd0 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi104 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:27.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:27.280+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f106c10aac0 con 0x7f10540b2880 2024-05-27T10:08:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:27 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi104 to 8544M 2024-05-27T10:08:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:27 smithi104 ceph-mon[34812]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:08:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:08:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:27 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi104 to 8544M 2024-05-27T10:08:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:27 smithi143 ceph-mon[35545]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:08:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:08:28.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:28.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:27 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi104 to 8544M 2024-05-27T10:08:28.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:27 smithi042 ceph-mon[31661]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:28.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:08:28.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:08:28.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:28.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:28 smithi104 ceph-mon[34812]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:08:29.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:28 smithi143 ceph-mon[35545]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:08:29.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:28 smithi042 ceph-mon[31661]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:08:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:29 smithi143 ceph-mon[35545]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:29 smithi104 ceph-mon[34812]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:30.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:29 smithi042 ceph-mon[31661]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:30.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:30 smithi104 ceph-mon[34812]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:31.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:30 smithi143 ceph-mon[35545]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:31.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:30 smithi042 ceph-mon[31661]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:31 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/2480537337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]: dispatch 2024-05-27T10:08:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:31 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]: dispatch 2024-05-27T10:08:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:31 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]': finished 2024-05-27T10:08:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:31 smithi104 ceph-mon[34812]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T10:08:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:31 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/2480537337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]: dispatch 2024-05-27T10:08:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:31 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]: dispatch 2024-05-27T10:08:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:31 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]': finished 2024-05-27T10:08:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:31 smithi143 ceph-mon[35545]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T10:08:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:32.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:31 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/2480537337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]: dispatch 2024-05-27T10:08:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:31 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]: dispatch 2024-05-27T10:08:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:31 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aba47e2-0c53-46c1-bd90-d7cf86d14f3a"}]': finished 2024-05-27T10:08:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:31 smithi042 ceph-mon[31661]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T10:08:32.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:32.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:32 smithi104 ceph-mon[34812]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:32.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:32 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/3885217284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:08:33.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:32 smithi143 ceph-mon[35545]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:33.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:32 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/3885217284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:08:33.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:32 smithi042 ceph-mon[31661]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:33.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:32 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/3885217284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:08:35.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:35 smithi042 ceph-mon[31661]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:35.652 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:35 smithi104 ceph-mon[34812]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:35 smithi143 ceph-mon[35545]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:37.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:37 smithi042 ceph-mon[31661]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:37.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:37 smithi104 ceph-mon[34812]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:37.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:37 smithi143 ceph-mon[35545]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:39.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:39 smithi042 ceph-mon[31661]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:39.659 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:39 smithi143 ceph-mon[35545]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:39.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:39 smithi104 ceph-mon[34812]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:41 smithi104 ceph-mon[34812]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T10:08:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:41.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:41 smithi143 ceph-mon[35545]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:41.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T10:08:41.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:41 smithi042 ceph-mon[31661]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T10:08:41.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:42.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:42 smithi104 ceph-mon[34812]: Deploying daemon osd.5 on smithi143 2024-05-27T10:08:42.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:42 smithi143 ceph-mon[35545]: Deploying daemon osd.5 on smithi143 2024-05-27T10:08:42.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:42 smithi042 ceph-mon[31661]: Deploying daemon osd.5 on smithi143 2024-05-27T10:08:43.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:43 smithi143 ceph-mon[35545]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:43 smithi104 ceph-mon[34812]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:43.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:43 smithi042 ceph-mon[31661]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:45 smithi104 ceph-mon[34812]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:45.760 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:45 smithi143 ceph-mon[35545]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:45 smithi042 ceph-mon[31661]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:47 smithi104 ceph-mon[34812]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:47 smithi143 ceph-mon[35545]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:47 smithi042 ceph-mon[31661]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:49.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:49 smithi143 ceph-mon[35545]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:49 smithi104 ceph-mon[34812]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:49.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:49 smithi042 ceph-mon[31661]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:51.291 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:51.291 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:51.291 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:51.291 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:51 smithi143 ceph-mon[35545]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:51.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:51.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:51.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:51.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:51 smithi042 ceph-mon[31661]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:51 smithi104 ceph-mon[34812]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:53.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:53 smithi042 ceph-mon[31661]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:53 smithi104 ceph-mon[34812]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:53.773 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:53 smithi143 ceph-mon[35545]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T10:08:55.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:54 smithi143 ceph-mon[35545]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T10:08:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:54 smithi042 ceph-mon[31661]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:55.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:08:55.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:08:55.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:55.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T10:08:55.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T10:08:55.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:54 smithi104 ceph-mon[34812]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:55.624 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 5 on host 'smithi143' 2024-05-27T10:08:55.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.623+0000 7f106a000700 1 -- 172.21.15.143:0/842265007 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f106c10aac0 con 0x7f10540b2880 2024-05-27T10:08:55.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.626+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10540b2880 msgr2=0x7f10540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:55.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10540b2880 0x7f10540b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1060005950 tx=0x7f106000a300 comp rx=0 tx=0).stop 2024-05-27T10:08:55.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 msgr2=0x7f106c1a3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:08:55.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c1a3ea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f10640062a0 tx=0x7f1064004f40 comp rx=0 tx=0).stop 2024-05-27T10:08:55.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 shutdown_connections 2024-05-27T10:08:55.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f106c109fd0 0x7f106c1a3ea0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:55.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10540b2880 0x7f10540b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:55.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f106c06ccb0 0x7f106c19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:55.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 --2- 172.21.15.143:0/842265007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f106c1074a0 0x7f106c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:08:55.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 >> 172.21.15.143:0/842265007 conn(0x7f106c0fe860 msgr2=0x7f106c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:08:55.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 shutdown_connections 2024-05-27T10:08:55.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:08:55.627+0000 7f1074600700 1 -- 172.21.15.143:0/842265007 wait complete. 2024-05-27T10:08:56.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:56.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T10:08:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:08:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T10:08:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:08:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:56.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:08:56.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:08:56.430 DEBUG:teuthology.orchestra.run.smithi143:osd.5> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.5.service 2024-05-27T10:08:56.435 INFO:tasks.cephadm:Deploying osd.6 on smithi042 with /dev/vg_nvme/lv_2... 2024-05-27T10:08:56.436 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T10:08:56.668 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:08:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:08:56.511+0000 7f9e24400700 -1 osd.5 0 waiting for initial osdmap 2024-05-27T10:08:56.668 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:08:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:08:56.535+0000 7f9e1a200700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T10:08:56.904 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:08:57.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:56 smithi143 ceph-mon[35545]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:08:57.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:56 smithi143 ceph-mon[35545]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T10:08:57.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:56 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:57.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:56 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:57.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:56 smithi143 ceph-mon[35545]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:57.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:56 smithi143 ceph-mon[35545]: from='osd.5 ' entity='osd.5' 2024-05-27T10:08:57.116 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:56 smithi042 ceph-mon[31661]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:08:57.116 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:56 smithi042 ceph-mon[31661]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T10:08:57.117 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:56 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:57.117 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:56 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:57.117 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:56 smithi042 ceph-mon[31661]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:57.117 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:56 smithi042 ceph-mon[31661]: from='osd.5 ' entity='osd.5' 2024-05-27T10:08:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:56 smithi104 ceph-mon[34812]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:08:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:56 smithi104 ceph-mon[34812]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T10:08:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:56 smithi104 ceph-mon[34812]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-27T10:08:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:56 smithi104 ceph-mon[34812]: from='osd.5 ' entity='osd.5' 2024-05-27T10:08:58.160 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:57 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:08:58.160 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:57 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:08:58.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:58.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:57 smithi042 ceph-mon[31661]: osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] boot 2024-05-27T10:08:58.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:57 smithi042 ceph-mon[31661]: osdmap e37: 6 total, 6 up, 6 in 2024-05-27T10:08:58.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:57 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:08:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:57 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:08:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:57 smithi104 ceph-mon[34812]: osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] boot 2024-05-27T10:08:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:57 smithi104 ceph-mon[34812]: osdmap e37: 6 total, 6 up, 6 in 2024-05-27T10:08:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:58.223 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:57 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:08:58.223 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:57 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:08:58.223 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:58.223 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:57 smithi143 ceph-mon[35545]: osd.5 [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] boot 2024-05-27T10:08:58.223 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:57 smithi143 ceph-mon[35545]: osdmap e37: 6 total, 6 up, 6 in 2024-05-27T10:08:58.223 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:08:59.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:58 smithi104 ceph-mon[34812]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T10:08:59.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:58 smithi104 ceph-mon[34812]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:08:59.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:58 smithi042 ceph-mon[31661]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T10:08:59.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:58 smithi042 ceph-mon[31661]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:08:59.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:58 smithi143 ceph-mon[35545]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T10:08:59.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:58 smithi143 ceph-mon[35545]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:00.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:08:59 smithi104 ceph-mon[34812]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T10:09:00.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:08:59 smithi143 ceph-mon[35545]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T10:09:00.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:08:59 smithi042 ceph-mon[31661]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T10:09:00.424 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T10:09:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T10:09:00.425 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-05-27T10:09:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-05-27T10:09:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0356113 s, 294 MB/s 2024-05-27T10:09:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-05-27T10:09:01.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:00 smithi104 ceph-mon[34812]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:01.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:00 smithi143 ceph-mon[35545]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:01.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:00 smithi042 ceph-mon[31661]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:01.479 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-05-27T10:09:01.926 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:09:02.611 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi143 2024-05-27T10:09:02.612 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.612 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.612 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:02.612 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:02.612 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi143 to 8544M 2024-05-27T10:09:02.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:02.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:02.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi143 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi143 to 8544M 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:02.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi143 2024-05-27T10:09:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi143 to 8544M 2024-05-27T10:09:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:03.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:03 smithi143 ceph-mon[35545]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:09:03.828 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:03 smithi042 ceph-mon[31661]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:09:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:03 smithi104 ceph-mon[34812]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:09:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 -- 172.21.15.42:0/2478467454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0106550 msgr2=0x7fc6c0106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 --2- 172.21.15.42:0/2478467454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0106550 0x7fc6c0106960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009990 tx=0x7fc6b0009ca0 comp rx=0 tx=0).stop 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 -- 172.21.15.42:0/2478467454 shutdown_connections 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 --2- 172.21.15.42:0/2478467454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0108020 0x7fc6c010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 --2- 172.21.15.42:0/2478467454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0106550 0x7fc6c0106960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 --2- 172.21.15.42:0/2478467454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0107680 0x7fc6c0107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 -- 172.21.15.42:0/2478467454 >> 172.21.15.42:0/2478467454 conn(0x7fc6c0101b90 msgr2=0x7fc6c0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 -- 172.21.15.42:0/2478467454 shutdown_connections 2024-05-27T10:09:04.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.481+0000 7fc6c6800700 1 -- 172.21.15.42:0/2478467454 wait complete. 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.482+0000 7fc6c6800700 1 Processor -- start 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 -- start start 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0106550 0x7fc6c019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 0x7fc6c019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0108020 0x7fc6c01a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111e50 con 0x7fc6c0107680 2024-05-27T10:09:04.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111cd0 con 0x7fc6c0106550 2024-05-27T10:09:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6c6800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111fd0 con 0x7fc6c0108020 2024-05-27T10:09:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6bea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 0x7fc6c019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6bea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 0x7fc6c019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52634/0 (socket says 172.21.15.42:52634) 2024-05-27T10:09:04.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.483+0000 7fc6bea00700 1 -- 172.21.15.42:0/53281812 learned_addr learned my addr 172.21.15.42:0/53281812 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:09:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bf400700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0106550 0x7fc6c019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:04.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bfe00700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0108020 0x7fc6c01a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bea00700 1 -- 172.21.15.42:0/53281812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0108020 msgr2=0x7fc6c01a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bea00700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0108020 0x7fc6c01a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bea00700 1 -- 172.21.15.42:0/53281812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0106550 msgr2=0x7fc6c019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bea00700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0106550 0x7fc6c019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bea00700 1 -- 172.21.15.42:0/53281812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c01a45b0 con 0x7fc6c0107680 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bfe00700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0108020 0x7fc6c01a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.484+0000 7fc6bea00700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 0x7fc6c019cf90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00d950 tx=0x7fc6ac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.485+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac016070 con 0x7fc6c0107680 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.485+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6ac010460 con 0x7fc6c0107680 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.485+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01a48a0 con 0x7fc6c0107680 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.485+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00f620 con 0x7fc6c0107680 2024-05-27T10:09:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.485+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01a4e30 con 0x7fc6c0107680 2024-05-27T10:09:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.486+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c00512b0 con 0x7fc6c0107680 2024-05-27T10:09:04.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.490+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc6ac019070 con 0x7fc6c0107680 2024-05-27T10:09:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.491+0000 7fc6b7e00700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6a40b2900 0x7fc6a40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.491+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7fc6ac0e0fb0 con 0x7fc6c0107680 2024-05-27T10:09:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.492+0000 7fc6bf400700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6a40b2900 0x7fc6a40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.493+0000 7fc6bf400700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6a40b2900 0x7fc6a40b4e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0003820 tx=0x7fc6b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:09:04.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.494+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6ac0a9af0 con 0x7fc6c0107680 2024-05-27T10:09:04.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:04.741+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc6c0063bf0 con 0x7fc6a40b2900 2024-05-27T10:09:05.793 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:05 smithi104 ceph-mon[34812]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T10:09:05.793 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:09:05.793 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:09:05.793 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:05.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:05 smithi042 ceph-mon[31661]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T10:09:05.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:09:05.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:09:05.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:05 smithi143 ceph-mon[35545]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T10:09:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:09:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:09:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:06.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:06 smithi143 ceph-mon[35545]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:09:06.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:06 smithi042 ceph-mon[31661]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:09:06.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:06 smithi104 ceph-mon[34812]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:09:07.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:07 smithi042 ceph-mon[31661]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-27T10:09:07.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:07 smithi104 ceph-mon[34812]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-27T10:09:08.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:07 smithi143 ceph-mon[35545]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-27T10:09:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:09 smithi042 ceph-mon[31661]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:09:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:09 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/459871972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc50666-edea-46b9-a6c0-0d8c27799fe6"}]: dispatch 2024-05-27T10:09:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:09 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/459871972' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cc50666-edea-46b9-a6c0-0d8c27799fe6"}]': finished 2024-05-27T10:09:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:09 smithi042 ceph-mon[31661]: osdmap e40: 7 total, 6 up, 7 in 2024-05-27T10:09:09.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:09 smithi104 ceph-mon[34812]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:09:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:09 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/459871972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc50666-edea-46b9-a6c0-0d8c27799fe6"}]: dispatch 2024-05-27T10:09:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:09 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/459871972' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cc50666-edea-46b9-a6c0-0d8c27799fe6"}]': finished 2024-05-27T10:09:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:09 smithi104 ceph-mon[34812]: osdmap e40: 7 total, 6 up, 7 in 2024-05-27T10:09:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:09 smithi143 ceph-mon[35545]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:09:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:09 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/459871972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cc50666-edea-46b9-a6c0-0d8c27799fe6"}]: dispatch 2024-05-27T10:09:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:09 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/459871972' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cc50666-edea-46b9-a6c0-0d8c27799fe6"}]': finished 2024-05-27T10:09:10.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:09 smithi143 ceph-mon[35545]: osdmap e40: 7 total, 6 up, 7 in 2024-05-27T10:09:10.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:10.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:10 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/843061545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:09:10.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:10 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/843061545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:09:11.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:10 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/843061545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:09:11.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:11 smithi042 ceph-mon[31661]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:09:11.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:11 smithi104 ceph-mon[34812]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:09:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:11 smithi143 ceph-mon[35545]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:09:13.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:13 smithi042 ceph-mon[31661]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:13.877 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:13 smithi104 ceph-mon[34812]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:13 smithi143 ceph-mon[35545]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:15.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:15 smithi042 ceph-mon[31661]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:15.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:15 smithi104 ceph-mon[34812]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:16.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:15 smithi143 ceph-mon[35545]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:17.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:17 smithi042 ceph-mon[31661]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:17.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:17 smithi104 ceph-mon[34812]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:17 smithi143 ceph-mon[35545]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:19.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:19 smithi042 ceph-mon[31661]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:19.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:19 smithi104 ceph-mon[34812]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:20.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:19 smithi143 ceph-mon[35545]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:20.896 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T10:09:20.896 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:20.896 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:20 smithi042 ceph-mon[31661]: Deploying daemon osd.6 on smithi042 2024-05-27T10:09:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T10:09:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:20 smithi104 ceph-mon[34812]: Deploying daemon osd.6 on smithi042 2024-05-27T10:09:21.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T10:09:21.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:21.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:20 smithi143 ceph-mon[35545]: Deploying daemon osd.6 on smithi042 2024-05-27T10:09:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:21 smithi104 ceph-mon[34812]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:21.984 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:21 smithi042 ceph-mon[31661]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:22.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:21 smithi143 ceph-mon[35545]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:23.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:23 smithi042 ceph-mon[31661]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:23 smithi104 ceph-mon[34812]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:24.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:23 smithi143 ceph-mon[35545]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:25.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:25 smithi042 ceph-mon[31661]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:25.891 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:25 smithi143 ceph-mon[35545]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:25.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:25 smithi104 ceph-mon[34812]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:27 smithi143 ceph-mon[35545]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:27 smithi104 ceph-mon[34812]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:28.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:27 smithi042 ceph-mon[31661]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:29.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:29 smithi042 ceph-mon[31661]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:29 smithi104 ceph-mon[34812]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:29 smithi143 ceph-mon[35545]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:31.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:09:31.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:30 smithi042 ceph-mon[31661]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:09:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:30 smithi104 ceph-mon[34812]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:09:31.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:30 smithi143 ceph-mon[35545]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:33.422 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:33 smithi042 ceph-mon[31661]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:33.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:33 smithi104 ceph-mon[34812]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:33 smithi143 ceph-mon[35545]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T10:09:35.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:34 smithi042 ceph-mon[31661]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:35.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:35.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:35.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T10:09:35.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:34 smithi104 ceph-mon[34812]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:35.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T10:09:35.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:34 smithi143 ceph-mon[35545]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:36.001 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 6 on host 'smithi042' 2024-05-27T10:09:36.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.000+0000 7fc6b7e00700 1 -- 172.21.15.42:0/53281812 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc6c0063bf0 con 0x7fc6a40b2900 2024-05-27T10:09:36.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.003+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6a40b2900 msgr2=0x7fc6a40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:36.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.003+0000 7fc6c6800700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6a40b2900 0x7fc6a40b4e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0003820 tx=0x7fc6b001a040 comp rx=0 tx=0).stop 2024-05-27T10:09:36.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 msgr2=0x7fc6c019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:36.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 0x7fc6c019cf90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00d950 tx=0x7fc6ac00dd10 comp rx=0 tx=0).stop 2024-05-27T10:09:36.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 shutdown_connections 2024-05-27T10:09:36.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6c0108020 0x7fc6c01a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:36.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6a40b2900 0x7fc6a40b4e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:36.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0106550 0x7fc6c019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:36.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 --2- 172.21.15.42:0/53281812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0107680 0x7fc6c019cf90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:36.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 >> 172.21.15.42:0/53281812 conn(0x7fc6c0101b90 msgr2=0x7fc6c010a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:09:36.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 shutdown_connections 2024-05-27T10:09:36.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:09:36.004+0000 7fc6c6800700 1 -- 172.21.15.42:0/53281812 wait complete. 2024-05-27T10:09:36.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:35 smithi104 ceph-mon[34812]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T10:09:36.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:35 smithi104 ceph-mon[34812]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T10:09:36.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:35 smithi104 ceph-mon[34812]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:09:36.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:36.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:35 smithi143 ceph-mon[35545]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T10:09:36.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:35 smithi143 ceph-mon[35545]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T10:09:36.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:35 smithi143 ceph-mon[35545]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:09:36.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:36.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:35 smithi042 ceph-mon[31661]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T10:09:36.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:35 smithi042 ceph-mon[31661]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T10:09:36.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:35 smithi042 ceph-mon[31661]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:09:36.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:36.869 DEBUG:teuthology.orchestra.run.smithi042:osd.6> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.6.service 2024-05-27T10:09:36.873 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2024-05-27T10:09:36.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T10:09:37.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:09:37.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:37.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:36 smithi042 ceph-mon[31661]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' 2024-05-27T10:09:37.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:36 smithi104 ceph-mon[34812]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' 2024-05-27T10:09:37.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-27T10:09:37.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:36 smithi143 ceph-mon[35545]: from='osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722]' entity='osd.6' 2024-05-27T10:09:37.316 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:09:38.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:37 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:09:38.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:37 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:09:38.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:38.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:37 smithi042 ceph-mon[31661]: osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] boot 2024-05-27T10:09:38.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:37 smithi042 ceph-mon[31661]: osdmap e43: 7 total, 7 up, 7 in 2024-05-27T10:09:38.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:38.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:37 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:09:38.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:37 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:09:38.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:38.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:37 smithi104 ceph-mon[34812]: osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] boot 2024-05-27T10:09:38.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:37 smithi104 ceph-mon[34812]: osdmap e43: 7 total, 7 up, 7 in 2024-05-27T10:09:38.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:37 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:09:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:37 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:09:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:38.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:37 smithi143 ceph-mon[35545]: osd.6 [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] boot 2024-05-27T10:09:38.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:37 smithi143 ceph-mon[35545]: osdmap e43: 7 total, 7 up, 7 in 2024-05-27T10:09:38.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:09:38.959 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:38 smithi042 ceph-mon[31661]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:39.142 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:38 smithi104 ceph-mon[34812]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:39.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:38 smithi143 ceph-mon[35545]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:40.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:39 smithi104 ceph-mon[34812]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T10:09:40.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:39 smithi143 ceph-mon[35545]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T10:09:40.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:39 smithi042 ceph-mon[31661]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T10:09:40.431 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T10:09:40.431 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T10:09:40.431 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-27T10:09:40.431 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-27T10:09:40.431 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0360373 s, 291 MB/s 2024-05-27T10:09:40.431 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-27T10:09:41.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:40 smithi104 ceph-mon[34812]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:41.293 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:40 smithi042 ceph-mon[31661]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:41.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:40 smithi143 ceph-mon[35545]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:41.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-05-27T10:09:41.735 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:09:43.534 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi042 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi042 to 5698M 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:43.535 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi042 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi042 to 5698M 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:43.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi042 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi042 to 5698M 2024-05-27T10:09:43.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:43.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:09:43.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.883+0000 7fd7c8800700 1 -- 172.21.15.104:0/811675631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0104360 msgr2=0x7fd7c0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.883+0000 7fd7c8800700 1 --2- 172.21.15.104:0/811675631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0104360 0x7fd7c0104770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009990 tx=0x7fd7b4009ca0 comp rx=0 tx=0).stop 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 -- 172.21.15.104:0/811675631 shutdown_connections 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 --2- 172.21.15.104:0/811675631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c0105e30 0x7fd7c010c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 --2- 172.21.15.104:0/811675631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0104360 0x7fd7c0104770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 --2- 172.21.15.104:0/811675631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0105490 0x7fd7c01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 -- 172.21.15.104:0/811675631 >> 172.21.15.104:0/811675631 conn(0x7fd7c00ff9c0 msgr2=0x7fd7c0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 -- 172.21.15.104:0/811675631 shutdown_connections 2024-05-27T10:09:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.884+0000 7fd7c8800700 1 -- 172.21.15.104:0/811675631 wait complete. 2024-05-27T10:09:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.885+0000 7fd7c8800700 1 Processor -- start 2024-05-27T10:09:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.885+0000 7fd7c8800700 1 -- start start 2024-05-27T10:09:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.885+0000 7fd7c8800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0104360 0x7fd7c0078560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.885+0000 7fd7c8800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c0105490 0x7fd7c0076c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c8800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0105e30 0x7fd7c0077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c8800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010fc50 con 0x7fd7c0105490 2024-05-27T10:09:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c8800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010fad0 con 0x7fd7c0105e30 2024-05-27T10:09:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c8800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010fdd0 con 0x7fd7c0104360 2024-05-27T10:09:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c5e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0104360 0x7fd7c0078560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c0105490 0x7fd7c0076c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c5e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0104360 0x7fd7c0078560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.104:47334/0 (socket says 172.21.15.104:47334) 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c5e00700 1 -- 172.21.15.104:0/4131772678 learned_addr learned my addr 172.21.15.104:0/4131772678 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.886+0000 7fd7c6800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0105e30 0x7fd7c0077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c6800700 1 -- 172.21.15.104:0/4131772678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0104360 msgr2=0x7fd7c0078560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c6800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0104360 0x7fd7c0078560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c6800700 1 -- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c0105490 msgr2=0x7fd7c0076c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c6800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c0105490 0x7fd7c0076c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:09:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c6800700 1 -- 172.21.15.104:0/4131772678 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c00778b0 con 0x7fd7c0105e30 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c6800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0105e30 0x7fd7c0077150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc007f10 tx=0x7fd7bc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc004db0 con 0x7fd7c0105e30 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7bc009e20 con 0x7fd7c0105e30 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.887+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01a48d0 con 0x7fd7c0105e30 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.888+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc015400 con 0x7fd7c0105e30 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.888+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01a4d60 con 0x7fd7c0105e30 2024-05-27T10:09:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.889+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c00512b0 con 0x7fd7c0105e30 2024-05-27T10:09:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.892+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd7bc005240 con 0x7fd7c0105e30 2024-05-27T10:09:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.893+0000 7fd7baa00700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd7a80b2880 0x7fd7a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:09:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.894+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fd7bc0e16e0 con 0x7fd7c0105e30 2024-05-27T10:09:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.894+0000 7fd7c5e00700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd7a80b2880 0x7fd7a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:09:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.895+0000 7fd7c5e00700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd7a80b2880 0x7fd7a80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400b5c0 tx=0x7fd7b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:09:43.897 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:43.897+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7bc010070 con 0x7fd7c0105e30 2024-05-27T10:09:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:09:44.136+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd7c0063bf0 con 0x7fd7a80b2880 2024-05-27T10:09:44.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:09:44.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:09:44.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:09:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:09:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:44.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:09:44.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:09:44.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:45.377 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:45 smithi104 ceph-mon[34812]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:09:45.377 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:45 smithi104 ceph-mon[34812]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:45.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:45 smithi143 ceph-mon[35545]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:09:45.726 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:45 smithi143 ceph-mon[35545]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:45.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:45 smithi042 ceph-mon[31661]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:09:45.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:45 smithi042 ceph-mon[31661]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:47.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:47 smithi143 ceph-mon[35545]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:47.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:47 smithi042 ceph-mon[31661]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:47 smithi104 ceph-mon[34812]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:49.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:49 smithi042 ceph-mon[31661]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:49.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:49 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/821353019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]: dispatch 2024-05-27T10:09:49.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:49 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]: dispatch 2024-05-27T10:09:49.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:49 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]': finished 2024-05-27T10:09:49.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:49 smithi042 ceph-mon[31661]: osdmap e45: 8 total, 7 up, 8 in 2024-05-27T10:09:49.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:09:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:49 smithi104 ceph-mon[34812]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:49.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:49 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/821353019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]: dispatch 2024-05-27T10:09:49.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:49 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]: dispatch 2024-05-27T10:09:49.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:49 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]': finished 2024-05-27T10:09:49.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:49 smithi104 ceph-mon[34812]: osdmap e45: 8 total, 7 up, 8 in 2024-05-27T10:09:49.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:09:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:49 smithi143 ceph-mon[35545]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:49 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/821353019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]: dispatch 2024-05-27T10:09:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:49 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]: dispatch 2024-05-27T10:09:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:49 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c58d76-682d-48b2-a195-ab8ec1b47d3f"}]': finished 2024-05-27T10:09:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:49 smithi143 ceph-mon[35545]: osdmap e45: 8 total, 7 up, 8 in 2024-05-27T10:09:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:09:50.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:50 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/3265800600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:09:50.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:50 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/3265800600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:09:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:50 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/3265800600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:09:51.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:51 smithi042 ceph-mon[31661]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:51.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:51 smithi104 ceph-mon[34812]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:51 smithi143 ceph-mon[35545]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:53.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:53 smithi042 ceph-mon[31661]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:53 smithi104 ceph-mon[34812]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:53.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:53 smithi143 ceph-mon[35545]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:55.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:55 smithi104 ceph-mon[34812]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:55.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:55 smithi042 ceph-mon[31661]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:55.610 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:55 smithi143 ceph-mon[35545]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:57.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:57 smithi104 ceph-mon[34812]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:57.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:57 smithi143 ceph-mon[35545]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:57.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:57 smithi042 ceph-mon[31661]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:59.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:59 smithi104 ceph-mon[34812]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:59.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T10:09:59.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:09:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:59.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:59 smithi143 ceph-mon[35545]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:59.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T10:09:59.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:09:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:09:59.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:59 smithi042 ceph-mon[31661]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:09:59.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T10:09:59.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:09:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:00.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:00 smithi104 ceph-mon[34812]: Deploying daemon osd.7 on smithi104 2024-05-27T10:10:00.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:00 smithi104 ceph-mon[34812]: overall HEALTH_OK 2024-05-27T10:10:00.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:00 smithi143 ceph-mon[35545]: Deploying daemon osd.7 on smithi104 2024-05-27T10:10:00.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:00 smithi143 ceph-mon[35545]: overall HEALTH_OK 2024-05-27T10:10:00.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:00 smithi042 ceph-mon[31661]: Deploying daemon osd.7 on smithi104 2024-05-27T10:10:00.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:00 smithi042 ceph-mon[31661]: overall HEALTH_OK 2024-05-27T10:10:01.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:01 smithi042 ceph-mon[31661]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:01.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:01 smithi104 ceph-mon[34812]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:01.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:01 smithi143 ceph-mon[35545]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:03.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:03 smithi042 ceph-mon[31661]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:03.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:03 smithi104 ceph-mon[34812]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:03.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:03 smithi143 ceph-mon[35545]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:05 smithi104 ceph-mon[34812]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:05 smithi143 ceph-mon[35545]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:05.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:05 smithi042 ceph-mon[31661]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:07.659 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:07 smithi104 ceph-mon[34812]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:07.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:07 smithi143 ceph-mon[35545]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:07.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:07 smithi042 ceph-mon[31661]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:08.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:08.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:08.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:08.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:08.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:08.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:09.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:09 smithi104 ceph-mon[34812]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:09 smithi143 ceph-mon[35545]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:09.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:09 smithi042 ceph-mon[31661]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:11.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:11 smithi104 ceph-mon[34812]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:11.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:11 smithi143 ceph-mon[35545]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:11.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:11 smithi042 ceph-mon[31661]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:12 smithi042 ceph-mon[31661]: from='osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T10:10:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:12 smithi042 ceph-mon[31661]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T10:10:13.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:12 smithi042 ceph-mon[31661]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:13.143 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.143 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.143 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:12 smithi104 ceph-mon[34812]: from='osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T10:10:13.143 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:12 smithi104 ceph-mon[34812]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T10:10:13.143 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:12 smithi104 ceph-mon[34812]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:13.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:12 smithi143 ceph-mon[35545]: from='osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T10:10:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:12 smithi143 ceph-mon[35545]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T10:10:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:12 smithi143 ceph-mon[35545]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:13.731 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.729+0000 7fd7baa00700 1 -- 172.21.15.104:0/4131772678 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7c0063bf0 con 0x7fd7a80b2880 2024-05-27T10:10:13.731 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-05-27T10:10:13.734 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.733+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd7a80b2880 msgr2=0x7fd7a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:13.734 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.733+0000 7fd7c8800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd7a80b2880 0x7fd7a80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400b5c0 tx=0x7fd7b401a040 comp rx=0 tx=0).stop 2024-05-27T10:10:13.734 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.733+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0105e30 msgr2=0x7fd7c0077150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.733+0000 7fd7c8800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0105e30 0x7fd7c0077150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc007f10 tx=0x7fd7bc00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:10:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 shutdown_connections 2024-05-27T10:10:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd7a80b2880 0x7fd7a80b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7c0105e30 0x7fd7c0077150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0104360 0x7fd7c0078560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 --2- 172.21.15.104:0/4131772678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c0105490 0x7fd7c0076c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:13.736 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 >> 172.21.15.104:0/4131772678 conn(0x7fd7c00ff9c0 msgr2=0x7fd7c01086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:10:13.736 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 shutdown_connections 2024-05-27T10:10:13.736 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:10:13.734+0000 7fd7c8800700 1 -- 172.21.15.104:0/4131772678 wait complete. 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:13.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:14.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:14.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:14.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:14.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:14.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:14.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:14.648 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.7.service 2024-05-27T10:10:14.673 INFO:tasks.cephadm:Deploying osd.8 on smithi143 with /dev/vg_nvme/lv_2... 2024-05-27T10:10:14.673 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T10:10:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:14 smithi143 ceph-mon[35545]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:10:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:14 smithi143 ceph-mon[35545]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T10:10:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:14 smithi143 ceph-mon[35545]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:15.124 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:10:15.184 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:14 smithi104 ceph-mon[34812]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:10:15.184 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:14 smithi104 ceph-mon[34812]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T10:10:15.184 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:15.184 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:15.184 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:14 smithi104 ceph-mon[34812]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:14 smithi042 ceph-mon[31661]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:10:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:14 smithi042 ceph-mon[31661]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T10:10:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:14 smithi042 ceph-mon[31661]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-27T10:10:16.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:15 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:10:16.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:15 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:10:16.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:15 smithi104 ceph-mon[34812]: osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] boot 2024-05-27T10:10:16.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:15 smithi104 ceph-mon[34812]: osdmap e48: 8 total, 8 up, 8 in 2024-05-27T10:10:16.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:16.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:15 smithi104 ceph-mon[34812]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T10:10:16.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:15 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:10:16.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:15 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:10:16.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:15 smithi143 ceph-mon[35545]: osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] boot 2024-05-27T10:10:16.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:15 smithi143 ceph-mon[35545]: osdmap e48: 8 total, 8 up, 8 in 2024-05-27T10:10:16.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:16.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:15 smithi143 ceph-mon[35545]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T10:10:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:15 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:10:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:15 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:10:16.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:15 smithi042 ceph-mon[31661]: osd.7 [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] boot 2024-05-27T10:10:16.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:15 smithi042 ceph-mon[31661]: osdmap e48: 8 total, 8 up, 8 in 2024-05-27T10:10:16.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:10:16.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:15 smithi042 ceph-mon[31661]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T10:10:16.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:16 smithi104 ceph-mon[34812]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:17.134 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:16 smithi143 ceph-mon[35545]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:17.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:16 smithi042 ceph-mon[31661]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:17 smithi143 ceph-mon[35545]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T10:10:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T10:10:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T10:10:18.123 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-27T10:10:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-27T10:10:18.123 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.061888 s, 169 MB/s 2024-05-27T10:10:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-27T10:10:18.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:17 smithi104 ceph-mon[34812]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T10:10:18.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:17 smithi042 ceph-mon[31661]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T10:10:18.959 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_2 2024-05-27T10:10:19.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:18 smithi104 ceph-mon[34812]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:19.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:18 smithi143 ceph-mon[35545]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:19.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:18 smithi042 ceph-mon[31661]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:10:20.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi104 2024-05-27T10:10:20.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi104 to 5696M 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:20.947 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:20 smithi143 ceph-mon[35545]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi104 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi104 to 5696M 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:21.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:21.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:20 smithi042 ceph-mon[31661]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:10:21.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi104 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi104 to 5696M 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:21.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:20 smithi104 ceph-mon[34812]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T10:10:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 -- 172.21.15.143:0/651812784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105c30 msgr2=0x7f776810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 --2- 172.21.15.143:0/651812784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105c30 0x7f776810c4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f776000b0a0 tx=0x7f776000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:10:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 -- 172.21.15.143:0/651812784 shutdown_connections 2024-05-27T10:10:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 --2- 172.21.15.143:0/651812784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105c30 0x7f776810c4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 --2- 172.21.15.143:0/651812784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7768104160 0x7f7768104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 --2- 172.21.15.143:0/651812784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7768105290 0x7f77681056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 -- 172.21.15.143:0/651812784 >> 172.21.15.143:0/651812784 conn(0x7f77680ff7a0 msgr2=0x7f7768101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:10:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 -- 172.21.15.143:0/651812784 shutdown_connections 2024-05-27T10:10:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.517+0000 7f776ca00700 1 -- 172.21.15.143:0/651812784 wait complete. 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.518+0000 7f776ca00700 1 Processor -- start 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.518+0000 7f776ca00700 1 -- start start 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f776ca00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7768104160 0x7f7768077450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f776ca00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 0x7f7768075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f776ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7768105c30 0x7f7768076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f776ca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f776810fae0 con 0x7f7768105c30 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f776ca00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f776810f960 con 0x7f7768104160 2024-05-27T10:10:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f776ca00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f776810fc60 con 0x7f7768105290 2024-05-27T10:10:21.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 0x7f7768075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:21.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 0x7f7768075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44730/0 (socket says 172.21.15.143:44730) 2024-05-27T10:10:21.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 -- 172.21.15.143:0/807291386 learned_addr learned my addr 172.21.15.143:0/807291386 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:10:21.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7767e00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7768105c30 0x7f7768076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:21.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7767400700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7768104160 0x7f7768077450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 -- 172.21.15.143:0/807291386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7768104160 msgr2=0x7f7768077450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7768104160 0x7f7768077450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 -- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7768105c30 msgr2=0x7f7768076040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7768105c30 0x7f7768076040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.519+0000 7f7766a00700 1 -- 172.21.15.143:0/807291386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77680767a0 con 0x7f7768105290 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.520+0000 7f7766a00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 0x7f7768075b00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f775400d950 tx=0x7f775400dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.520+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754016070 con 0x7f7768105290 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.520+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7754010460 con 0x7f7768105290 2024-05-27T10:10:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.520+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775400f620 con 0x7f7768105290 2024-05-27T10:10:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.520+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77681a4750 con 0x7f7768105290 2024-05-27T10:10:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.520+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77681a4be0 con 0x7f7768105290 2024-05-27T10:10:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.521+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77680512b0 con 0x7f7768105290 2024-05-27T10:10:21.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.527+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7754019070 con 0x7f7768105290 2024-05-27T10:10:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.529+0000 7f775fe00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f774c0b2880 0x7f774c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.529+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f77540e1300 con 0x7f7768105290 2024-05-27T10:10:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.529+0000 7f7767400700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f774c0b2880 0x7f774c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:21.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.531+0000 7f7767400700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f774c0b2880 0x7f774c0b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7758005950 tx=0x7f77580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:10:21.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.534+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77540a9ae0 con 0x7f7768105290 2024-05-27T10:10:21.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:21.775+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f776802ea30 con 0x7f774c0b2880 2024-05-27T10:10:22.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:10:22.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:10:22.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:21 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:22.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:21 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:10:22.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:21 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:10:22.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:21 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:22.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:21 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:10:22.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:21 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:10:22.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:21 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:23.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:22 smithi143 ceph-mon[35545]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:10:23.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:22 smithi143 ceph-mon[35545]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-27T10:10:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:22 smithi042 ceph-mon[31661]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:10:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:22 smithi042 ceph-mon[31661]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-27T10:10:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:22 smithi104 ceph-mon[34812]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:10:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:22 smithi104 ceph-mon[34812]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-27T10:10:25.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:25 smithi104 ceph-mon[34812]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T10:10:25.709 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:25 smithi143 ceph-mon[35545]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T10:10:25.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:25 smithi042 ceph-mon[31661]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T10:10:26.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:26 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/178691624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]: dispatch 2024-05-27T10:10:26.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:26 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]: dispatch 2024-05-27T10:10:26.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:26 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]': finished 2024-05-27T10:10:26.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:26 smithi104 ceph-mon[34812]: osdmap e51: 9 total, 8 up, 9 in 2024-05-27T10:10:26.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:26 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:26.724 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:26 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/178691624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]: dispatch 2024-05-27T10:10:26.725 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:26 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]: dispatch 2024-05-27T10:10:26.725 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:26 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]': finished 2024-05-27T10:10:26.725 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:26 smithi143 ceph-mon[35545]: osdmap e51: 9 total, 8 up, 9 in 2024-05-27T10:10:26.725 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:26 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:26.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:26 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/178691624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]: dispatch 2024-05-27T10:10:26.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:26 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]: dispatch 2024-05-27T10:10:26.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:26 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59bbe26-23f9-4728-957e-7ca5e03e6bd5"}]': finished 2024-05-27T10:10:26.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:26 smithi042 ceph-mon[31661]: osdmap e51: 9 total, 8 up, 9 in 2024-05-27T10:10:26.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:26 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:27.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:27 smithi104 ceph-mon[34812]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T10:10:27.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:27 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/4137338774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:10:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:27 smithi143 ceph-mon[35545]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T10:10:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:27 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/4137338774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:10:27.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:27 smithi042 ceph-mon[31661]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T10:10:27.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:27 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/4137338774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:10:29.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:29 smithi104 ceph-mon[34812]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:10:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:29 smithi143 ceph-mon[35545]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:10:29.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:29 smithi042 ceph-mon[31661]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:10:31.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:31 smithi143 ceph-mon[35545]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:31.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:31 smithi104 ceph-mon[34812]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:31.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:31 smithi042 ceph-mon[31661]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:33.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:33 smithi104 ceph-mon[34812]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:33.781 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:33 smithi143 ceph-mon[35545]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:33.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:33 smithi042 ceph-mon[31661]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:35 smithi104 ceph-mon[34812]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:35 smithi143 ceph-mon[35545]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:35 smithi042 ceph-mon[31661]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T10:10:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:36.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:36 smithi104 ceph-mon[34812]: Deploying daemon osd.8 on smithi143 2024-05-27T10:10:36.709 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T10:10:36.709 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:36.709 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:36 smithi143 ceph-mon[35545]: Deploying daemon osd.8 on smithi143 2024-05-27T10:10:36.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T10:10:36.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:36.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:36 smithi042 ceph-mon[31661]: Deploying daemon osd.8 on smithi143 2024-05-27T10:10:37.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:37 smithi104 ceph-mon[34812]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:37.700 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:37 smithi143 ceph-mon[35545]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:37 smithi042 ceph-mon[31661]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:39.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:39 smithi104 ceph-mon[34812]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:39.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:39 smithi143 ceph-mon[35545]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:39.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:39 smithi042 ceph-mon[31661]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:41.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:41 smithi143 ceph-mon[35545]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:41.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:41 smithi104 ceph-mon[34812]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:41 smithi042 ceph-mon[31661]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:43 smithi104 ceph-mon[34812]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:43.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:43 smithi143 ceph-mon[35545]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:43.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:43 smithi042 ceph-mon[31661]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:45.723 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:45 smithi143 ceph-mon[35545]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:45.723 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:45.723 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:45.724 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:45 smithi042 ceph-mon[31661]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:45.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:45.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:45 smithi104 ceph-mon[34812]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:47.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:47 smithi143 ceph-mon[35545]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:47 smithi104 ceph-mon[34812]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:47 smithi042 ceph-mon[31661]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:49 smithi143 ceph-mon[35545]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:49 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:49 smithi143 ceph-mon[35545]: from='osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T10:10:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:49 smithi143 ceph-mon[35545]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T10:10:49.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:49 smithi042 ceph-mon[31661]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:49 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:49 smithi042 ceph-mon[31661]: from='osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T10:10:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:49 smithi042 ceph-mon[31661]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T10:10:49.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:49 smithi104 ceph-mon[34812]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:49 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:49 smithi104 ceph-mon[34812]: from='osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T10:10:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:49 smithi104 ceph-mon[34812]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T10:10:50.351 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 8 on host 'smithi143' 2024-05-27T10:10:50.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.350+0000 7f775fe00700 1 -- 172.21.15.143:0/807291386 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f776802ea30 con 0x7f774c0b2880 2024-05-27T10:10:50.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.354+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f774c0b2880 msgr2=0x7f774c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:50.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.354+0000 7f776ca00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f774c0b2880 0x7f774c0b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7758005950 tx=0x7f77580058e0 comp rx=0 tx=0).stop 2024-05-27T10:10:50.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.354+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 msgr2=0x7f7768075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.354+0000 7f776ca00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 0x7f7768075b00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f775400d950 tx=0x7f775400dd10 comp rx=0 tx=0).stop 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 shutdown_connections 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7768105c30 0x7f7768076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f774c0b2880 0x7f774c0b4d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7768104160 0x7f7768077450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 --2- 172.21.15.143:0/807291386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7768105290 0x7f7768075b00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:50.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 >> 172.21.15.143:0/807291386 conn(0x7f77680ff7a0 msgr2=0x7f77681084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:10:50.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 shutdown_connections 2024-05-27T10:10:50.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:10:50.355+0000 7f776ca00700 1 -- 172.21.15.143:0/807291386 wait complete. 2024-05-27T10:10:50.792 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T10:10:50.792 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:10:50.792 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T10:10:50.792 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:50.792 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:10:50.792 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:50.793 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:50.793 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.793 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.793 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.793 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:50.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:50.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:10:51.210 DEBUG:teuthology.orchestra.run.smithi143:osd.8> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.8.service 2024-05-27T10:10:51.215 INFO:tasks.cephadm:Deploying osd.9 on smithi042 with /dev/vg_nvme/lv_1... 2024-05-27T10:10:51.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T10:10:51.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:51.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: from='osd.8 ' entity='osd.8' 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] boot 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: osdmap e54: 9 total, 9 up, 9 in 2024-05-27T10:10:51.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.684 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: from='osd.8 ' entity='osd.8' 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] boot 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: osdmap e54: 9 total, 9 up, 9 in 2024-05-27T10:10:51.713 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-27T10:10:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:10:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T10:10:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: from='osd.8 ' entity='osd.8' 2024-05-27T10:10:51.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: osd.8 [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] boot 2024-05-27T10:10:51.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: osdmap e54: 9 total, 9 up, 9 in 2024-05-27T10:10:51.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:10:52.789 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:52 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:10:52.789 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:52 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:10:52.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:52 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:10:52.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:52 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:10:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:52 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:10:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:52 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:10:53.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:53 smithi143 ceph-mon[35545]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:10:53.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:53 smithi143 ceph-mon[35545]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T10:10:53.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:53 smithi042 ceph-mon[31661]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:10:53.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:53 smithi042 ceph-mon[31661]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T10:10:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:53 smithi104 ceph-mon[34812]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:10:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:53 smithi104 ceph-mon[34812]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T10:10:54.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:54 smithi143 ceph-mon[35545]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T10:10:54.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:54 smithi042 ceph-mon[31661]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T10:10:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:54 smithi104 ceph-mon[34812]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T10:10:54.950 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T10:10:54.950 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T10:10:54.950 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-05-27T10:10:54.950 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-05-27T10:10:54.950 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367029 s, 286 MB/s 2024-05-27T10:10:54.951 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-05-27T10:10:55.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:55 smithi143 ceph-mon[35545]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:10:55.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:55 smithi042 ceph-mon[31661]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:10:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:55 smithi104 ceph-mon[34812]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:10:56.014 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_1 2024-05-27T10:10:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi143 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi143 to 5696M 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:57.218 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:57.219 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.219 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:57 smithi042 ceph-mon[31661]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-27T10:10:57.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi143 2024-05-27T10:10:57.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi143 to 5696M 2024-05-27T10:10:57.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:57.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:57.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:57 smithi143 ceph-mon[35545]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-27T10:10:57.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi143 2024-05-27T10:10:57.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi143 to 5696M 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:10:57.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:57 smithi104 ceph-mon[34812]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 -- 172.21.15.42:0/2133790148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 msgr2=0x7f08f010e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 --2- 172.21.15.42:0/2133790148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f010e6f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b0a0 tx=0x7f08e800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 -- 172.21.15.42:0/2133790148 shutdown_connections 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 --2- 172.21.15.42:0/2133790148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f010e6f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 --2- 172.21.15.42:0/2133790148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08f00776d0 0x7f08f0075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 --2- 172.21.15.42:0/2133790148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f00760e0 0x7f08f0076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 -- 172.21.15.42:0/2133790148 >> 172.21.15.42:0/2133790148 conn(0x7f08f00ff9b0 msgr2=0x7f08f0101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:10:59.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 -- 172.21.15.42:0/2133790148 shutdown_connections 2024-05-27T10:10:59.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.098+0000 7f08f6200700 1 -- 172.21.15.42:0/2133790148 wait complete. 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.099+0000 7f08f6200700 1 Processor -- start 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.099+0000 7f08f6200700 1 -- start start 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08f6200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f00760e0 0x7f08f019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08f6200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08f00776d0 0x7f08f019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08f6200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f01a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08f6200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0111e90 con 0x7f08f010c1e0 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08f6200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0111d10 con 0x7f08f00776d0 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08f6200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0112010 con 0x7f08f00760e0 2024-05-27T10:10:59.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08efe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f01a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08efe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f01a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43496/0 (socket says 172.21.15.42:43496) 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08efe00700 1 -- 172.21.15.42:0/4218665255 learned_addr learned my addr 172.21.15.42:0/4218665255 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08ef400700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f00760e0 0x7f08f019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.100+0000 7f08e6a00700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08f00776d0 0x7f08f019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08efe00700 1 -- 172.21.15.42:0/4218665255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f00760e0 msgr2=0x7f08f019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08efe00700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f00760e0 0x7f08f019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08efe00700 1 -- 172.21.15.42:0/4218665255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08f00776d0 msgr2=0x7f08f019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:10:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08efe00700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08f00776d0 0x7f08f019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08efe00700 1 -- 172.21.15.42:0/4218665255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f01a4540 con 0x7f08f010c1e0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08efe00700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f01a3ec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b830 tx=0x7f08e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8021070 con 0x7f08f010c1e0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08e8007c80 con 0x7f08f010c1e0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f01a47d0 con 0x7f08f010c1e0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.101+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e801b7b0 con 0x7f08f010c1e0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.102+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f01a4c20 con 0x7f08f010c1e0 2024-05-27T10:10:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.103+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f00512b0 con 0x7f08f010c1e0 2024-05-27T10:10:59.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.106+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f08e8024070 con 0x7f08f010c1e0 2024-05-27T10:10:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.107+0000 7f08ecc00700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f08d40b2880 0x7f08d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:10:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.107+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f08e800f020 con 0x7f08f010c1e0 2024-05-27T10:10:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.107+0000 7f08ef400700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f08d40b2880 0x7f08d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:10:59.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.108+0000 7f08ef400700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f08d40b2880 0x7f08d40b4d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08e0005950 tx=0x7f08e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:10:59.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.110+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08e802b020 con 0x7f08f010c1e0 2024-05-27T10:10:59.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:10:59.359+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f08f0063cb0 con 0x7f08d40b2880 2024-05-27T10:10:59.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:59 smithi104 ceph-mon[34812]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-27T10:10:59.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:10:59.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:10:59.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:10:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:59.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:59 smithi143 ceph-mon[35545]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-27T10:10:59.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:10:59.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:10:59.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:10:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:10:59.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:59 smithi042 ceph-mon[31661]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-27T10:10:59.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:10:59.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:10:59.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:10:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:00.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:00 smithi042 ceph-mon[31661]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:11:00.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:00 smithi104 ceph-mon[34812]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:11:00.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:00 smithi143 ceph-mon[35545]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:11:01.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:01 smithi042 ceph-mon[31661]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T10:11:01.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:01 smithi104 ceph-mon[34812]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T10:11:01.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:01 smithi143 ceph-mon[35545]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T10:11:03.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:03 smithi104 ceph-mon[34812]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T10:11:03.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:03 smithi143 ceph-mon[35545]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T10:11:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:03 smithi042 ceph-mon[31661]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T10:11:04.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:04 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2217111684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d4d17-d8c3-495b-8295-667f0bfaaef3"}]: dispatch 2024-05-27T10:11:04.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:04 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2217111684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951d4d17-d8c3-495b-8295-667f0bfaaef3"}]': finished 2024-05-27T10:11:04.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:04 smithi104 ceph-mon[34812]: osdmap e57: 10 total, 9 up, 10 in 2024-05-27T10:11:04.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:04 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:04.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:04 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2217111684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d4d17-d8c3-495b-8295-667f0bfaaef3"}]: dispatch 2024-05-27T10:11:04.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:04 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2217111684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951d4d17-d8c3-495b-8295-667f0bfaaef3"}]': finished 2024-05-27T10:11:04.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:04 smithi143 ceph-mon[35545]: osdmap e57: 10 total, 9 up, 10 in 2024-05-27T10:11:04.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:04 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:04 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2217111684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d4d17-d8c3-495b-8295-667f0bfaaef3"}]: dispatch 2024-05-27T10:11:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:04 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2217111684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951d4d17-d8c3-495b-8295-667f0bfaaef3"}]': finished 2024-05-27T10:11:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:04 smithi042 ceph-mon[31661]: osdmap e57: 10 total, 9 up, 10 in 2024-05-27T10:11:04.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:04 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:05 smithi104 ceph-mon[34812]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:11:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:05 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3790685409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:11:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:05 smithi143 ceph-mon[35545]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:11:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:05 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3790685409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:11:05.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:05 smithi042 ceph-mon[31661]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:11:05.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:05 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3790685409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:11:07.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:07 smithi104 ceph-mon[34812]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:07.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:07 smithi143 ceph-mon[35545]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:07.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:07 smithi042 ceph-mon[31661]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:09.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:09 smithi104 ceph-mon[34812]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:09 smithi143 ceph-mon[35545]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:09.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:09 smithi042 ceph-mon[31661]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:11.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:11 smithi104 ceph-mon[34812]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:11.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:11 smithi042 ceph-mon[31661]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:11.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:11 smithi143 ceph-mon[35545]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:13.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:13 smithi104 ceph-mon[34812]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:13.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:13 smithi143 ceph-mon[35545]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:13.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:13 smithi042 ceph-mon[31661]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:14.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T10:11:14.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:14.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T10:11:14.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:14.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T10:11:14.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:15.556 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:15 smithi042 ceph-mon[31661]: Deploying daemon osd.9 on smithi042 2024-05-27T10:11:15.557 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:15 smithi042 ceph-mon[31661]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:15 smithi104 ceph-mon[34812]: Deploying daemon osd.9 on smithi042 2024-05-27T10:11:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:15 smithi104 ceph-mon[34812]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:15.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:15 smithi143 ceph-mon[35545]: Deploying daemon osd.9 on smithi042 2024-05-27T10:11:15.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:15 smithi143 ceph-mon[35545]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:17.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:17 smithi042 ceph-mon[31661]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:17.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:17 smithi143 ceph-mon[35545]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:17.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:17 smithi104 ceph-mon[34812]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:19.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:19 smithi042 ceph-mon[31661]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:19.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:19 smithi143 ceph-mon[35545]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:19.952 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:19 smithi104 ceph-mon[34812]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:21.746 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:21 smithi042 ceph-mon[31661]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:21.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:21 smithi143 ceph-mon[35545]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:21 smithi104 ceph-mon[34812]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:23 smithi042 ceph-mon[31661]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:23 smithi143 ceph-mon[35545]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:23 smithi104 ceph-mon[34812]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:25.277 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:25.277 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:25.277 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:25 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:11:25.277 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:25 smithi042 ceph-mon[31661]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:25.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:25.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:25.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:25 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:11:25.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:25 smithi143 ceph-mon[35545]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:25.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:25.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:25.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:25 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:11:25.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:25 smithi104 ceph-mon[34812]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:27.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:27 smithi042 ceph-mon[31661]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:27.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:27 smithi104 ceph-mon[34812]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:27 smithi143 ceph-mon[35545]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:29.305 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:29.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:29.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:28 smithi042 ceph-mon[31661]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:29.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:28 smithi042 ceph-mon[31661]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T10:11:29.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:29.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:29.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:28 smithi143 ceph-mon[35545]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:29.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:28 smithi143 ceph-mon[35545]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T10:11:29.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:29.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:29.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:28 smithi104 ceph-mon[34812]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:29.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:28 smithi104 ceph-mon[34812]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T10:11:29.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.775+0000 7f08ecc00700 1 -- 172.21.15.42:0/4218665255 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f08f0063cb0 con 0x7f08d40b2880 2024-05-27T10:11:29.776 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 9 on host 'smithi042' 2024-05-27T10:11:29.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f08d40b2880 msgr2=0x7f08d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f08d40b2880 0x7f08d40b4d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08e0005950 tx=0x7f08e00058e0 comp rx=0 tx=0).stop 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 msgr2=0x7f08f01a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f01a3ec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b830 tx=0x7f08e800bd70 comp rx=0 tx=0).stop 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 shutdown_connections 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f010c1e0 0x7f08f01a3ec0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f00760e0 0x7f08f019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f08d40b2880 0x7f08d40b4d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 --2- 172.21.15.42:0/4218665255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08f00776d0 0x7f08f019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.779+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 >> 172.21.15.42:0/4218665255 conn(0x7f08f00ff9b0 msgr2=0x7f08f0108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.780+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 shutdown_connections 2024-05-27T10:11:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:11:29.780+0000 7f08f6200700 1 -- 172.21.15.42:0/4218665255 wait complete. 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:11:30.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:11:30.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T10:11:30.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T10:11:30.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-05-27T10:11:30.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:30.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:30.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:11:30.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:11:30.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:30.801 DEBUG:teuthology.orchestra.run.smithi042:osd.9> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.9.service 2024-05-27T10:11:30.806 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2024-05-27T10:11:30.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T10:11:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:30 smithi104 ceph-mon[34812]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:11:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:30 smithi104 ceph-mon[34812]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T10:11:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:30 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:30 smithi104 ceph-mon[34812]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:11:31.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:30 smithi042 ceph-mon[31661]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:11:31.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:30 smithi042 ceph-mon[31661]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T10:11:31.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:31.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:30 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:31.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:30 smithi042 ceph-mon[31661]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:30 smithi143 ceph-mon[35545]: from='osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-05-27T10:11:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:30 smithi143 ceph-mon[35545]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T10:11:31.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:31.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:30 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:31.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:30 smithi143 ceph-mon[35545]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-27T10:11:32.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:31 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:11:32.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:31 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:11:32.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:32.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:31 smithi104 ceph-mon[34812]: osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] boot 2024-05-27T10:11:32.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:31 smithi104 ceph-mon[34812]: osdmap e60: 10 total, 10 up, 10 in 2024-05-27T10:11:32.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:32.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:31 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:11:32.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:31 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:11:32.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:32.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:31 smithi143 ceph-mon[35545]: osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] boot 2024-05-27T10:11:32.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:31 smithi143 ceph-mon[35545]: osdmap e60: 10 total, 10 up, 10 in 2024-05-27T10:11:32.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:32.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:31 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:11:32.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:31 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:11:32.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:32.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:31 smithi042 ceph-mon[31661]: osd.9 [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] boot 2024-05-27T10:11:32.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:31 smithi042 ceph-mon[31661]: osdmap e60: 10 total, 10 up, 10 in 2024-05-27T10:11:32.338 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:11:33.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:33 smithi143 ceph-mon[35545]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T10:11:33.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:33 smithi143 ceph-mon[35545]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:33.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:33 smithi104 ceph-mon[34812]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T10:11:33.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:33 smithi104 ceph-mon[34812]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:33.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:33 smithi042 ceph-mon[31661]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T10:11:33.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:33 smithi042 ceph-mon[31661]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:34.435 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T10:11:34.435 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T10:11:34.435 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-27T10:11:34.435 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-27T10:11:34.435 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0564594 s, 186 MB/s 2024-05-27T10:11:34.435 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-27T10:11:35.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-05-27T10:11:35.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:35 smithi104 ceph-mon[34812]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:35.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:35 smithi143 ceph-mon[35545]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:35.829 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:11:35.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:35 smithi042 ceph-mon[31661]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:37.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi042 2024-05-27T10:11:37.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi042 to 4273M 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:37 smithi104 ceph-mon[34812]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:37.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi042 2024-05-27T10:11:37.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi042 to 4273M 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:37 smithi042 ceph-mon[31661]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:37.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi042 2024-05-27T10:11:37.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:11:37.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi042 to 4273M 2024-05-27T10:11:37.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:37.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:11:37.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:11:37.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:37 smithi143 ceph-mon[35545]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:38.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 -- 172.21.15.104:0/623674055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 msgr2=0x7f0c0410e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:11:38.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 --2- 172.21.15.104:0/623674055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c0410e500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00b0a0 tx=0x7f0bfc00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 -- 172.21.15.104:0/623674055 shutdown_connections 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 --2- 172.21.15.104:0/623674055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c0410e500 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 --2- 172.21.15.104:0/623674055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c040776d0 0x7f0c04075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 --2- 172.21.15.104:0/623674055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c040760e0 0x7f0c04076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 -- 172.21.15.104:0/623674055 >> 172.21.15.104:0/623674055 conn(0x7f0c040ff7a0 msgr2=0x7f0c04101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.256+0000 7f0c03e00700 1 -- 172.21.15.104:0/623674055 shutdown_connections 2024-05-27T10:11:38.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.257+0000 7f0c03e00700 1 -- 172.21.15.104:0/623674055 wait complete. 2024-05-27T10:11:38.260 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 Processor -- start 2024-05-27T10:11:38.262 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 -- start start 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c040760e0 0x7f0c0419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c040776d0 0x7f0c0419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c041a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111c90 con 0x7f0c040760e0 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111b10 con 0x7f0c0410bff0 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111e10 con 0x7f0c040776d0 2024-05-27T10:11:38.263 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.258+0000 7f0c03400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c041a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c041a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53388/0 (socket says 172.21.15.104:53388) 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 -- 172.21.15.104:0/3249921644 learned_addr learned my addr 172.21.15.104:0/3249921644 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c02a00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c040760e0 0x7f0c0419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c02000700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c040776d0 0x7f0c0419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 -- 172.21.15.104:0/3249921644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c040776d0 msgr2=0x7f0c0419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c040776d0 0x7f0c0419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 -- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c040760e0 msgr2=0x7f0c0419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c040760e0 0x7f0c0419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 -- 172.21.15.104:0/3249921644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c041a4370 con 0x7f0c0410bff0 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.259+0000 7f0c03400700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c041a3cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00b830 tx=0x7f0bfc00bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.260+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc021070 con 0x7f0c0410bff0 2024-05-27T10:11:38.264 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.260+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bfc007c80 con 0x7f0c0410bff0 2024-05-27T10:11:38.265 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.260+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01b7b0 con 0x7f0c0410bff0 2024-05-27T10:11:38.265 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.260+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c041a4600 con 0x7f0c0410bff0 2024-05-27T10:11:38.265 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.260+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041a4a00 con 0x7f0c0410bff0 2024-05-27T10:11:38.265 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.261+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c040512b0 con 0x7f0c0410bff0 2024-05-27T10:11:38.265 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.263+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0bfc024070 con 0x7f0c0410bff0 2024-05-27T10:11:38.266 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.265+0000 7f0bfb400700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0be80b2900 0x7f0be80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:11:38.266 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.265+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f0bfc00f020 con 0x7f0c0410bff0 2024-05-27T10:11:38.266 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.266+0000 7f0c02a00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0be80b2900 0x7f0be80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:11:38.267 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.267+0000 7f0c02a00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0be80b2900 0x7f0be80b4e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005950 tx=0x7f0bf40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:11:38.268 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.267+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0bfc02b020 con 0x7f0c0410bff0 2024-05-27T10:11:38.507 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:11:38.506+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0c04063cb0 con 0x7f0be80b2900 2024-05-27T10:11:39.666 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:39 smithi104 ceph-mon[34812]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:39.667 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:39 smithi104 ceph-mon[34812]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:11:39.667 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:11:39.667 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:11:39.667 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:39.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:39 smithi143 ceph-mon[35545]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:39.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:39 smithi143 ceph-mon[35545]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:11:39.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:11:39.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:11:39.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:39 smithi042 ceph-mon[31661]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:39 smithi042 ceph-mon[31661]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:11:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:11:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:11:39.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:41.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:41 smithi104 ceph-mon[34812]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:41.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:41 smithi143 ceph-mon[35545]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:41 smithi042 ceph-mon[31661]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:43 smithi104 ceph-mon[34812]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:43 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/3233399633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]: dispatch 2024-05-27T10:11:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:43 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]: dispatch 2024-05-27T10:11:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:43 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]': finished 2024-05-27T10:11:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:43 smithi104 ceph-mon[34812]: osdmap e62: 11 total, 10 up, 11 in 2024-05-27T10:11:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:11:43.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:43 smithi143 ceph-mon[35545]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:43.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:43 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/3233399633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]: dispatch 2024-05-27T10:11:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:43 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]: dispatch 2024-05-27T10:11:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:43 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]': finished 2024-05-27T10:11:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:43 smithi143 ceph-mon[35545]: osdmap e62: 11 total, 10 up, 11 in 2024-05-27T10:11:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:11:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:43 smithi042 ceph-mon[31661]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:43 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/3233399633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]: dispatch 2024-05-27T10:11:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:43 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]: dispatch 2024-05-27T10:11:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:43 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d114e0d1-848d-453c-a61d-65262d9c404e"}]': finished 2024-05-27T10:11:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:43 smithi042 ceph-mon[31661]: osdmap e62: 11 total, 10 up, 11 in 2024-05-27T10:11:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:11:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:44 smithi104 ceph-mon[34812]: from='client.? 172.21.15.104:0/1265334904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:11:44.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:44 smithi143 ceph-mon[35545]: from='client.? 172.21.15.104:0/1265334904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:11:44.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:44 smithi042 ceph-mon[31661]: from='client.? 172.21.15.104:0/1265334904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:11:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:45 smithi104 ceph-mon[34812]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:45.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:45 smithi143 ceph-mon[35545]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:45 smithi042 ceph-mon[31661]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:47 smithi104 ceph-mon[34812]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:47 smithi143 ceph-mon[35545]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:47 smithi042 ceph-mon[31661]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:49 smithi104 ceph-mon[34812]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:49 smithi143 ceph-mon[35545]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:49 smithi042 ceph-mon[31661]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:51.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:51 smithi104 ceph-mon[34812]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:51 smithi143 ceph-mon[35545]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:51.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:51 smithi042 ceph-mon[31661]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:53 smithi104 ceph-mon[34812]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T10:11:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:53 smithi104 ceph-mon[34812]: Deploying daemon osd.10 on smithi104 2024-05-27T10:11:53.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:53 smithi143 ceph-mon[35545]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:53.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T10:11:53.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:53.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:53 smithi143 ceph-mon[35545]: Deploying daemon osd.10 on smithi104 2024-05-27T10:11:53.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:53 smithi042 ceph-mon[31661]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:53.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T10:11:53.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:11:53.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:53 smithi042 ceph-mon[31661]: Deploying daemon osd.10 on smithi104 2024-05-27T10:11:55.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:55 smithi104 ceph-mon[34812]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:55.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:55 smithi143 ceph-mon[35545]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:55.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:55 smithi042 ceph-mon[31661]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:57.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:57 smithi104 ceph-mon[34812]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:57.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:57 smithi143 ceph-mon[35545]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:57.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:57 smithi042 ceph-mon[31661]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:59.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:11:59 smithi143 ceph-mon[35545]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:59.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:11:59 smithi042 ceph-mon[31661]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:11:59.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:11:59 smithi104 ceph-mon[34812]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:01 smithi143 ceph-mon[35545]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:01.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:01 smithi104 ceph-mon[34812]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:01 smithi042 ceph-mon[31661]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:03.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:03.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:03.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:03.305 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:03 smithi104 ceph-mon[34812]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:03.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:03.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:03.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:03.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:03 smithi143 ceph-mon[35545]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:03.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:03.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:03.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:03.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:03 smithi042 ceph-mon[31661]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:05.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:05 smithi104 ceph-mon[34812]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:05 smithi143 ceph-mon[35545]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:05.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:05 smithi042 ceph-mon[31661]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T10:12:07.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:06 smithi104 ceph-mon[34812]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:07.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:07.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:07.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T10:12:07.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T10:12:07.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:06 smithi143 ceph-mon[35545]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T10:12:07.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:06 smithi042 ceph-mon[31661]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:07.781 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2024-05-27T10:12:07.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.780+0000 7f0bfb400700 1 -- 172.21.15.104:0/3249921644 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0c04063cb0 con 0x7f0be80b2900 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0be80b2900 msgr2=0x7f0be80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0be80b2900 0x7f0be80b4e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005950 tx=0x7f0bf40058e0 comp rx=0 tx=0).stop 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 msgr2=0x7f0c041a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c041a3cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00b830 tx=0x7f0bfc00bd70 comp rx=0 tx=0).stop 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 shutdown_connections 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0be80b2900 0x7f0be80b4e10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c0410bff0 0x7f0c041a3cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c040760e0 0x7f0c0419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 --2- 172.21.15.104:0/3249921644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c040776d0 0x7f0c0419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 >> 172.21.15.104:0/3249921644 conn(0x7f0c040ff7a0 msgr2=0x7f0c04107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:07.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 shutdown_connections 2024-05-27T10:12:07.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-27T10:12:07.784+0000 7f0c03e00700 1 -- 172.21.15.104:0/3249921644 wait complete. 2024-05-27T10:12:08.226 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T10:12:08.227 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:12:08.227 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T10:12:08.227 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:08.227 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:12:08.227 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:08.227 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:08.228 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:07 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:08.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:08.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:07 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:08.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T10:12:08.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:12:08.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T10:12:08.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:08.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-27T10:12:08.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:08.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:08.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:07 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:08.669 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.10.service 2024-05-27T10:12:08.674 INFO:tasks.cephadm:Deploying osd.11 on smithi143 with /dev/vg_nvme/lv_1... 2024-05-27T10:12:08.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T10:12:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:12:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T10:12:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] boot 2024-05-27T10:12:09.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: osdmap e65: 11 total, 11 up, 11 in 2024-05-27T10:12:09.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.113 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] boot 2024-05-27T10:12:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: osdmap e65: 11 total, 11 up, 11 in 2024-05-27T10:12:09.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: osd.10 [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] boot 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: osdmap e65: 11 total, 11 up, 11 in 2024-05-27T10:12:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:12:10.190 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:09 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:12:10.190 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:09 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:12:10.282 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:09 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:12:10.283 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:09 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:12:10.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:09 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:12:10.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:09 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:12:11.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:11 smithi143 ceph-mon[35545]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T10:12:11.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:11 smithi143 ceph-mon[35545]: pgmap v311: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:11.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:11 smithi104 ceph-mon[34812]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T10:12:11.202 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:11 smithi104 ceph-mon[34812]: pgmap v311: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:11.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:11 smithi042 ceph-mon[31661]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T10:12:11.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:11 smithi042 ceph-mon[31661]: pgmap v311: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:11.990 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T10:12:11.990 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T10:12:11.990 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-27T10:12:11.990 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-27T10:12:11.991 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0353869 s, 296 MB/s 2024-05-27T10:12:11.991 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-27T10:12:12.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:12 smithi143 ceph-mon[35545]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T10:12:12.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:12 smithi042 ceph-mon[31661]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T10:12:12.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:12 smithi104 ceph-mon[34812]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T10:12:12.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_1 2024-05-27T10:12:13.265 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:12:13.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:13 smithi143 ceph-mon[35545]: pgmap v313: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:13.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:13 smithi042 ceph-mon[31661]: pgmap v313: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:13 smithi104 ceph-mon[34812]: pgmap v313: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:14.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi104 2024-05-27T10:12:14.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi104 to 4272M 2024-05-27T10:12:14.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:14.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:14.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:14 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi104 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi104 to 4272M 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.903 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:14.904 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:14.904 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:14 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi104 2024-05-27T10:12:14.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi104 to 4272M 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:14.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:14 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:15.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 -- 172.21.15.143:0/544800886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 msgr2=0x7f4194106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 --2- 172.21.15.143:0/544800886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f4194106960 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4184009990 tx=0x7f4184009ca0 comp rx=0 tx=0).stop 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 -- 172.21.15.143:0/544800886 shutdown_connections 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 --2- 172.21.15.143:0/544800886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4194108020 0x7f419410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 --2- 172.21.15.143:0/544800886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f4194106960 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 --2- 172.21.15.143:0/544800886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4194107680 0x7f4194107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 -- 172.21.15.143:0/544800886 >> 172.21.15.143:0/544800886 conn(0x7f4194101b90 msgr2=0x7f4194104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 -- 172.21.15.143:0/544800886 shutdown_connections 2024-05-27T10:12:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.512+0000 7f4199000700 1 -- 172.21.15.143:0/544800886 wait complete. 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.513+0000 7f4199000700 1 Processor -- start 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.513+0000 7f4199000700 1 -- start start 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4199000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f419419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4199000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4194107680 0x7f419419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4199000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4194108020 0x7f41941a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4199000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4194111e90 con 0x7f4194107680 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4199000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4194111d10 con 0x7f4194108020 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4199000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4194112010 con 0x7f4194106550 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4191e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f419419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4191e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f419419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39640/0 (socket says 172.21.15.143:39640) 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4191e00700 1 -- 172.21.15.143:0/3193219973 learned_addr learned my addr 172.21.15.143:0/3193219973 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:12:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4191400700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4194107680 0x7f419419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:15.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.514+0000 7f4192800700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4194108020 0x7f41941a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:15.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.515+0000 7f4191e00700 1 -- 172.21.15.143:0/3193219973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4194108020 msgr2=0x7f41941a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:15.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.515+0000 7f4191e00700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4194108020 0x7f41941a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.515+0000 7f4191e00700 1 -- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4194107680 msgr2=0x7f419419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.515+0000 7f4191e00700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4194107680 0x7f419419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.515+0000 7f4191e00700 1 -- 172.21.15.143:0/3193219973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41941a46b0 con 0x7f4194106550 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.515+0000 7f4191e00700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f419419cb50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4184003820 tx=0x7f41840047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.516+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184026070 con 0x7f4194106550 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.516+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41941a4940 con 0x7f4194106550 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.516+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4184003f60 con 0x7f4194106550 2024-05-27T10:12:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.516+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418400fca0 con 0x7f4194106550 2024-05-27T10:12:15.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.516+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941a4da0 con 0x7f4194106550 2024-05-27T10:12:15.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.517+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41940512b0 con 0x7f4194106550 2024-05-27T10:12:15.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.520+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4184029070 con 0x7f4194106550 2024-05-27T10:12:15.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.521+0000 7f418aa00700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f41780b2880 0x7f41780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:15.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.522+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f41840e2ac0 con 0x7f4194106550 2024-05-27T10:12:15.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.522+0000 7f4191400700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f41780b2880 0x7f41780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:15.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.523+0000 7f4191400700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f41780b2880 0x7f41780b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f419419dc00 tx=0x7f418000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:15.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.525+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4184030080 con 0x7f4194106550 2024-05-27T10:12:15.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:15.765+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4194063bf0 con 0x7f41780b2880 2024-05-27T10:12:15.767 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:15 smithi143 ceph-mon[35545]: pgmap v314: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:15.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:15 smithi104 ceph-mon[34812]: pgmap v314: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:16.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:15 smithi042 ceph-mon[31661]: pgmap v314: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:16.912 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:16 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:12:16.913 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:16 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:12:16.913 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:16 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:16 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:12:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:16 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:12:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:16 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:16 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T10:12:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:16 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T10:12:17.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:16 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:17.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:17 smithi104 ceph-mon[34812]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:12:17.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:17 smithi104 ceph-mon[34812]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T10:12:18.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:17 smithi143 ceph-mon[35545]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:12:18.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:17 smithi143 ceph-mon[35545]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T10:12:18.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:17 smithi042 ceph-mon[31661]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:12:18.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:17 smithi042 ceph-mon[31661]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T10:12:19.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:19 smithi104 ceph-mon[34812]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-27T10:12:20.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:19 smithi143 ceph-mon[35545]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-27T10:12:20.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:19 smithi042 ceph-mon[31661]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-27T10:12:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:20 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/527950811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]: dispatch 2024-05-27T10:12:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:20 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]: dispatch 2024-05-27T10:12:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:20 smithi143 ceph-mon[35545]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]': finished 2024-05-27T10:12:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:20 smithi143 ceph-mon[35545]: osdmap e68: 12 total, 11 up, 12 in 2024-05-27T10:12:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:20 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/527950811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]: dispatch 2024-05-27T10:12:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:20 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]: dispatch 2024-05-27T10:12:20.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:20 smithi104 ceph-mon[34812]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]': finished 2024-05-27T10:12:20.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:20 smithi104 ceph-mon[34812]: osdmap e68: 12 total, 11 up, 12 in 2024-05-27T10:12:20.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:21.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:20 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/527950811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]: dispatch 2024-05-27T10:12:21.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:20 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]: dispatch 2024-05-27T10:12:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:20 smithi042 ceph-mon[31661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a925b882-f488-4723-a316-04230d44cfb7"}]': finished 2024-05-27T10:12:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:20 smithi042 ceph-mon[31661]: osdmap e68: 12 total, 11 up, 12 in 2024-05-27T10:12:21.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:21 smithi104 ceph-mon[34812]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-27T10:12:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:21 smithi104 ceph-mon[34812]: from='client.? 172.21.15.143:0/4130354862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:12:21.995 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:21 smithi042 ceph-mon[31661]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-27T10:12:21.995 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:21 smithi042 ceph-mon[31661]: from='client.? 172.21.15.143:0/4130354862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:12:22.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:21 smithi143 ceph-mon[35545]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-27T10:12:22.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:21 smithi143 ceph-mon[35545]: from='client.? 172.21.15.143:0/4130354862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T10:12:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:23 smithi104 ceph-mon[34812]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:12:24.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:23 smithi143 ceph-mon[35545]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:12:24.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:23 smithi042 ceph-mon[31661]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:12:25.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:25 smithi104 ceph-mon[34812]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:12:26.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:25 smithi143 ceph-mon[35545]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:12:26.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:25 smithi042 ceph-mon[31661]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T10:12:27.908 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:27 smithi143 ceph-mon[35545]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:27 smithi104 ceph-mon[34812]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:28.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:27 smithi042 ceph-mon[31661]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:29 smithi104 ceph-mon[34812]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T10:12:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:29 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:29 smithi143 ceph-mon[35545]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:30.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T10:12:30.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:29 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:30.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:29 smithi042 ceph-mon[31661]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T10:12:30.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:29 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:30.764 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:30 smithi143 ceph-mon[35545]: Deploying daemon osd.11 on smithi143 2024-05-27T10:12:31.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:30 smithi042 ceph-mon[31661]: Deploying daemon osd.11 on smithi143 2024-05-27T10:12:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:30 smithi104 ceph-mon[34812]: Deploying daemon osd.11 on smithi143 2024-05-27T10:12:32.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:31 smithi143 ceph-mon[35545]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:32.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:31 smithi042 ceph-mon[31661]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:32.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:31 smithi104 ceph-mon[34812]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:33.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:32 smithi143 ceph-mon[35545]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:33.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:32 smithi042 ceph-mon[31661]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:33.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:32 smithi104 ceph-mon[34812]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:35.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:35 smithi143 ceph-mon[35545]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:35.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:35 smithi104 ceph-mon[34812]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:35 smithi042 ceph-mon[31661]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:37.685 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:37 smithi143 ceph-mon[35545]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:37.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:37 smithi104 ceph-mon[34812]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:37 smithi042 ceph-mon[31661]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:39.849 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:39 smithi143 ceph-mon[35545]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:39.850 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:39.850 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:39.850 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:39 smithi104 ceph-mon[34812]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:40.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:39 smithi042 ceph-mon[31661]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:40.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:40.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:40.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:41.733 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:41 smithi143 ceph-mon[35545]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:41 smithi104 ceph-mon[34812]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:42.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:41 smithi042 ceph-mon[31661]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T10:12:43.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:43 smithi042 ceph-mon[31661]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T10:12:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:43.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.702 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T10:12:43.702 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:43 smithi104 ceph-mon[34812]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T10:12:43.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:43 smithi143 ceph-mon[35545]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T10:12:44.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.300+0000 7f418aa00700 1 -- 172.21.15.143:0/3193219973 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4194063bf0 con 0x7f41780b2880 2024-05-27T10:12:44.300 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 11 on host 'smithi143' 2024-05-27T10:12:44.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f41780b2880 msgr2=0x7f41780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:44.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f41780b2880 0x7f41780b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f419419dc00 tx=0x7f418000b410 comp rx=0 tx=0).stop 2024-05-27T10:12:44.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 msgr2=0x7f419419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:44.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f419419cb50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4184003820 tx=0x7f41840047b0 comp rx=0 tx=0).stop 2024-05-27T10:12:44.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 shutdown_connections 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f41780b2880 0x7f41780b4d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4194108020 0x7f41941a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4194106550 0x7f419419cb50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 --2- 172.21.15.143:0/3193219973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4194107680 0x7f419419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 >> 172.21.15.143:0/3193219973 conn(0x7f4194101b90 msgr2=0x7f419410a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 shutdown_connections 2024-05-27T10:12:44.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:12:44.304+0000 7f4199000700 1 -- 172.21.15.143:0/3193219973 wait complete. 2024-05-27T10:12:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T10:12:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T10:12:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:12:44.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:12:44.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:44.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:44.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:44 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:44.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:44.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:44 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:44 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:12:45.021 DEBUG:teuthology.orchestra.run.smithi143:osd.11> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.11.service 2024-05-27T10:12:45.025 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-27T10:12:45.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-27T10:12:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:12:45.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:45 smithi143 ceph-mon[35545]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:45.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:45 smithi143 ceph-mon[35545]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:12:45.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:45 smithi143 ceph-mon[35545]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T10:12:45.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:45.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:45.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:45 smithi042 ceph-mon[31661]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:45.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:45 smithi042 ceph-mon[31661]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:12:45.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:45 smithi042 ceph-mon[31661]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T10:12:45.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:45.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:45 smithi104 ceph-mon[34812]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:45 smithi104 ceph-mon[34812]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-27T10:12:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:45 smithi104 ceph-mon[34812]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T10:12:45.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:45.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:46 smithi104 ceph-mon[34812]: purged_snaps scrub starts 2024-05-27T10:12:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:46 smithi104 ceph-mon[34812]: purged_snaps scrub ok 2024-05-27T10:12:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:46 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:46 smithi104 ceph-mon[34812]: osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] boot 2024-05-27T10:12:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:46 smithi104 ceph-mon[34812]: osdmap e71: 12 total, 12 up, 12 in 2024-05-27T10:12:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:46 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:46.718 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:46 smithi042 ceph-mon[31661]: purged_snaps scrub starts 2024-05-27T10:12:46.718 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:46 smithi042 ceph-mon[31661]: purged_snaps scrub ok 2024-05-27T10:12:46.718 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:46 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:46.718 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:46 smithi042 ceph-mon[31661]: osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] boot 2024-05-27T10:12:46.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:46 smithi042 ceph-mon[31661]: osdmap e71: 12 total, 12 up, 12 in 2024-05-27T10:12:46.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:46 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:46.769 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:46 smithi143 ceph-mon[35545]: purged_snaps scrub starts 2024-05-27T10:12:46.769 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:46 smithi143 ceph-mon[35545]: purged_snaps scrub ok 2024-05-27T10:12:46.769 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:46 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:46.769 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:46 smithi143 ceph-mon[35545]: osd.11 [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] boot 2024-05-27T10:12:46.770 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:46 smithi143 ceph-mon[35545]: osdmap e71: 12 total, 12 up, 12 in 2024-05-27T10:12:46.770 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:46 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:12:47.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:47 smithi042 ceph-mon[31661]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:47.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:47 smithi042 ceph-mon[31661]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T10:12:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:47 smithi104 ceph-mon[34812]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:47 smithi104 ceph-mon[34812]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T10:12:47.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:47 smithi143 ceph-mon[35545]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-27T10:12:47.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:47 smithi143 ceph-mon[35545]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T10:12:48.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 -- 172.21.15.42:0/1035270758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c108010 msgr2=0x7f5d0c10e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:48.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 --2- 172.21.15.42:0/1035270758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c108010 0x7f5d0c10e8c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b0a0 tx=0x7f5d0400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 -- 172.21.15.42:0/1035270758 shutdown_connections 2024-05-27T10:12:48.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 --2- 172.21.15.42:0/1035270758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c108010 0x7f5d0c10e8c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 --2- 172.21.15.42:0/1035270758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c106540 0x7f5d0c106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 --2- 172.21.15.42:0/1035270758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d0c107670 0x7f5d0c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 -- 172.21.15.42:0/1035270758 >> 172.21.15.42:0/1035270758 conn(0x7f5d0c101bc0 msgr2=0x7f5d0c104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:48.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 -- 172.21.15.42:0/1035270758 shutdown_connections 2024-05-27T10:12:48.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.326+0000 7f5d14000700 1 -- 172.21.15.42:0/1035270758 wait complete. 2024-05-27T10:12:48.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 Processor -- start 2024-05-27T10:12:48.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 -- start start 2024-05-27T10:12:48.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d0c106540 0x7f5d0c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:48.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c107670 0x7f5d0c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:48.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 0x7f5d0c1a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:48.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c111e50 con 0x7f5d0c108010 2024-05-27T10:12:48.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c111cd0 con 0x7f5d0c106540 2024-05-27T10:12:48.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.328+0000 7f5d14000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c111fd0 con 0x7f5d0c107670 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 0x7f5d0c1a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 0x7f5d0c1a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56150/0 (socket says 172.21.15.42:56150) 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 -- 172.21.15.42:0/828048262 learned_addr learned my addr 172.21.15.42:0/828048262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d10c00700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c107670 0x7f5d0c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d11600700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d0c106540 0x7f5d0c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 -- 172.21.15.42:0/828048262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c107670 msgr2=0x7f5d0c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c107670 0x7f5d0c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 -- 172.21.15.42:0/828048262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d0c106540 msgr2=0x7f5d0c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d0c106540 0x7f5d0c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.329+0000 7f5d12000700 1 -- 172.21.15.42:0/828048262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c1a44f0 con 0x7f5d0c108010 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.330+0000 7f5d12000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 0x7f5d0c1a3e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d04004d00 tx=0x7f5d04004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:48.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.330+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04021070 con 0x7f5d0c108010 2024-05-27T10:12:48.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.330+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d04007b80 con 0x7f5d0c108010 2024-05-27T10:12:48.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.330+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c1a4780 con 0x7f5d0c108010 2024-05-27T10:12:48.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.330+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04005040 con 0x7f5d0c108010 2024-05-27T10:12:48.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.330+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1a4bd0 con 0x7f5d0c108010 2024-05-27T10:12:48.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.332+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c0512b0 con 0x7f5d0c108010 2024-05-27T10:12:48.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.334+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5d04024070 con 0x7f5d0c108010 2024-05-27T10:12:48.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.336+0000 7f5d0a000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5cf40b2880 0x7f5cf40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:48.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.336+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f5d0400f020 con 0x7f5d0c108010 2024-05-27T10:12:48.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.336+0000 7f5d11600700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5cf40b2880 0x7f5cf40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:48.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.337+0000 7f5d11600700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5cf40b2880 0x7f5cf40b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c107570 tx=0x7f5d00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.338+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d0402b020 con 0x7f5d0c108010 2024-05-27T10:12:48.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.575+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5d0c0643c0 con 0x7f5d0c108010 2024-05-27T10:12:48.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.576+0000 7f5d0a000700 1 -- 172.21.15.42:0/828048262 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f5d040ab860 con 0x7f5d0c108010 2024-05-27T10:12:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:12:48.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.579+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5cf40b2880 msgr2=0x7f5cf40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.579+0000 7f5d14000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5cf40b2880 0x7f5cf40b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c107570 tx=0x7f5d00009260 comp rx=0 tx=0).stop 2024-05-27T10:12:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.579+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 msgr2=0x7f5d0c1a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.579+0000 7f5d14000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 0x7f5d0c1a3e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d04004d00 tx=0x7f5d04004d30 comp rx=0 tx=0).stop 2024-05-27T10:12:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 shutdown_connections 2024-05-27T10:12:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0c108010 0x7f5d0c1a3e70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5cf40b2880 0x7f5cf40b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d0c106540 0x7f5d0c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 --2- 172.21.15.42:0/828048262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c107670 0x7f5d0c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 >> 172.21.15.42:0/828048262 conn(0x7f5d0c101bc0 msgr2=0x7f5d0c10a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:48.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 shutdown_connections 2024-05-27T10:12:48.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:48.580+0000 7f5d14000700 1 -- 172.21.15.42:0/828048262 wait complete. 2024-05-27T10:12:49.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:49 smithi042 ceph-mon[31661]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:49.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:49 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/828048262' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T10:12:49.659 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1716804765,"num_in_osds":12,"osd_in_since":1716804739,"num_remapped_pgs":0} 2024-05-27T10:12:49.659 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:12:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:49 smithi104 ceph-mon[34812]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:49.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:49 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/828048262' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T10:12:49.735 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:49 smithi143 ceph-mon[35545]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:49.735 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:49 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/828048262' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T10:12:50.103 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:12:51.204 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: Detected new or changed devices on smithi143 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi143 to 4272M 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:50 smithi104 ceph-mon[34812]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: Detected new or changed devices on smithi143 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi143 to 4272M 2024-05-27T10:12:51.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:51.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:51.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:50 smithi042 ceph-mon[31661]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:51.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: Detected new or changed devices on smithi143 2024-05-27T10:12:51.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi143 to 4272M 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:12:51.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:50 smithi143 ceph-mon[35545]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:52.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 -- 172.21.15.42:0/2271746729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105300 msgr2=0x7efe38105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:52.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 --2- 172.21.15.42:0/2271746729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105300 0x7efe38105710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe28009990 tx=0x7efe28009ca0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 -- 172.21.15.42:0/2271746729 shutdown_connections 2024-05-27T10:12:52.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 --2- 172.21.15.42:0/2271746729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 0x7efe38110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 --2- 172.21.15.42:0/2271746729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105300 0x7efe38105710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 --2- 172.21.15.42:0/2271746729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe38105c50 0x7efe3810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 -- 172.21.15.42:0/2271746729 >> 172.21.15.42:0/2271746729 conn(0x7efe38078ab0 msgr2=0x7efe38078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 -- 172.21.15.42:0/2271746729 shutdown_connections 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.704+0000 7efe3d000700 1 -- 172.21.15.42:0/2271746729 wait complete. 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.706+0000 7efe3d000700 1 Processor -- start 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.706+0000 7efe3d000700 1 -- start start 2024-05-27T10:12:52.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe3d000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe38105300 0x7efe3819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe3d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 0x7efe3819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe3d000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 0x7efe381a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe3d000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe38114140 con 0x7efe38105c50 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe3d000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe38113fc0 con 0x7efe38105300 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe3d000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe381142c0 con 0x7efe3810e730 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe36a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 0x7efe3819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe36a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 0x7efe3819d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56178/0 (socket says 172.21.15.42:56178) 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.707+0000 7efe36a00700 1 -- 172.21.15.42:0/3388570083 learned_addr learned my addr 172.21.15.42:0/3388570083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe37400700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe38105300 0x7efe3819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe37e00700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 0x7efe381a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe36a00700 1 -- 172.21.15.42:0/3388570083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 msgr2=0x7efe381a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe36a00700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 0x7efe381a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe36a00700 1 -- 172.21.15.42:0/3388570083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe38105300 msgr2=0x7efe3819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:52.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe36a00700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe38105300 0x7efe3819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe36a00700 1 -- 172.21.15.42:0/3388570083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381a1c00 con 0x7efe38105c50 2024-05-27T10:12:52.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe37e00700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 0x7efe381a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:12:52.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe36a00700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 0x7efe3819d2d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe2400d950 tx=0x7efe2400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:52.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24016070 con 0x7efe38105c50 2024-05-27T10:12:52.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.708+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe24010460 con 0x7efe38105c50 2024-05-27T10:12:52.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.709+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe381a1ef0 con 0x7efe38105c50 2024-05-27T10:12:52.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.709+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2400f620 con 0x7efe38105c50 2024-05-27T10:12:52.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.709+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381a9810 con 0x7efe38105c50 2024-05-27T10:12:52.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.710+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe380512b0 con 0x7efe38105c50 2024-05-27T10:12:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.716+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efe24019070 con 0x7efe38105c50 2024-05-27T10:12:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.717+0000 7efe2fe00700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efe1c0b2900 0x7efe1c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:52.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.718+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7efe240e1d50 con 0x7efe38105c50 2024-05-27T10:12:52.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.718+0000 7efe37400700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efe1c0b2900 0x7efe1c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:52.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.719+0000 7efe37400700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efe1c0b2900 0x7efe1c0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b5c0 tx=0x7efe28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:52.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.720+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe240a9e70 con 0x7efe38105c50 2024-05-27T10:12:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.961+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe381a99e0 con 0x7efe38105c50 2024-05-27T10:12:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.964+0000 7efe2fe00700 1 -- 172.21.15.42:0/3388570083 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18469 (secure 0 0 0) 0x7efe240a9770 con 0x7efe38105c50 2024-05-27T10:12:52.964 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:12:52.964 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":72,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:12:46.403666+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:12:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efe1c0b2900 msgr2=0x7efe1c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efe1c0b2900 0x7efe1c0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b5c0 tx=0x7efe28005fb0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 msgr2=0x7efe3819d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 0x7efe3819d2d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe2400d950 tx=0x7efe2400dd10 comp rx=0 tx=0).stop 2024-05-27T10:12:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 shutdown_connections 2024-05-27T10:12:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3810e730 0x7efe381a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efe1c0b2900 0x7efe1c0b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe38105300 0x7efe3819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 --2- 172.21.15.42:0/3388570083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe38105c50 0x7efe3819d2d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 >> 172.21.15.42:0/3388570083 conn(0x7efe38078ab0 msgr2=0x7efe38104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 shutdown_connections 2024-05-27T10:12:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:52.967+0000 7efe3d000700 1 -- 172.21.15.42:0/3388570083 wait complete. 2024-05-27T10:12:53.669 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:53 smithi143 ceph-mon[35545]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:53.669 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:53 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3388570083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:12:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:53 smithi104 ceph-mon[34812]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:53.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:53 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3388570083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:12:53.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:53 smithi042 ceph-mon[31661]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:53.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:53 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3388570083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:12:53.856 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T10:07:06.385341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T10:12:53.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-27T10:12:54.317 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:12:55.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:55 smithi104 ceph-mon[34812]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:55.728 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:55 smithi042 ceph-mon[31661]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:55.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:55 smithi143 ceph-mon[35545]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:56.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 -- 172.21.15.42:0/1451064413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 msgr2=0x7fc714110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:56.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 --2- 172.21.15.42:0/1451064413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc714110d70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc708009990 tx=0x7fc708009ca0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 -- 172.21.15.42:0/1451064413 shutdown_connections 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 --2- 172.21.15.42:0/1451064413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71410b2d0 0x7fc714117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 --2- 172.21.15.42:0/1451064413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc714110d70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 --2- 172.21.15.42:0/1451064413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71410a980 0x7fc71410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 -- 172.21.15.42:0/1451064413 >> 172.21.15.42:0/1451064413 conn(0x7fc714077c70 msgr2=0x7fc7140780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 -- 172.21.15.42:0/1451064413 shutdown_connections 2024-05-27T10:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.677+0000 7fc71c200700 1 -- 172.21.15.42:0/1451064413 wait complete. 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.678+0000 7fc71c200700 1 Processor -- start 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 -- start start 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71410a980 0x7fc7141a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71410b2d0 0x7fc7141a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc7141a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc71411a910 con 0x7fc714110960 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc71411a790 con 0x7fc71410a980 2024-05-27T10:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.679+0000 7fc71c200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc71411aa90 con 0x7fc71410b2d0 2024-05-27T10:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc7141a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc7141a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51956/0 (socket says 172.21.15.42:51956) 2024-05-27T10:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 -- 172.21.15.42:0/1716700828 learned_addr learned my addr 172.21.15.42:0/1716700828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc719800700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71410a980 0x7fc7141a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc718e00700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71410b2d0 0x7fc7141a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 -- 172.21.15.42:0/1716700828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71410b2d0 msgr2=0x7fc7141a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71410b2d0 0x7fc7141a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 -- 172.21.15.42:0/1716700828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71410a980 msgr2=0x7fc7141a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71410a980 0x7fc7141a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 -- 172.21.15.42:0/1716700828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141a6270 con 0x7fc714110960 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.680+0000 7fc71a200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc7141a5bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc70c007ee0 tx=0x7fc70c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.681+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004bb0 con 0x7fc714110960 2024-05-27T10:12:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.681+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc70c004d50 con 0x7fc714110960 2024-05-27T10:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.681+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0058b0 con 0x7fc714110960 2024-05-27T10:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.681+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7141a6560 con 0x7fc714110960 2024-05-27T10:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.681+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7141ade80 con 0x7fc714110960 2024-05-27T10:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.682+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71410c070 con 0x7fc714110960 2024-05-27T10:12:56.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.685+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc70c01a020 con 0x7fc714110960 2024-05-27T10:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.687+0000 7fc712000700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6fc0b2900 0x7fc6fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:12:56.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.687+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc70c0e1370 con 0x7fc714110960 2024-05-27T10:12:56.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.687+0000 7fc719800700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6fc0b2900 0x7fc6fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:12:56.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.688+0000 7fc719800700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6fc0b2900 0x7fc6fc0b4e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b2e0 tx=0x7fc708005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:12:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.692+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc70c010070 con 0x7fc714110960 2024-05-27T10:12:56.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.929+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc71402e5e0 con 0x7fc714110960 2024-05-27T10:12:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.930+0000 7fc712000700 1 -- 172.21.15.42:0/1716700828 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fc70c0a9490 con 0x7fc714110960 2024-05-27T10:12:56.930 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-27T10:12:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6fc0b2900 msgr2=0x7fc6fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6fc0b2900 0x7fc6fc0b4e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b2e0 tx=0x7fc708005fb0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 msgr2=0x7fc7141a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:12:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc7141a5bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc70c007ee0 tx=0x7fc70c00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 shutdown_connections 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc714110960 0x7fc7141a5bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6fc0b2900 0x7fc6fc0b4e10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc71410a980 0x7fc7141a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 --2- 172.21.15.42:0/1716700828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71410b2d0 0x7fc7141a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.933+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 >> 172.21.15.42:0/1716700828 conn(0x7fc714077c70 msgr2=0x7fc71407bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.934+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 shutdown_connections 2024-05-27T10:12:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:12:56.934+0000 7fc71c200700 1 -- 172.21.15.42:0/1716700828 wait complete. 2024-05-27T10:12:57.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:57 smithi143 ceph-mon[35545]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:57.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:57 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1716700828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:12:57.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:57 smithi042 ceph-mon[31661]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:57.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:57 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1716700828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:12:57.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:57 smithi104 ceph-mon[34812]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:57.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:57 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1716700828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:12:57.783 INFO:tasks.cephadm:Adding mds.a on smithi042 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.d on smithi042 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.g on smithi042 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.j on smithi042 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.b on smithi104 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.e on smithi104 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.h on smithi104 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.k on smithi104 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.c on smithi143 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.f on smithi143 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.i on smithi143 2024-05-27T10:12:57.784 INFO:tasks.cephadm:Adding mds.l on smithi143 2024-05-27T10:12:57.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l' 2024-05-27T10:12:58.197 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:12:59.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:12:59 smithi143 ceph-mon[35545]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:59.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:12:59 smithi104 ceph-mon[34812]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:12:59.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:12:59 smithi042 ceph-mon[31661]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:00.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 -- 172.21.15.143:0/1419966646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105e00 msgr2=0x7f7cac10c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 --2- 172.21.15.143:0/1419966646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105e00 0x7f7cac10c6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400b0a0 tx=0x7f7ca400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 -- 172.21.15.143:0/1419966646 shutdown_connections 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 --2- 172.21.15.143:0/1419966646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105e00 0x7f7cac10c6b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 --2- 172.21.15.143:0/1419966646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cac104330 0x7f7cac104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 --2- 172.21.15.143:0/1419966646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105460 0x7f7cac105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 -- 172.21.15.143:0/1419966646 >> 172.21.15.143:0/1419966646 conn(0x7f7cac0ff9b0 msgr2=0x7f7cac101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 -- 172.21.15.143:0/1419966646 shutdown_connections 2024-05-27T10:13:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.477+0000 7f7cb4000700 1 -- 172.21.15.143:0/1419966646 wait complete. 2024-05-27T10:13:00.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.478+0000 7f7cb4000700 1 Processor -- start 2024-05-27T10:13:00.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.478+0000 7f7cb4000700 1 -- start start 2024-05-27T10:13:00.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb4000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cac104330 0x7f7cac19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:00.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb4000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 0x7f7cac19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:00.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb4000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105e00 0x7f7cac1a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb4000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac10fc50 con 0x7f7cac104330 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb4000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac10fad0 con 0x7f7cac105e00 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb4000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac10fdd0 con 0x7f7cac105460 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb0c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 0x7f7cac19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb0c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 0x7f7cac19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44736/0 (socket says 172.21.15.143:44736) 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb0c00700 1 -- 172.21.15.143:0/3223324580 learned_addr learned my addr 172.21.15.143:0/3223324580 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb1600700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cac104330 0x7f7cac19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.479+0000 7f7cb2000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105e00 0x7f7cac1a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb0c00700 1 -- 172.21.15.143:0/3223324580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105e00 msgr2=0x7f7cac1a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb0c00700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105e00 0x7f7cac1a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb0c00700 1 -- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cac104330 msgr2=0x7f7cac19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb0c00700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cac104330 0x7f7cac19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb0c00700 1 -- 172.21.15.143:0/3223324580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac1a4530 con 0x7f7cac105460 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb2000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105e00 0x7f7cac1a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:13:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.480+0000 7f7cb0c00700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 0x7f7cac19cf10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d950 tx=0x7f7c9800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.481+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98016070 con 0x7f7cac105460 2024-05-27T10:13:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.481+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c98009dd0 con 0x7f7cac105460 2024-05-27T10:13:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.481+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac1a4820 con 0x7f7cac105460 2024-05-27T10:13:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.481+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1a4d60 con 0x7f7cac105460 2024-05-27T10:13:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.481+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800f3c0 con 0x7f7cac105460 2024-05-27T10:13:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.482+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac0512b0 con 0x7f7cac105460 2024-05-27T10:13:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.487+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7c98019070 con 0x7f7cac105460 2024-05-27T10:13:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.489+0000 7f7caa000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7c940b2880 0x7f7c940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.489+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7c980e1a20 con 0x7f7cac105460 2024-05-27T10:13:00.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.489+0000 7f7cb1600700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7c940b2880 0x7f7c940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.491+0000 7f7cb1600700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7c940b2880 0x7f7c940b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cac105360 tx=0x7f7ca000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.491+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c980a9b40 con 0x7f7cac105460 2024-05-27T10:13:00.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.729+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}) -- 0x7f7cac077200 con 0x7f7c940b2880 2024-05-27T10:13:00.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.748+0000 7f7caa000700 1 -- 172.21.15.143:0/3223324580 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7cac077200 con 0x7f7c940b2880 2024-05-27T10:13:00.749 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mds.all update... 2024-05-27T10:13:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7c940b2880 msgr2=0x7f7c940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7c940b2880 0x7f7c940b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cac105360 tx=0x7f7ca000b410 comp rx=0 tx=0).stop 2024-05-27T10:13:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 msgr2=0x7f7cac19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 0x7f7cac19cf10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d950 tx=0x7f7c9800dc60 comp rx=0 tx=0).stop 2024-05-27T10:13:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 shutdown_connections 2024-05-27T10:13:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7c940b2880 0x7f7c940b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cac105e00 0x7f7cac1a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cac104330 0x7f7cac19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 --2- 172.21.15.143:0/3223324580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7cac105460 0x7f7cac19cf10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 >> 172.21.15.143:0/3223324580 conn(0x7f7cac0ff9b0 msgr2=0x7f7cac0759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 shutdown_connections 2024-05-27T10:13:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-27T10:13:00.752+0000 7f7cb4000700 1 -- 172.21.15.143:0/3223324580 wait complete. 2024-05-27T10:13:01.600 DEBUG:teuthology.orchestra.run.smithi042:mds.a> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.a.service 2024-05-27T10:13:01.608 DEBUG:teuthology.orchestra.run.smithi042:mds.d> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.d.service 2024-05-27T10:13:01.654 DEBUG:teuthology.orchestra.run.smithi042:mds.g> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.g.service 2024-05-27T10:13:01.701 DEBUG:teuthology.orchestra.run.smithi042:mds.j> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.j.service 2024-05-27T10:13:01.718 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:01.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:01.722 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.b.service 2024-05-27T10:13:01.730 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.e.service 2024-05-27T10:13:01.777 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.h.service 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:01.802 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:01.805 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service 2024-05-27T10:13:01.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:01.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:01.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:13:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:13:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:01.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:01.851 DEBUG:teuthology.orchestra.run.smithi143:mds.c> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.c.service 2024-05-27T10:13:01.858 DEBUG:teuthology.orchestra.run.smithi143:mds.f> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.f.service 2024-05-27T10:13:01.904 DEBUG:teuthology.orchestra.run.smithi143:mds.i> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.i.service 2024-05-27T10:13:01.949 DEBUG:teuthology.orchestra.run.smithi143:mds.l> sudo journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.l.service 2024-05-27T10:13:01.994 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-27T10:13:01.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:02.447 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:02.646 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:02 smithi042 ceph-mon[31661]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:13:02.646 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:02 smithi042 ceph-mon[31661]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-27T10:13:02.646 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:02 smithi042 ceph-mon[31661]: Deploying daemon mds.l on smithi143 2024-05-27T10:13:02.722 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:02 smithi143 ceph-mon[35545]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:13:02.722 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:02 smithi143 ceph-mon[35545]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-27T10:13:02.722 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:02 smithi143 ceph-mon[35545]: Deploying daemon mds.l on smithi143 2024-05-27T10:13:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:02 smithi104 ceph-mon[34812]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:13:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:02 smithi104 ceph-mon[34812]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-27T10:13:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:02 smithi104 ceph-mon[34812]: Deploying daemon mds.l on smithi143 2024-05-27T10:13:03.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:03 smithi143 ceph-mon[35545]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:03.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:03 smithi042 ceph-mon[31661]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:03 smithi104 ceph-mon[34812]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:04.259 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:03 smithi143 systemd[1]: Starting Ceph mds.l for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:04.534 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 podman[65723]: 2024-05-27 10:13:04.163290935 +0000 UTC m=+0.035439977 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:04.535 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 podman[65723]: 2024-05-27 10:13:04.271091079 +0000 UTC m=+0.143240153 container create c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS) 2024-05-27T10:13:04.535 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 podman[65723]: 2024-05-27 10:13:04.533440115 +0000 UTC m=+0.405589172 container init c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T10:13:04.808 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 podman[65723]: 2024-05-27 10:13:04.544364361 +0000 UTC m=+0.416513442 container start c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T10:13:04.809 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 bash[65723]: c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d 2024-05-27T10:13:04.809 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 systemd[1]: Started Ceph mds.l for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:04.809 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 ceph-mds[65742]: -- 172.21.15.143:0/411153679 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3957efa40 con 0x55e395837600 2024-05-27T10:13:04.809 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:13:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l[65738]: starting mds.l at 2024-05-27T10:13:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.824+0000 7f79cb200700 1 -- 172.21.15.42:0/1776673504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 msgr2=0x7f79cc105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.824+0000 7f79cb200700 1 --2- 172.21.15.42:0/1776673504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc105690 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009990 tx=0x7f79bc009ca0 comp rx=0 tx=0).stop 2024-05-27T10:13:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 -- 172.21.15.42:0/1776673504 shutdown_connections 2024-05-27T10:13:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 --2- 172.21.15.42:0/1776673504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79cc10e6f0 0x7f79cc110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 --2- 172.21.15.42:0/1776673504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc105690 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 --2- 172.21.15.42:0/1776673504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79cc105bd0 0x7f79cc10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 -- 172.21.15.42:0/1776673504 >> 172.21.15.42:0/1776673504 conn(0x7f79cc078ab0 msgr2=0x7f79cc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 -- 172.21.15.42:0/1776673504 shutdown_connections 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.825+0000 7f79cb200700 1 -- 172.21.15.42:0/1776673504 wait complete. 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.826+0000 7f79cb200700 1 Processor -- start 2024-05-27T10:13:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.826+0000 7f79cb200700 1 -- start start 2024-05-27T10:13:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79cb200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79cb200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79cc105bd0 0x7f79cc1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79cb200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79cc10e6f0 0x7f79cc1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79cb200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1141c0 con 0x7f79cc105280 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79cb200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc114040 con 0x7f79cc105bd0 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79cb200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc114340 con 0x7f79cc10e6f0 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79c9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79c9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39024/0 (socket says 172.21.15.42:39024) 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.827+0000 7f79c9e00700 1 -- 172.21.15.42:0/1962989070 learned_addr learned my addr 172.21.15.42:0/1962989070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79ca800700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79cc10e6f0 0x7f79cc1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79c9400700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79cc105bd0 0x7f79cc1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79c9e00700 1 -- 172.21.15.42:0/1962989070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79cc10e6f0 msgr2=0x7f79cc1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79c9e00700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79cc10e6f0 0x7f79cc1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79c9e00700 1 -- 172.21.15.42:0/1962989070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79cc105bd0 msgr2=0x7f79cc1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:04.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79c9e00700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79cc105bd0 0x7f79cc1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:04.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.828+0000 7f79c9e00700 1 -- 172.21.15.42:0/1962989070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc1a8f60 con 0x7f79cc105280 2024-05-27T10:13:04.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.829+0000 7f79c9e00700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc102f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005910 tx=0x7f79bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:04.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.831+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc027070 con 0x7f79cc105280 2024-05-27T10:13:04.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.831+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79bc00fc00 con 0x7f79cc105280 2024-05-27T10:13:04.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.831+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc1a91f0 con 0x7f79cc105280 2024-05-27T10:13:04.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.831+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc022dc0 con 0x7f79cc105280 2024-05-27T10:13:04.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.831+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc1a9720 con 0x7f79cc105280 2024-05-27T10:13:04.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.833+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79cc0512b0 con 0x7f79cc105280 2024-05-27T10:13:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.836+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f79bc02a070 con 0x7f79cc105280 2024-05-27T10:13:04.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.838+0000 7f79c2a00700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f79b00b2900 0x7f79b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:04.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.838+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f79bc0e2c50 con 0x7f79cc105280 2024-05-27T10:13:04.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.838+0000 7f79c9400700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f79b00b2900 0x7f79b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:04.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.839+0000 7f79c9400700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f79b00b2900 0x7f79b00b4e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79cc1027b0 tx=0x7f79b800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:04.843+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79bc013070 con 0x7f79cc105280 2024-05-27T10:13:05.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.136+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79cc02e5e0 con 0x7f79cc105280 2024-05-27T10:13:05.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.137+0000 7f79c2a00700 1 -- 172.21.15.42:0/1962989070 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f79bc0a4e20 con 0x7f79cc105280 2024-05-27T10:13:05.138 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f79b00b2900 msgr2=0x7f79b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f79b00b2900 0x7f79b00b4e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79cc1027b0 tx=0x7f79b800a380 comp rx=0 tx=0).stop 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 msgr2=0x7f79cc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc102f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005910 tx=0x7f79bc0047b0 comp rx=0 tx=0).stop 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 shutdown_connections 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79cc10e6f0 0x7f79cc1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79cc105280 0x7f79cc102f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f79b00b2900 0x7f79b00b4e10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 --2- 172.21.15.42:0/1962989070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79cc105bd0 0x7f79cc1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:05.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 >> 172.21.15.42:0/1962989070 conn(0x7f79cc078ab0 msgr2=0x7f79cc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:05.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 shutdown_connections 2024-05-27T10:13:05.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:05.141+0000 7f79cb200700 1 -- 172.21.15.42:0/1962989070 wait complete. 2024-05-27T10:13:05.143 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-05-27T10:13:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.143:6832/3395724894,v1:172.21.15.143:6833/3395724894] up:boot 2024-05-27T10:13:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: fsmap 1 up:standby 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:05.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:05 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1962989070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.143:6832/3395724894,v1:172.21.15.143:6833/3395724894] up:boot 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: fsmap 1 up:standby 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:05.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:05.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:05.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:05.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:05 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1962989070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.143:6832/3395724894,v1:172.21.15.143:6833/3395724894] up:boot 2024-05-27T10:13:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: fsmap 1 up:standby 2024-05-27T10:13:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T10:13:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:06.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:05 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1962989070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:06.916 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:06 smithi104 ceph-mon[34812]: Deploying daemon mds.k on smithi104 2024-05-27T10:13:07.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:06 smithi143 ceph-mon[35545]: Deploying daemon mds.k on smithi104 2024-05-27T10:13:07.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:06 smithi042 ceph-mon[31661]: Deploying daemon mds.k on smithi104 2024-05-27T10:13:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:07 smithi104 ceph-mon[34812]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:08.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:08.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:07 smithi143 ceph-mon[35545]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:08.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:07 smithi042 ceph-mon[31661]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:08.335 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:07 smithi104 systemd[1]: Starting Ceph mds.k for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:08.488 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:08.698 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 podman[64224]: 2024-05-27 10:13:08.238786354 +0000 UTC m=+0.030985778 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:08.699 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 podman[64224]: 2024-05-27 10:13:08.378807728 +0000 UTC m=+0.171007219 container create 53fa28eed6215ffe279d89ea8e94a03f8077d06536a8c96bf550b223053ba4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T10:13:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:08 smithi143 ceph-mon[35545]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:08.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:08 smithi042 ceph-mon[31661]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:08 smithi104 ceph-mon[34812]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:09.199 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 podman[64224]: 2024-05-27 10:13:08.708232832 +0000 UTC m=+0.500432343 container init 53fa28eed6215ffe279d89ea8e94a03f8077d06536a8c96bf550b223053ba4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:13:09.199 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 podman[64224]: 2024-05-27 10:13:08.718479072 +0000 UTC m=+0.510678515 container start 53fa28eed6215ffe279d89ea8e94a03f8077d06536a8c96bf550b223053ba4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:13:09.199 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 bash[64224]: 53fa28eed6215ffe279d89ea8e94a03f8077d06536a8c96bf550b223053ba4b8 2024-05-27T10:13:09.199 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 systemd[1]: Started Ceph mds.k for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:09.199 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 ceph-mds[64243]: -- 172.21.15.104:0/4089981706 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620072d3a40 con 0x56200731ba80 2024-05-27T10:13:09.199 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:13:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: starting mds.k at 2024-05-27T10:13:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: Deploying daemon mds.j on smithi042 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6832/3150585498,v1:172.21.15.104:6833/3150585498] up:boot 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: fsmap 2 up:standby 2024-05-27T10:13:10.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: Deploying daemon mds.j on smithi042 2024-05-27T10:13:10.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6832/3150585498,v1:172.21.15.104:6833/3150585498] up:boot 2024-05-27T10:13:10.268 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: fsmap 2 up:standby 2024-05-27T10:13:10.268 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:13:10.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: Deploying daemon mds.j on smithi042 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6832/3150585498,v1:172.21.15.104:6833/3150585498] up:boot 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: fsmap 2 up:standby 2024-05-27T10:13:10.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:13:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.907+0000 7f1097200700 1 -- 172.21.15.42:0/575287061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10981043f0 msgr2=0x7f1098104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.907+0000 7f1097200700 1 --2- 172.21.15.42:0/575287061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10981043f0 0x7f1098104800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10880099c0 tx=0x7f1088009cd0 comp rx=0 tx=0).stop 2024-05-27T10:13:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 -- 172.21.15.42:0/575287061 shutdown_connections 2024-05-27T10:13:10.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 --2- 172.21.15.42:0/575287061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098109670 0x7f1098110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:10.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 --2- 172.21.15.42:0/575287061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10981043f0 0x7f1098104800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:10.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 --2- 172.21.15.42:0/575287061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1098108d20 0x7f1098109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:10.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 -- 172.21.15.42:0/575287061 >> 172.21.15.42:0/575287061 conn(0x7f1098079d00 msgr2=0x7f109807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:10.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 -- 172.21.15.42:0/575287061 shutdown_connections 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.908+0000 7f1097200700 1 -- 172.21.15.42:0/575287061 wait complete. 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.909+0000 7f1097200700 1 Processor -- start 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.909+0000 7f1097200700 1 -- start start 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1097200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981043f0 0x7f109819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1097200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098108d20 0x7f109819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1097200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 0x7f10981a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1097200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1098114220 con 0x7f1098109670 2024-05-27T10:13:10.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1097200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f10981140a0 con 0x7f10981043f0 2024-05-27T10:13:10.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1097200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f10981143a0 con 0x7f1098108d20 2024-05-27T10:13:10.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1096800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 0x7f10981a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:10.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1096800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 0x7f10981a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39040/0 (socket says 172.21.15.42:39040) 2024-05-27T10:13:10.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1096800700 1 -- 172.21.15.42:0/2751121628 learned_addr learned my addr 172.21.15.42:0/2751121628 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:10.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f1095e00700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981043f0 0x7f109819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:10.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.910+0000 7f108d400700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098108d20 0x7f109819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:10.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f1096800700 1 -- 172.21.15.42:0/2751121628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098108d20 msgr2=0x7f109819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:10.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f1096800700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098108d20 0x7f109819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f1096800700 1 -- 172.21.15.42:0/2751121628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981043f0 msgr2=0x7f109819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f1096800700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981043f0 0x7f109819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f1096800700 1 -- 172.21.15.42:0/2751121628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981a1d20 con 0x7f1098109670 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f108d400700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098108d20 0x7f109819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f1096800700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 0x7f10981a1680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f109000b970 tx=0x7f109000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090019070 con 0x7f1098109670 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.911+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f109000cdc0 con 0x7f1098109670 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.912+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981a2010 con 0x7f1098109670 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.912+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090014e30 con 0x7f1098109670 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.912+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a9930 con 0x7f1098109670 2024-05-27T10:13:10.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.913+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10981058a0 con 0x7f1098109670 2024-05-27T10:13:10.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.916+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f109001c070 con 0x7f1098109670 2024-05-27T10:13:10.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.917+0000 7f108f400700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f107c0b2880 0x7f107c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:10.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.917+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f10900e1600 con 0x7f1098109670 2024-05-27T10:13:10.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.918+0000 7f1095e00700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f107c0b2880 0x7f107c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:10.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.918+0000 7f1095e00700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f107c0b2880 0x7f107c0b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1088000c00 tx=0x7f1088005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:10.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:10.920+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10900a9720 con 0x7f1098109670 2024-05-27T10:13:11.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:10 smithi042 ceph-mon[31661]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:11.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:10 smithi104 ceph-mon[34812]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.219+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10980512b0 con 0x7f1098109670 2024-05-27T10:13:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.220+0000 7f108f400700 1 -- 172.21.15.42:0/2751121628 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f10980512b0 con 0x7f1098109670 2024-05-27T10:13:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:11.221 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T10:13:11.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f107c0b2880 msgr2=0x7f107c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:11.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f107c0b2880 0x7f107c0b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1088000c00 tx=0x7f1088005fb0 comp rx=0 tx=0).stop 2024-05-27T10:13:11.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 msgr2=0x7f10981a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:11.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 0x7f10981a1680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f109000b970 tx=0x7f109000bc80 comp rx=0 tx=0).stop 2024-05-27T10:13:11.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 shutdown_connections 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1098109670 0x7f10981a1680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f107c0b2880 0x7f107c0b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981043f0 0x7f109819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 --2- 172.21.15.42:0/2751121628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1098108d20 0x7f109819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.224+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 >> 172.21.15.42:0/2751121628 conn(0x7f1098079d00 msgr2=0x7f1098101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.225+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 shutdown_connections 2024-05-27T10:13:11.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:11.225+0000 7f1097200700 1 -- 172.21.15.42:0/2751121628 wait complete. 2024-05-27T10:13:11.226 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-05-27T10:13:11.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:10 smithi143 ceph-mon[35545]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:11 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2751121628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:12.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:11 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2751121628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:12.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:11 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2751121628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:13.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:12 smithi104 ceph-mon[34812]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:13.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:12 smithi143 ceph-mon[35545]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:13.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:12 smithi042 ceph-mon[31661]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:13.727 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:13 smithi042 systemd[1]: Starting Ceph mds.j for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:14.077 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:13 smithi042 podman[73026]: 2024-05-27 10:13:13.630868671 +0000 UTC m=+0.036738561 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:14.077 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:13 smithi042 podman[73026]: 2024-05-27 10:13:13.77199941 +0000 UTC m=+0.177869331 container create 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T10:13:14.077 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:14 smithi042 podman[73026]: 2024-05-27 10:13:14.076015422 +0000 UTC m=+0.481885390 container init 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:13:14.344 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:14 smithi042 podman[73026]: 2024-05-27 10:13:14.087571707 +0000 UTC m=+0.493441605 container start 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521) 2024-05-27T10:13:14.344 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:14 smithi042 bash[73026]: 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 2024-05-27T10:13:14.344 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:14 smithi042 systemd[1]: Started Ceph mds.j for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:14.344 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:14 smithi042 ceph-mds[73045]: -- 172.21.15.42:0/661570284 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f5ac75a40 con 0x563f5acbd180 2024-05-27T10:13:14.344 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:13:14 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j[73041]: starting mds.j at 2024-05-27T10:13:14.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.42:6834/781814611,v1:172.21.15.42:6835/781814611] up:boot 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: fsmap 3 up:standby 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: Deploying daemon mds.d on smithi042 2024-05-27T10:13:15.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:15 smithi042 ceph-mon[31661]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:15.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.42:6834/781814611,v1:172.21.15.42:6835/781814611] up:boot 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: fsmap 3 up:standby 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: Deploying daemon mds.d on smithi042 2024-05-27T10:13:15.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:15 smithi143 ceph-mon[35545]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.42:6834/781814611,v1:172.21.15.42:6835/781814611] up:boot 2024-05-27T10:13:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: fsmap 3 up:standby 2024-05-27T10:13:15.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: Deploying daemon mds.d on smithi042 2024-05-27T10:13:15.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:15 smithi104 ceph-mon[34812]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:16.727 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:17.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:17 smithi042 ceph-mon[31661]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:17.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:17 smithi104 ceph-mon[34812]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:17.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:17 smithi143 ceph-mon[35545]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:18.594 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:18 smithi042 podman[73432]: 2024-05-27 10:13:18.449925396 +0000 UTC m=+0.041720646 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:18.906 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:18 smithi042 podman[73432]: 2024-05-27 10:13:18.620519885 +0000 UTC m=+0.212315167 container create 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:13:19.219 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:18 smithi042 podman[73432]: 2024-05-27 10:13:18.917108045 +0000 UTC m=+0.508903368 container init 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-27T10:13:19.219 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:18 smithi042 podman[73432]: 2024-05-27 10:13:18.927925771 +0000 UTC m=+0.519721044 container start 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:13:19.219 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:19 smithi042 bash[73432]: 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc 2024-05-27T10:13:19.220 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:19 smithi042 systemd[1]: Started Ceph mds.d for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:19.220 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mds[73467]: -- 172.21.15.42:0/548193896 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2907ffa40 con 0x55a290847600 2024-05-27T10:13:19.220 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:13:19 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d[73462]: starting mds.d at 2024-05-27T10:13:19.474 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:19.474 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:19.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:19.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:19 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:19.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:19 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:19.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:19 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:19.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 -- 172.21.15.42:0/466837246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 msgr2=0x7fdbb0110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 --2- 172.21.15.42:0/466837246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb0110d70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009990 tx=0x7fdba0009ca0 comp rx=0 tx=0).stop 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 -- 172.21.15.42:0/466837246 shutdown_connections 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 --2- 172.21.15.42:0/466837246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb010b2d0 0x7fdbb0117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 --2- 172.21.15.42:0/466837246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb0110d70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 --2- 172.21.15.42:0/466837246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb010a980 0x7fdbb010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 -- 172.21.15.42:0/466837246 >> 172.21.15.42:0/466837246 conn(0x7fdbb0077c70 msgr2=0x7fdbb00780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 -- 172.21.15.42:0/466837246 shutdown_connections 2024-05-27T10:13:19.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.902+0000 7fdbb5600700 1 -- 172.21.15.42:0/466837246 wait complete. 2024-05-27T10:13:19.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.903+0000 7fdbb5600700 1 Processor -- start 2024-05-27T10:13:19.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 -- start start 2024-05-27T10:13:19.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb010a980 0x7fdbb01a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:19.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb010b2d0 0x7fdbb01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb01a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb011a9e0 con 0x7fdbb0110960 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb011a860 con 0x7fdbb010a980 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.904+0000 7fdbb5600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb011ab60 con 0x7fdbb010b2d0 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb01a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb01a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42846/0 (socket says 172.21.15.42:42846) 2024-05-27T10:13:19.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 -- 172.21.15.42:0/4028731052 learned_addr learned my addr 172.21.15.42:0/4028731052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:19.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaea00700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb010a980 0x7fdbb01a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:19.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdba7e00700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb010b2d0 0x7fdbb01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:19.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 -- 172.21.15.42:0/4028731052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb010b2d0 msgr2=0x7fdbb01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:19.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb010b2d0 0x7fdbb01a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 -- 172.21.15.42:0/4028731052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb010a980 msgr2=0x7fdbb01a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb010a980 0x7fdbb01a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.905+0000 7fdbaf400700 1 -- 172.21.15.42:0/4028731052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb01a62e0 con 0x7fdbb0110960 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.906+0000 7fdbaf400700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb01a5c40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdba8007ee0 tx=0x7fdba800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.906+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba800ddd0 con 0x7fdbb0110960 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.906+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdba8004020 con 0x7fdbb0110960 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.906+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8015660 con 0x7fdbb0110960 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.906+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a65d0 con 0x7fdbb0110960 2024-05-27T10:13:19.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.906+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01adef0 con 0x7fdbb0110960 2024-05-27T10:13:19.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.907+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb00512b0 con 0x7fdbb0110960 2024-05-27T10:13:19.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.910+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdba80041d0 con 0x7fdbb0110960 2024-05-27T10:13:19.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.913+0000 7fdba7400700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdb940b2900 0x7fdb940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.913+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fdba80e1fe0 con 0x7fdbb0110960 2024-05-27T10:13:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.913+0000 7fdbaea00700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdb940b2900 0x7fdb940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.914+0000 7fdbaea00700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdb940b2900 0x7fdb940b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006010 tx=0x7fdba000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:19.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:19.916+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdba8010070 con 0x7fdbb0110960 2024-05-27T10:13:20.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.211+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb002e5e0 con 0x7fdbb0110960 2024-05-27T10:13:20.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.212+0000 7fdba7400700 1 -- 172.21.15.42:0/4028731052 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fdba80aa160 con 0x7fdbb0110960 2024-05-27T10:13:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:20.212 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T10:13:20.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdb940b2900 msgr2=0x7fdb940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:20.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdb940b2900 0x7fdb940b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006010 tx=0x7fdba000b540 comp rx=0 tx=0).stop 2024-05-27T10:13:20.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 msgr2=0x7fdbb01a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:20.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb01a5c40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdba8007ee0 tx=0x7fdba800d3b0 comp rx=0 tx=0).stop 2024-05-27T10:13:20.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 shutdown_connections 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb0110960 0x7fdbb01a5c40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdb940b2900 0x7fdb940b4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb010a980 0x7fdbb01a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 --2- 172.21.15.42:0/4028731052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb010b2d0 0x7fdbb01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 >> 172.21.15.42:0/4028731052 conn(0x7fdbb0077c70 msgr2=0x7fdbb007bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 shutdown_connections 2024-05-27T10:13:20.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:20.215+0000 7fdbb5600700 1 -- 172.21.15.42:0/4028731052 wait complete. 2024-05-27T10:13:20.217 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2024-05-27T10:13:20.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:20 smithi042 ceph-mon[31661]: Deploying daemon mds.e on smithi104 2024-05-27T10:13:20.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:20 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.42:6836/1839056289,v1:172.21.15.42:6837/1839056289] up:boot 2024-05-27T10:13:20.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:20 smithi042 ceph-mon[31661]: fsmap 4 up:standby 2024-05-27T10:13:20.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:20 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T10:13:20.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:20 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4028731052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:20.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:20 smithi143 ceph-mon[35545]: Deploying daemon mds.e on smithi104 2024-05-27T10:13:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:20 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.42:6836/1839056289,v1:172.21.15.42:6837/1839056289] up:boot 2024-05-27T10:13:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:20 smithi143 ceph-mon[35545]: fsmap 4 up:standby 2024-05-27T10:13:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:20 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T10:13:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:20 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4028731052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:20 smithi104 ceph-mon[34812]: Deploying daemon mds.e on smithi104 2024-05-27T10:13:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:20 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.42:6836/1839056289,v1:172.21.15.42:6837/1839056289] up:boot 2024-05-27T10:13:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:20 smithi104 ceph-mon[34812]: fsmap 4 up:standby 2024-05-27T10:13:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:20 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T10:13:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:20 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4028731052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:21.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:21 smithi143 ceph-mon[35545]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:21.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:21 smithi042 ceph-mon[31661]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:21 smithi104 ceph-mon[34812]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:22.697 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:22 smithi104 systemd[1]: Starting Ceph mds.e for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:22.948 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:22 smithi104 podman[64524]: 2024-05-27 10:13:22.605889429 +0000 UTC m=+0.042045975 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:22.948 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:22 smithi104 podman[64524]: 2024-05-27 10:13:22.749939729 +0000 UTC m=+0.186096286 container create 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T10:13:23.102 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:23.449 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:23 smithi104 podman[64524]: 2024-05-27 10:13:23.037068611 +0000 UTC m=+0.473225221 container init 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:13:23.449 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:23 smithi104 podman[64524]: 2024-05-27 10:13:23.047046476 +0000 UTC m=+0.483203026 container start 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T10:13:23.449 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mds[64543]: -- 172.21.15.104:0/1995963314 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556faa443a40 con 0x556faa48b180 2024-05-27T10:13:23.449 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:23 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e[64539]: starting mds.e at 2024-05-27T10:13:23.449 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:23 smithi104 bash[64524]: 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e 2024-05-27T10:13:23.449 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:13:23 smithi104 systemd[1]: Started Ceph mds.e for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:23.580 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:23.734 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:23 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:23 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:23.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:23.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:23.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:23.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:23 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:24.680 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:24 smithi143 ceph-mon[35545]: Deploying daemon mds.f on smithi143 2024-05-27T10:13:24.680 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:24 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] up:boot 2024-05-27T10:13:24.680 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:24 smithi143 ceph-mon[35545]: fsmap 5 up:standby 2024-05-27T10:13:24.680 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:24 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T10:13:24.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:24 smithi042 ceph-mon[31661]: Deploying daemon mds.f on smithi143 2024-05-27T10:13:24.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:24 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] up:boot 2024-05-27T10:13:24.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:24 smithi042 ceph-mon[31661]: fsmap 5 up:standby 2024-05-27T10:13:24.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:24 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T10:13:24.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:24 smithi104 ceph-mon[34812]: Deploying daemon mds.f on smithi143 2024-05-27T10:13:24.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:24 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] up:boot 2024-05-27T10:13:24.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:24 smithi104 ceph-mon[34812]: fsmap 5 up:standby 2024-05-27T10:13:24.951 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:24 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T10:13:25.675 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:25 smithi042 ceph-mon[31661]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:25.707 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:25 smithi143 ceph-mon[35545]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:25.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:25 smithi104 ceph-mon[34812]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:26.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 -- 172.21.15.42:0/3202229658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00115040 msgr2=0x7faa00117460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:26.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 --2- 172.21.15.42:0/3202229658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00115040 0x7faa00117460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b0a0 tx=0x7fa9f800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 -- 172.21.15.42:0/3202229658 shutdown_connections 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 --2- 172.21.15.42:0/3202229658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00115040 0x7faa00117460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 --2- 172.21.15.42:0/3202229658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa00077ce0 0x7faa000780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 --2- 172.21.15.42:0/3202229658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00078630 0x7faa00114b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 -- 172.21.15.42:0/3202229658 >> 172.21.15.42:0/3202229658 conn(0x7faa00102020 msgr2=0x7faa00104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 -- 172.21.15.42:0/3202229658 shutdown_connections 2024-05-27T10:13:26.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.082+0000 7faa06800700 1 -- 172.21.15.42:0/3202229658 wait complete. 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.083+0000 7faa06800700 1 Processor -- start 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 -- start start 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa00077ce0 0x7faa001a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 0x7faa001a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00115040 0x7faa001a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa0011a790 con 0x7faa00078630 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa0011a610 con 0x7faa00077ce0 2024-05-27T10:13:26.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.084+0000 7faa06800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faa0011a910 con 0x7faa00115040 2024-05-27T10:13:26.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 0x7faa001a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:26.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 0x7faa001a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55468/0 (socket says 172.21.15.42:55468) 2024-05-27T10:13:26.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 -- 172.21.15.42:0/1926521660 learned_addr learned my addr 172.21.15.42:0/1926521660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:26.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa05e00700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00115040 0x7faa001a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:26.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa05400700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa00077ce0 0x7faa001a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 -- 172.21.15.42:0/1926521660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00115040 msgr2=0x7faa001a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00115040 0x7faa001a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 -- 172.21.15.42:0/1926521660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa00077ce0 msgr2=0x7faa001a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa00077ce0 0x7faa001a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa04a00700 1 -- 172.21.15.42:0/1926521660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001a5fb0 con 0x7faa00078630 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.085+0000 7faa05e00700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00115040 0x7faa001a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:13:26.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.086+0000 7faa04a00700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 0x7faa001a1680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d980 tx=0x7fa9ec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:26.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.086+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec017070 con 0x7faa00078630 2024-05-27T10:13:26.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.086+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001a62a0 con 0x7faa00078630 2024-05-27T10:13:26.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.086+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9ec010460 con 0x7faa00078630 2024-05-27T10:13:26.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.086+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00de60 con 0x7faa00078630 2024-05-27T10:13:26.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.086+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001adbc0 con 0x7faa00078630 2024-05-27T10:13:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.087+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa000512b0 con 0x7faa00078630 2024-05-27T10:13:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.090+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa9ec010610 con 0x7faa00078630 2024-05-27T10:13:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.092+0000 7fa9fe000700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa9e80b2880 0x7fa9e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.092+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa9ec0e18a0 con 0x7faa00078630 2024-05-27T10:13:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.092+0000 7faa05400700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa9e80b2880 0x7fa9e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.093+0000 7faa05400700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa9e80b2880 0x7fa9e80b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005950 tx=0x7fa9f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.094+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9ec0a99c0 con 0x7faa00078630 2024-05-27T10:13:26.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.385+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa001a2440 con 0x7faa00078630 2024-05-27T10:13:26.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.386+0000 7fa9fe000700 1 -- 172.21.15.42:0/1926521660 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fa9ec0564a0 con 0x7faa00078630 2024-05-27T10:13:26.386 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:26.386 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T10:13:26.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa9e80b2880 msgr2=0x7fa9e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:26.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa9e80b2880 0x7fa9e80b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005950 tx=0x7fa9f40058e0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 msgr2=0x7faa001a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 0x7faa001a1680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d980 tx=0x7fa9ec00dd40 comp rx=0 tx=0).stop 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 shutdown_connections 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa00115040 0x7faa001a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa9e80b2880 0x7fa9e80b4d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.389+0000 7faa06800700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa00077ce0 0x7faa001a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.390+0000 7faa06800700 1 --2- 172.21.15.42:0/1926521660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa00078630 0x7faa001a1680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.390+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 >> 172.21.15.42:0/1926521660 conn(0x7faa00102020 msgr2=0x7faa0007ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:26.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.390+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 shutdown_connections 2024-05-27T10:13:26.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:26.390+0000 7faa06800700 1 -- 172.21.15.42:0/1926521660 wait complete. 2024-05-27T10:13:26.392 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 6 2024-05-27T10:13:26.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:26 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1926521660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:26.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:26 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1926521660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:26 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1926521660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:27.180 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:26 smithi143 podman[66026]: 2024-05-27 10:13:26.926981621 +0000 UTC m=+0.147784310 container create ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:13:27.181 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:27 smithi143 podman[66026]: 2024-05-27 10:13:27.179831994 +0000 UTC m=+0.400634665 container init ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T10:13:27.512 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:27 smithi143 podman[66026]: 2024-05-27 10:13:27.188801549 +0000 UTC m=+0.409604224 container start ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T10:13:27.512 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:27 smithi143 bash[66026]: ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da 2024-05-27T10:13:27.512 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:27 smithi143 systemd[1]: Started Ceph mds.f for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:27.512 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mds[66052]: -- 172.21.15.143:0/1251409156 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2f0513a40 con 0x55a2f055ba80 2024-05-27T10:13:27.513 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:13:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f[66046]: starting mds.f at 2024-05-27T10:13:27.773 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:27.774 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.774 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.774 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.774 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:27.774 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:27.774 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:27 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:27.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:27.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:27.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:27.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:27 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:27 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:28 smithi143 ceph-mon[35545]: Deploying daemon mds.c on smithi143 2024-05-27T10:13:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:28 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.143:6834/3892720784,v1:172.21.15.143:6835/3892720784] up:boot 2024-05-27T10:13:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:28 smithi143 ceph-mon[35545]: fsmap 6 up:standby 2024-05-27T10:13:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:28 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T10:13:28.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:28 smithi042 ceph-mon[31661]: Deploying daemon mds.c on smithi143 2024-05-27T10:13:28.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:28 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.143:6834/3892720784,v1:172.21.15.143:6835/3892720784] up:boot 2024-05-27T10:13:28.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:28 smithi042 ceph-mon[31661]: fsmap 6 up:standby 2024-05-27T10:13:28.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:28 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T10:13:28.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:28 smithi104 ceph-mon[34812]: Deploying daemon mds.c on smithi143 2024-05-27T10:13:28.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:28 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.143:6834/3892720784,v1:172.21.15.143:6835/3892720784] up:boot 2024-05-27T10:13:28.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:28 smithi104 ceph-mon[34812]: fsmap 6 up:standby 2024-05-27T10:13:28.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:28 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T10:13:29.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:29.671 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:29 smithi143 ceph-mon[35545]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:29.827 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:29 smithi042 ceph-mon[31661]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:29.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:29 smithi104 ceph-mon[34812]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:30.931 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:30 smithi143 systemd[1]: Starting Ceph mds.c for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:31.255 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:30 smithi143 podman[66335]: 2024-05-27 10:13:30.835047671 +0000 UTC m=+0.032298053 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:31.255 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:30 smithi143 podman[66335]: 2024-05-27 10:13:30.969689107 +0000 UTC m=+0.166939526 container create 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:13:31.255 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:31 smithi143 podman[66335]: 2024-05-27 10:13:31.253690181 +0000 UTC m=+0.450940647 container init 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T10:13:31.558 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:31 smithi143 podman[66335]: 2024-05-27 10:13:31.263653185 +0000 UTC m=+0.460903638 container start 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-05-27T10:13:31.558 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:31 smithi143 bash[66335]: 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b 2024-05-27T10:13:31.558 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:31 smithi143 systemd[1]: Started Ceph mds.c for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:31.558 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-mds[66354]: -- 172.21.15.143:0/1485479413 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593934ada40 con 0x5593934f5a80 2024-05-27T10:13:31.558 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c[66350]: starting mds.c at 2024-05-27T10:13:31.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:31 smithi042 ceph-mon[31661]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:31.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:31.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:31.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:31.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:31 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:31 smithi104 ceph-mon[34812]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:31 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-mon[35545]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:32.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:32.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:32.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:31 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 -- 172.21.15.42:0/313735834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a6810e8e0 msgr2=0x7f1a68110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 --2- 172.21.15.42:0/313735834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a6810e8e0 0x7f1a68110dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000b0a0 tx=0x7f1a6000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 -- 172.21.15.42:0/313735834 shutdown_connections 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 --2- 172.21.15.42:0/313735834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a6810e8e0 0x7f1a68110dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 --2- 172.21.15.42:0/313735834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a681054b0 0x7f1a681058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 --2- 172.21.15.42:0/313735834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a68105e00 0x7f1a6810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.154+0000 7f1a6de00700 1 -- 172.21.15.42:0/313735834 >> 172.21.15.42:0/313735834 conn(0x7f1a680fedb0 msgr2=0x7f1a681011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.155+0000 7f1a6de00700 1 -- 172.21.15.42:0/313735834 shutdown_connections 2024-05-27T10:13:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.155+0000 7f1a6de00700 1 -- 172.21.15.42:0/313735834 wait complete. 2024-05-27T10:13:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.156+0000 7f1a6de00700 1 Processor -- start 2024-05-27T10:13:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 -- start start 2024-05-27T10:13:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 0x7f1a681a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a68105e00 0x7f1a681a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a6810e8e0 0x7f1a681ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68114340 con 0x7f1a68105e00 2024-05-27T10:13:32.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681141c0 con 0x7f1a681054b0 2024-05-27T10:13:32.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.157+0000 7f1a6de00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681144c0 con 0x7f1a6810e8e0 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.158+0000 7f1a5e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a68105e00 0x7f1a681a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.158+0000 7f1a66a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 0x7f1a681a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.158+0000 7f1a66a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 0x7f1a681a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45650/0 (socket says 172.21.15.42:45650) 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.158+0000 7f1a66a00700 1 -- 172.21.15.42:0/907283535 learned_addr learned my addr 172.21.15.42:0/907283535 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.158+0000 7f1a67400700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a6810e8e0 0x7f1a681ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a66a00700 1 -- 172.21.15.42:0/907283535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a6810e8e0 msgr2=0x7f1a681ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a66a00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a6810e8e0 0x7f1a681ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a66a00700 1 -- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a68105e00 msgr2=0x7f1a681a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:32.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a66a00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a68105e00 0x7f1a681a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a66a00700 1 -- 172.21.15.42:0/907283535 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a681ad5e0 con 0x7f1a681054b0 2024-05-27T10:13:32.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a66a00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 0x7f1a681a2020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800d950 tx=0x7f1a5800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:32.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.159+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58016070 con 0x7f1a681054b0 2024-05-27T10:13:32.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.160+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a681ad7b0 con 0x7f1a681054b0 2024-05-27T10:13:32.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.160+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a5800fd20 con 0x7f1a681054b0 2024-05-27T10:13:32.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.160+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800e3b0 con 0x7f1a681054b0 2024-05-27T10:13:32.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.160+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a681adc60 con 0x7f1a681054b0 2024-05-27T10:13:32.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.161+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a680512b0 con 0x7f1a681054b0 2024-05-27T10:13:32.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.164+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1a58019070 con 0x7f1a681054b0 2024-05-27T10:13:32.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.167+0000 7f1a6ca00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1a4c0b2880 0x7f1a4c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:32.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.167+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1a580e18f0 con 0x7f1a681054b0 2024-05-27T10:13:32.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.167+0000 7f1a5e000700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1a4c0b2880 0x7f1a4c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:32.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.168+0000 7f1a5e000700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1a4c0b2880 0x7f1a4c0b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a68104c20 tx=0x7f1a50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:32.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.179+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a580a9a10 con 0x7f1a681054b0 2024-05-27T10:13:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.474+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a681adec0 con 0x7f1a681054b0 2024-05-27T10:13:32.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.475+0000 7f1a6ca00700 1 -- 172.21.15.42:0/907283535 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f1a580a9310 con 0x7f1a681054b0 2024-05-27T10:13:32.476 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:32.476 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T10:13:32.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.479+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1a4c0b2880 msgr2=0x7f1a4c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.479+0000 7f1a6de00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1a4c0b2880 0x7f1a4c0b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a68104c20 tx=0x7f1a50009260 comp rx=0 tx=0).stop 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.479+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 msgr2=0x7f1a681a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.479+0000 7f1a6de00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 0x7f1a681a2020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800d950 tx=0x7f1a5800dc60 comp rx=0 tx=0).stop 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 shutdown_connections 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a6810e8e0 0x7f1a681ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1a4c0b2880 0x7f1a4c0b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a681054b0 0x7f1a681a2020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 --2- 172.21.15.42:0/907283535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a68105e00 0x7f1a681a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 >> 172.21.15.42:0/907283535 conn(0x7f1a680fedb0 msgr2=0x7f1a68103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 shutdown_connections 2024-05-27T10:13:32.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:32.480+0000 7f1a6de00700 1 -- 172.21.15.42:0/907283535 wait complete. 2024-05-27T10:13:32.481 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: Deploying daemon mds.h on smithi104 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.143:6836/4166809554,v1:172.21.15.143:6837/4166809554] up:boot 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: fsmap 7 up:standby 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T10:13:32.846 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:32 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/907283535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: Deploying daemon mds.h on smithi104 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.143:6836/4166809554,v1:172.21.15.143:6837/4166809554] up:boot 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: fsmap 7 up:standby 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T10:13:32.907 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:32 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/907283535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:33.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:33.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:33.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: Deploying daemon mds.h on smithi104 2024-05-27T10:13:33.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.143:6836/4166809554,v1:172.21.15.143:6837/4166809554] up:boot 2024-05-27T10:13:33.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: fsmap 7 up:standby 2024-05-27T10:13:33.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T10:13:33.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:32 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/907283535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:33.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:33 smithi104 ceph-mon[34812]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:33.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:33 smithi042 ceph-mon[31661]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:34.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:33 smithi143 ceph-mon[35545]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:34.949 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:34 smithi104 systemd[1]: Starting Ceph mds.h for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:35.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:34 smithi104 podman[64823]: 2024-05-27 10:13:34.887088047 +0000 UTC m=+0.034202508 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:35.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 podman[64823]: 2024-05-27 10:13:35.004697131 +0000 UTC m=+0.151811628 container create 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T10:13:35.325 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 podman[64823]: 2024-05-27 10:13:35.324310792 +0000 UTC m=+0.471425295 container init 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T10:13:35.346 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:35.579 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 podman[64823]: 2024-05-27 10:13:35.335039784 +0000 UTC m=+0.482154291 container start 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:13:35.579 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 bash[64823]: 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d 2024-05-27T10:13:35.579 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 systemd[1]: Started Ceph mds.h for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:35.579 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 ceph-mds[64842]: -- 172.21.15.104:0/3745121320 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dce96ba40 con 0x558dce9b3600 2024-05-27T10:13:35.580 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:13:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: starting mds.h at 2024-05-27T10:13:35.817 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:35 smithi042 ceph-mon[31661]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:35 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:35 smithi104 ceph-mon[34812]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:35 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:36.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:35 smithi143 ceph-mon[35545]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:36.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:36.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:35 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:36.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:boot 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: fsmap 8 up:standby 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:36.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:36 smithi042 ceph-mon[31661]: Deploying daemon mds.i on smithi143 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:boot 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: fsmap 8 up:standby 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:36.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:36.863 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:36 smithi143 ceph-mon[35545]: Deploying daemon mds.i on smithi143 2024-05-27T10:13:36.905 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:boot 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: fsmap 8 up:standby 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:36.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:36 smithi104 ceph-mon[34812]: Deploying daemon mds.i on smithi143 2024-05-27T10:13:37.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:37 smithi143 ceph-mon[35545]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:37.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:37 smithi042 ceph-mon[31661]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:37 smithi104 ceph-mon[34812]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:38.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 -- 172.21.15.42:0/1780042733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc1045e0 msgr2=0x7f87dc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:38.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 --2- 172.21.15.42:0/1780042733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc1045e0 0x7f87dc1049f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0099c0 tx=0x7f87cc009cd0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 -- 172.21.15.42:0/1780042733 shutdown_connections 2024-05-27T10:13:38.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 --2- 172.21.15.42:0/1780042733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc109860 0x7f87dc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 --2- 172.21.15.42:0/1780042733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc1045e0 0x7f87dc1049f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 --2- 172.21.15.42:0/1780042733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87dc108f10 0x7f87dc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 -- 172.21.15.42:0/1780042733 >> 172.21.15.42:0/1780042733 conn(0x7f87dc100030 msgr2=0x7f87dc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:38.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 -- 172.21.15.42:0/1780042733 shutdown_connections 2024-05-27T10:13:38.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.209+0000 7f87e3200700 1 -- 172.21.15.42:0/1780042733 wait complete. 2024-05-27T10:13:38.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.210+0000 7f87e3200700 1 Processor -- start 2024-05-27T10:13:38.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 -- start start 2024-05-27T10:13:38.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87dc1045e0 0x7f87dc19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:38.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc108f10 0x7f87dc19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 0x7f87dc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc114390 con 0x7f87dc109860 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc114210 con 0x7f87dc108f10 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.211+0000 7f87e3200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc114510 con 0x7f87dc1045e0 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 0x7f87dc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 0x7f87dc1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59066/0 (socket says 172.21.15.42:59066) 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 -- 172.21.15.42:0/1122224580 learned_addr learned my addr 172.21.15.42:0/1122224580 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87dbe00700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87dc1045e0 0x7f87dc19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87db400700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc108f10 0x7f87dc19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:38.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 -- 172.21.15.42:0/1122224580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87dc1045e0 msgr2=0x7f87dc19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:38.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87dc1045e0 0x7f87dc19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 -- 172.21.15.42:0/1122224580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc108f10 msgr2=0x7f87dc19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:38.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc108f10 0x7f87dc19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 -- 172.21.15.42:0/1122224580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc1a1ef0 con 0x7f87dc109860 2024-05-27T10:13:38.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87db400700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc108f10 0x7f87dc19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.212+0000 7f87e1200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 0x7f87dc1a1850 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007ee0 tx=0x7f87d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.213+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d400dbd0 con 0x7f87dc109860 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.213+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87d400dd70 con 0x7f87dc109860 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.213+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc1a21e0 con 0x7f87dc109860 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.213+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1a9b00 con 0x7f87dc109860 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.213+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d40058d0 con 0x7f87dc109860 2024-05-27T10:13:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.214+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc0512b0 con 0x7f87dc109860 2024-05-27T10:13:38.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.217+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f87d401a020 con 0x7f87dc109860 2024-05-27T10:13:38.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.218+0000 7f87d8c00700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f87c00b2880 0x7f87c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:38.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.218+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f87d40e1560 con 0x7f87dc109860 2024-05-27T10:13:38.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.219+0000 7f87dbe00700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f87c00b2880 0x7f87c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:38.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.219+0000 7f87dbe00700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f87c00b2880 0x7f87c00b4d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f87cc000c00 tx=0x7f87cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:38.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.223+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87d4010070 con 0x7f87dc109860 2024-05-27T10:13:38.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.512+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87dc1a9cd0 con 0x7f87dc109860 2024-05-27T10:13:38.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.514+0000 7f87d8c00700 1 -- 172.21.15.42:0/1122224580 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f87d40a9680 con 0x7f87dc109860 2024-05-27T10:13:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:38.514 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34466,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f87c00b2880 msgr2=0x7f87c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f87c00b2880 0x7f87c00b4d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f87cc000c00 tx=0x7f87cc005fb0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 msgr2=0x7f87dc1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 0x7f87dc1a1850 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007ee0 tx=0x7f87d400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 shutdown_connections 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87dc109860 0x7f87dc1a1850 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87dc1045e0 0x7f87dc19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f87c00b2880 0x7f87c00b4d90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 --2- 172.21.15.42:0/1122224580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87dc108f10 0x7f87dc19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:38.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.517+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 >> 172.21.15.42:0/1122224580 conn(0x7f87dc100030 msgr2=0x7f87dc10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:38.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.518+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 shutdown_connections 2024-05-27T10:13:38.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:38.518+0000 7f87e3200700 1 -- 172.21.15.42:0/1122224580 wait complete. 2024-05-27T10:13:38.519 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2024-05-27T10:13:38.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:38 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1122224580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:38.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:38 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1122224580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:38 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1122224580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:39.058 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:38 smithi143 systemd[1]: Starting Ceph mds.i for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:39.393 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 podman[66642]: 2024-05-27 10:13:39.016866776 +0000 UTC m=+0.035659382 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:39.393 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 podman[66642]: 2024-05-27 10:13:39.13004416 +0000 UTC m=+0.148836812 container create b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T10:13:39.393 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 podman[66642]: 2024-05-27 10:13:39.392196404 +0000 UTC m=+0.410989099 container init b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-27T10:13:39.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:39 smithi143 ceph-mon[35545]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:39 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 podman[66642]: 2024-05-27 10:13:39.402618278 +0000 UTC m=+0.421410882 container start b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 bash[66642]: b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 systemd[1]: Started Ceph mds.i for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 ceph-mds[66661]: -- 172.21.15.143:0/3595474101 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565014295a40 con 0x5650142dd180 2024-05-27T10:13:39.808 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:13:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i[66657]: starting mds.i at 2024-05-27T10:13:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:39 smithi104 ceph-mon[34812]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:39 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:40.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:39 smithi042 ceph-mon[31661]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:40.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:40.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:39 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.143:6838/517039305,v1:172.21.15.143:6839/517039305] up:boot 2024-05-27T10:13:40.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: fsmap 9 up:standby 2024-05-27T10:13:40.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T10:13:40.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:40.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:40.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:40.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:40.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:40 smithi104 ceph-mon[34812]: Deploying daemon mds.g on smithi042 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.143:6838/517039305,v1:172.21.15.143:6839/517039305] up:boot 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: fsmap 9 up:standby 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:40.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:40 smithi042 ceph-mon[31661]: Deploying daemon mds.g on smithi042 2024-05-27T10:13:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.143:6838/517039305,v1:172.21.15.143:6839/517039305] up:boot 2024-05-27T10:13:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: fsmap 9 up:standby 2024-05-27T10:13:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T10:13:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:41.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:40 smithi143 ceph-mon[35545]: Deploying daemon mds.g on smithi042 2024-05-27T10:13:41.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:41 smithi104 ceph-mon[34812]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:42.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:41 smithi143 ceph-mon[35545]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:42.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:41 smithi042 ceph-mon[31661]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:42.226 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:43.095 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:42 smithi042 systemd[1]: Starting Ceph mds.g for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:43.944 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:43 smithi042 ceph-mon[31661]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:43.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:43 smithi104 ceph-mon[34812]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:43 smithi143 ceph-mon[35545]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:44.208 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:43 smithi042 podman[74366]: 2024-05-27 10:13:43.848914219 +0000 UTC m=+0.034757325 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:44.208 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 podman[74366]: 2024-05-27 10:13:44.0066099 +0000 UTC m=+0.192453043 container create 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T10:13:44.907 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 podman[74366]: 2024-05-27 10:13:44.655419872 +0000 UTC m=+0.841263039 container init 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:13:44.908 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 podman[74366]: 2024-05-27 10:13:44.669251459 +0000 UTC m=+0.855094636 container start 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T10:13:44.908 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 ceph-mds[74401]: -- 172.21.15.42:0/2055113262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b813ab5a40 con 0x55b813afda80 2024-05-27T10:13:44.908 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g[74392]: starting mds.g at 2024-05-27T10:13:44.908 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 bash[74366]: 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 2024-05-27T10:13:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 -- 172.21.15.42:0/2594712125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d4108750 msgr2=0x7f60d41100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 --2- 172.21.15.42:0/2594712125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d4108750 0x7f60d41100b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00b0d0 tx=0x7f60cc00b3e0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 -- 172.21.15.42:0/2594712125 shutdown_connections 2024-05-27T10:13:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 --2- 172.21.15.42:0/2594712125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d4108750 0x7f60d41100b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 --2- 172.21.15.42:0/2594712125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d41034d0 0x7f60d41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 --2- 172.21.15.42:0/2594712125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4107e00 0x7f60d4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 -- 172.21.15.42:0/2594712125 >> 172.21.15.42:0/2594712125 conn(0x7f60d40fef20 msgr2=0x7f60d4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 -- 172.21.15.42:0/2594712125 shutdown_connections 2024-05-27T10:13:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.048+0000 7f60db800700 1 -- 172.21.15.42:0/2594712125 wait complete. 2024-05-27T10:13:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.050+0000 7f60db800700 1 Processor -- start 2024-05-27T10:13:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.050+0000 7f60db800700 1 -- start start 2024-05-27T10:13:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.050+0000 7f60db800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 0x7f60d419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.050+0000 7f60db800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d4107e00 0x7f60d419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60db800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4108750 0x7f60d41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60db800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41143a0 con 0x7f60d41034d0 2024-05-27T10:13:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60db800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4114220 con 0x7f60d4107e00 2024-05-27T10:13:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60db800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4114520 con 0x7f60d4108750 2024-05-27T10:13:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 0x7f60d419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 0x7f60d419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52224/0 (socket says 172.21.15.42:52224) 2024-05-27T10:13:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 -- 172.21.15.42:0/2034028580 learned_addr learned my addr 172.21.15.42:0/2034028580 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d3e00700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d4107e00 0x7f60d419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d9800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4108750 0x7f60d41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 -- 172.21.15.42:0/2034028580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4108750 msgr2=0x7f60d41a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4108750 0x7f60d41a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 -- 172.21.15.42:0/2034028580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d4107e00 msgr2=0x7f60d419d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d4107e00 0x7f60d419d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.051+0000 7f60d8e00700 1 -- 172.21.15.42:0/2034028580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d41a1e20 con 0x7f60d41034d0 2024-05-27T10:13:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.052+0000 7f60d8e00700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 0x7f60d419cf50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60c800d950 tx=0x7f60c800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.052+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8016070 con 0x7f60d41034d0 2024-05-27T10:13:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.052+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d41a2110 con 0x7f60d41034d0 2024-05-27T10:13:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.052+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60c8010460 con 0x7f60d41034d0 2024-05-27T10:13:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.052+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c800f620 con 0x7f60d41034d0 2024-05-27T10:13:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.052+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d41a9a30 con 0x7f60d41034d0 2024-05-27T10:13:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.053+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d40512b0 con 0x7f60d41034d0 2024-05-27T10:13:45.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.056+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f60c8019070 con 0x7f60d41034d0 2024-05-27T10:13:45.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.058+0000 7f60d1600700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f60bc0b2900 0x7f60bc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:45.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.058+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f60c80e1d50 con 0x7f60d41034d0 2024-05-27T10:13:45.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.059+0000 7f60d3e00700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f60bc0b2900 0x7f60bc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:45.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.059+0000 7f60d3e00700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f60bc0b2900 0x7f60bc0b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005950 tx=0x7f60c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.064+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60c80a9e70 con 0x7f60d41034d0 2024-05-27T10:13:45.343 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:13:44 smithi042 systemd[1]: Started Ceph mds.g for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:45.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.357+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d419e2b0 con 0x7f60d41034d0 2024-05-27T10:13:45.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.358+0000 7f60d1600700 1 -- 172.21.15.42:0/2034028580 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f60c80a9770 con 0x7f60d41034d0 2024-05-27T10:13:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:45.359 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34466,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f60bc0b2900 msgr2=0x7f60bc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f60bc0b2900 0x7f60bc0b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005950 tx=0x7f60c00058e0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 msgr2=0x7f60d419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 0x7f60d419cf50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60c800d950 tx=0x7f60c800dd10 comp rx=0 tx=0).stop 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 shutdown_connections 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4108750 0x7f60d41a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60d41034d0 0x7f60d419cf50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f60bc0b2900 0x7f60bc0b4e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 --2- 172.21.15.42:0/2034028580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d4107e00 0x7f60d419d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:45.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 >> 172.21.15.42:0/2034028580 conn(0x7f60d40fef20 msgr2=0x7f60d41097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:45.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 shutdown_connections 2024-05-27T10:13:45.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:45.361+0000 7f60db800700 1 -- 172.21.15.42:0/2034028580 wait complete. 2024-05-27T10:13:45.363 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2024-05-27T10:13:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:45.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:45.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:45 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2034028580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:46.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:46.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:46.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:46.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:46.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:46.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2034028580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:46.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:45 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2034028580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:46.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:46 smithi042 ceph-mon[31661]: Deploying daemon mds.a on smithi042 2024-05-27T10:13:46.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:46 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.42:6838/4181502432,v1:172.21.15.42:6839/4181502432] up:boot 2024-05-27T10:13:46.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:46 smithi042 ceph-mon[31661]: fsmap 10 up:standby 2024-05-27T10:13:46.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:46 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T10:13:47.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:46 smithi143 ceph-mon[35545]: Deploying daemon mds.a on smithi042 2024-05-27T10:13:47.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:46 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.42:6838/4181502432,v1:172.21.15.42:6839/4181502432] up:boot 2024-05-27T10:13:47.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:46 smithi143 ceph-mon[35545]: fsmap 10 up:standby 2024-05-27T10:13:47.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:46 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T10:13:47.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:46 smithi104 ceph-mon[34812]: Deploying daemon mds.a on smithi042 2024-05-27T10:13:47.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:46 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.42:6838/4181502432,v1:172.21.15.42:6839/4181502432] up:boot 2024-05-27T10:13:47.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:46 smithi104 ceph-mon[34812]: fsmap 10 up:standby 2024-05-27T10:13:47.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:46 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T10:13:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:47 smithi042 ceph-mon[31661]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:47 smithi104 ceph-mon[34812]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:47 smithi143 ceph-mon[35545]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:48.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:48.785 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:50.017 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:49 smithi042 ceph-mon[31661]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:50.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:49 smithi143 ceph-mon[35545]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:50.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:49 smithi104 ceph-mon[34812]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:50.326 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 podman[74805]: 2024-05-27 10:13:50.145964204 +0000 UTC m=+0.698198841 container create 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T10:13:50.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:50 smithi042 ceph-mon[31661]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:50.935 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 podman[74805]: 2024-05-27 10:13:50.635573819 +0000 UTC m=+1.187808463 container init 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-27T10:13:50.936 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 podman[74805]: 2024-05-27 10:13:50.647756357 +0000 UTC m=+1.199991000 container start 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T10:13:50.936 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 ceph-mds[74839]: -- 172.21.15.42:0/170140555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5583825924e0 con 0x5583825dda80 2024-05-27T10:13:50.936 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 ceph-mds[74839]: -- 172.21.15.42:0/170140555 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558382595a40 con 0x5583825dda80 2024-05-27T10:13:50.936 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a[74835]: starting mds.a at 2024-05-27T10:13:51.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:50 smithi143 ceph-mon[35545]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:50 smithi104 ceph-mon[34812]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:51.314 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 bash[74805]: 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b 2024-05-27T10:13:51.314 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:13:50 smithi042 systemd[1]: Started Ceph mds.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 -- 172.21.15.42:0/2964762882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc088108f10 msgr2=0x7fc088109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 --2- 172.21.15.42:0/2964762882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc088108f10 0x7fc088109320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0700099c0 tx=0x7fc070009cd0 comp rx=0 tx=0).stop 2024-05-27T10:13:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 -- 172.21.15.42:0/2964762882 shutdown_connections 2024-05-27T10:13:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 --2- 172.21.15.42:0/2964762882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc088109860 0x7fc0881111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 --2- 172.21.15.42:0/2964762882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0881045e0 0x7fc0881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 --2- 172.21.15.42:0/2964762882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc088108f10 0x7fc088109320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 -- 172.21.15.42:0/2964762882 >> 172.21.15.42:0/2964762882 conn(0x7fc088100030 msgr2=0x7fc088102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 -- 172.21.15.42:0/2964762882 shutdown_connections 2024-05-27T10:13:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.763+0000 7fc08f600700 1 -- 172.21.15.42:0/2964762882 wait complete. 2024-05-27T10:13:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.764+0000 7fc08f600700 1 Processor -- start 2024-05-27T10:13:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.764+0000 7fc08f600700 1 -- start start 2024-05-27T10:13:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08f600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 0x7fc08819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08f600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc088108f10 0x7fc08819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08f600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc088109860 0x7fc08819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08f600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc088114370 con 0x7fc0881045e0 2024-05-27T10:13:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08f600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881141f0 con 0x7fc088108f10 2024-05-27T10:13:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08f600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881144f0 con 0x7fc088109860 2024-05-27T10:13:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 0x7fc08819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 0x7fc08819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52256/0 (socket says 172.21.15.42:52256) 2024-05-27T10:13:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08cc00700 1 -- 172.21.15.42:0/2902121105 learned_addr learned my addr 172.21.15.42:0/2902121105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc07fe00700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc088108f10 0x7fc08819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.765+0000 7fc08d600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc088109860 0x7fc08819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08cc00700 1 -- 172.21.15.42:0/2902121105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc088109860 msgr2=0x7fc08819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08cc00700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc088109860 0x7fc08819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08cc00700 1 -- 172.21.15.42:0/2902121105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc088108f10 msgr2=0x7fc08819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08cc00700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc088108f10 0x7fc08819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08cc00700 1 -- 172.21.15.42:0/2902121105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08819fc00 con 0x7fc0881045e0 2024-05-27T10:13:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08d600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc088109860 0x7fc08819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:13:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08cc00700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 0x7fc08819ade0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d950 tx=0x7fc07800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078016070 con 0x7fc0881045e0 2024-05-27T10:13:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc078009dd0 con 0x7fc0881045e0 2024-05-27T10:13:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800f3c0 con 0x7fc0881045e0 2024-05-27T10:13:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.766+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08819fef0 con 0x7fc0881045e0 2024-05-27T10:13:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.767+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881a03b0 con 0x7fc0881045e0 2024-05-27T10:13:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.768+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0880512b0 con 0x7fc0881045e0 2024-05-27T10:13:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.770+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc078019070 con 0x7fc0881045e0 2024-05-27T10:13:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.772+0000 7fc086000700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc06c0b2900 0x7fc06c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.772+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc0780e19f0 con 0x7fc0881045e0 2024-05-27T10:13:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.772+0000 7fc07fe00700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc06c0b2900 0x7fc06c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.773+0000 7fc07fe00700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc06c0b2900 0x7fc06c0b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc070009990 tx=0x7fc070005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:51.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:51.777+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0780a9b10 con 0x7fc0881045e0 2024-05-27T10:13:52.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.064+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08802e5e0 con 0x7fc0881045e0 2024-05-27T10:13:52.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.065+0000 7fc086000700 1 -- 172.21.15.42:0/2902121105 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fc0780a9410 con 0x7fc0881045e0 2024-05-27T10:13:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34466,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T10:13:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc06c0b2900 msgr2=0x7fc06c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc06c0b2900 0x7fc06c0b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc070009990 tx=0x7fc070005fb0 comp rx=0 tx=0).stop 2024-05-27T10:13:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 msgr2=0x7fc08819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 0x7fc08819ade0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d950 tx=0x7fc07800dc60 comp rx=0 tx=0).stop 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 shutdown_connections 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc088109860 0x7fc08819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0881045e0 0x7fc08819ade0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc06c0b2900 0x7fc06c0b4e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 --2- 172.21.15.42:0/2902121105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc088108f10 0x7fc08819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.068+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 >> 172.21.15.42:0/2902121105 conn(0x7fc088100030 msgr2=0x7fc08810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.069+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 shutdown_connections 2024-05-27T10:13:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:52.069+0000 7fc08f600700 1 -- 172.21.15.42:0/2902121105 wait complete. 2024-05-27T10:13:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-05-27T10:13:52.307 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.307 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.307 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.307 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:52.307 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:52.307 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:52.308 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: Deploying daemon mds.b on smithi104 2024-05-27T10:13:52.308 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.42:6840/2889621451,v1:172.21.15.42:6841/2889621451] up:boot 2024-05-27T10:13:52.308 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: fsmap 11 up:standby 2024-05-27T10:13:52.308 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:52 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T10:13:52.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: Deploying daemon mds.b on smithi104 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.42:6840/2889621451,v1:172.21.15.42:6841/2889621451] up:boot 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: fsmap 11 up:standby 2024-05-27T10:13:52.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:52 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T10:13:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T10:13:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T10:13:52.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:13:52.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: Deploying daemon mds.b on smithi104 2024-05-27T10:13:52.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.42:6840/2889621451,v1:172.21.15.42:6841/2889621451] up:boot 2024-05-27T10:13:52.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: fsmap 11 up:standby 2024-05-27T10:13:52.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:52 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T10:13:53.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:53 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2902121105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:53.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:53 smithi143 ceph-mon[35545]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:53.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:53 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2902121105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:53.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:53 smithi104 ceph-mon[34812]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:53.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:53 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2902121105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:53.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:53 smithi042 ceph-mon[31661]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:54.448 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 systemd[1]: Starting Ceph mds.b for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:13:54.749 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 podman[65141]: 2024-05-27 10:13:54.383915921 +0000 UTC m=+0.031943314 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:13:54.749 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 podman[65141]: 2024-05-27 10:13:54.491950709 +0000 UTC m=+0.139978113 container create 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:13:54.749 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 podman[65141]: 2024-05-27 10:13:54.748368372 +0000 UTC m=+0.396395848 container init 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:13:55.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:13:55.198 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 podman[65141]: 2024-05-27 10:13:54.758307936 +0000 UTC m=+0.406335346 container start 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T10:13:55.198 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 bash[65141]: 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 2024-05-27T10:13:55.198 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 systemd[1]: Started Ceph mds.b for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:13:55.198 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 ceph-mds[65164]: -- 172.21.15.104:0/1222632499 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a166f1ba40 con 0x55a166f63a80 2024-05-27T10:13:55.198 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:13:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b[65158]: starting mds.b at 2024-05-27T10:13:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:13:55.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:55 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:13:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:55 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:13:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:55.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:55 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:13:56.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:56 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] up:boot 2024-05-27T10:13:56.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:56 smithi104 ceph-mon[34812]: fsmap 12 up:standby 2024-05-27T10:13:56.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T10:13:56.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:56.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:56 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:56.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:56 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] up:boot 2024-05-27T10:13:56.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:56 smithi143 ceph-mon[35545]: fsmap 12 up:standby 2024-05-27T10:13:56.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:56 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T10:13:56.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:56 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:56.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:56 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:56.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:56 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] up:boot 2024-05-27T10:13:56.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:56 smithi042 ceph-mon[31661]: fsmap 12 up:standby 2024-05-27T10:13:56.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:56 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T10:13:56.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:56 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:56.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:56 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:57.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:57 smithi042 ceph-mon[31661]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:57.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:57 smithi143 ceph-mon[35545]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:57.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 -- 172.21.15.42:0/2853025604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 msgr2=0x7fab58113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:57.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 --2- 172.21.15.42:0/2853025604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab58113350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b0a0 tx=0x7fab5000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 -- 172.21.15.42:0/2853025604 shutdown_connections 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 --2- 172.21.15.42:0/2853025604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab58113350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 --2- 172.21.15.42:0/2853025604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 0x7fab5810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 --2- 172.21.15.42:0/2853025604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 0x7fab58106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 -- 172.21.15.42:0/2853025604 >> 172.21.15.42:0/2853025604 conn(0x7fab58102070 msgr2=0x7fab581044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 -- 172.21.15.42:0/2853025604 shutdown_connections 2024-05-27T10:13:57.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.910+0000 7fab5d800700 1 -- 172.21.15.42:0/2853025604 wait complete. 2024-05-27T10:13:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.911+0000 7fab5d800700 1 Processor -- start 2024-05-27T10:13:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.911+0000 7fab5d800700 1 -- start start 2024-05-27T10:13:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab5d800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 0x7fab5819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab5d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab5819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab5d800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 0x7fab581a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab5d800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab58116590 con 0x7fab58106ec0 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab5d800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab58116410 con 0x7fab5810c5b0 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab5d800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab58116710 con 0x7fab58106570 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab4fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab5819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab56a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 0x7fab5819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab4fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab5819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59008/0 (socket says 172.21.15.42:59008) 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.912+0000 7fab4fe00700 1 -- 172.21.15.42:0/3111329880 learned_addr learned my addr 172.21.15.42:0/3111329880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:13:57.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab57400700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 0x7fab581a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab56a00700 1 -- 172.21.15.42:0/3111329880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 msgr2=0x7fab581a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab56a00700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 0x7fab581a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab56a00700 1 -- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 msgr2=0x7fab5819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab56a00700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab5819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab56a00700 1 -- 172.21.15.42:0/3111329880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab581a1db0 con 0x7fab58106570 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab57400700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 0x7fab581a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.913+0000 7fab56a00700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 0x7fab5819cee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab4800d720 tx=0x7fab4800da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:57.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.914+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48016070 con 0x7fab58106570 2024-05-27T10:13:57.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.914+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab581a20a0 con 0x7fab58106570 2024-05-27T10:13:57.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.914+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab48004830 con 0x7fab58106570 2024-05-27T10:13:57.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.914+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48005490 con 0x7fab58106570 2024-05-27T10:13:57.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.914+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a99c0 con 0x7fab58106570 2024-05-27T10:13:57.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.915+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab580512b0 con 0x7fab58106570 2024-05-27T10:13:57.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.918+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fab48004220 con 0x7fab58106570 2024-05-27T10:13:57.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.919+0000 7fab4f400700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fab3c0b2880 0x7fab3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:13:57.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.919+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fab480e20b0 con 0x7fab58106570 2024-05-27T10:13:57.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.919+0000 7fab4fe00700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fab3c0b2880 0x7fab3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:13:57.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.920+0000 7fab4fe00700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fab3c0b2880 0x7fab3c0b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab58103cd0 tx=0x7fab40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:13:57.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:57.929+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab4801f020 con 0x7fab58106570 2024-05-27T10:13:57.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:57 smithi104 ceph-mon[34812]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:58.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.229+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab5819e240 con 0x7fab58106570 2024-05-27T10:13:58.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.235+0000 7fab4f400700 1 -- 172.21.15.42:0/3111329880 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fab480aa1d0 con 0x7fab58106570 2024-05-27T10:13:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:13:58.236 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34421,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34466,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fab3c0b2880 msgr2=0x7fab3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fab3c0b2880 0x7fab3c0b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab58103cd0 tx=0x7fab40009260 comp rx=0 tx=0).stop 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 msgr2=0x7fab5819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 0x7fab5819cee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab4800d720 tx=0x7fab4800da30 comp rx=0 tx=0).stop 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 shutdown_connections 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fab3c0b2880 0x7fab3c0b4d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5810c5b0 0x7fab581a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab58106570 0x7fab5819cee0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 --2- 172.21.15.42:0/3111329880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab58106ec0 0x7fab5819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:13:58.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 >> 172.21.15.42:0/3111329880 conn(0x7fab58102070 msgr2=0x7fab58104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:13:58.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 shutdown_connections 2024-05-27T10:13:58.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:13:58.238+0000 7fab5d800700 1 -- 172.21.15.42:0/3111329880 wait complete. 2024-05-27T10:13:58.241 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-05-27T10:13:59.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:59 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:59 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3111329880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:59.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:13:59 smithi042 ceph-mon[31661]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:59.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:59 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:59 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3111329880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:59.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:13:59 smithi104 ceph-mon[34812]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:59.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:59 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:13:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:59 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3111329880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:13:59.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:13:59 smithi143 ceph-mon[35545]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:13:59.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-27T10:14:00.034 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:01 smithi042 ceph-mon[31661]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:01 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:01.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:01 smithi104 ceph-mon[34812]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:01.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:01.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:01 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:02.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:01 smithi143 ceph-mon[35545]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:02.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:02.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:01 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:02.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 -- 172.21.15.42:0/650095630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c076670 msgr2=0x7fe16c110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:02.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 --2- 172.21.15.42:0/650095630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c076670 0x7fe16c110460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe154009990 tx=0x7fe154009ca0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 -- 172.21.15.42:0/650095630 shutdown_connections 2024-05-27T10:14:02.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 --2- 172.21.15.42:0/650095630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe16c1109a0 0x7fe16c112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 --2- 172.21.15.42:0/650095630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c075d20 0x7fe16c076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 --2- 172.21.15.42:0/650095630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c076670 0x7fe16c110460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 -- 172.21.15.42:0/650095630 >> 172.21.15.42:0/650095630 conn(0x7fe16c079d00 msgr2=0x7fe16c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 -- 172.21.15.42:0/650095630 shutdown_connections 2024-05-27T10:14:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.542+0000 7fe170e00700 1 -- 172.21.15.42:0/650095630 wait complete. 2024-05-27T10:14:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.544+0000 7fe170e00700 1 Processor -- start 2024-05-27T10:14:02.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.544+0000 7fe170e00700 1 -- start start 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.544+0000 7fe170e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 0x7fe16c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe170e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe16c076670 0x7fe16c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe170e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c1109a0 0x7fe16c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe170e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c116350 con 0x7fe16c075d20 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe170e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1161d0 con 0x7fe16c076670 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe170e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1164d0 con 0x7fe16c1109a0 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 0x7fe16c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 0x7fe16c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59028/0 (socket says 172.21.15.42:59028) 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 -- 172.21.15.42:0/3774073552 learned_addr learned my addr 172.21.15.42:0/3774073552 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16be00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c1109a0 0x7fe16c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:02.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe163e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe16c076670 0x7fe16c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:02.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 -- 172.21.15.42:0/3774073552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c1109a0 msgr2=0x7fe16c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c1109a0 0x7fe16c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 -- 172.21.15.42:0/3774073552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe16c076670 msgr2=0x7fe16c19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.545+0000 7fe16b400700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe16c076670 0x7fe16c19d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe16b400700 1 -- 172.21.15.42:0/3774073552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c1a1cb0 con 0x7fe16c075d20 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe16b400700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 0x7fe16c19cde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00e9a0 tx=0x7fe15c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00cbd0 con 0x7fe16c075d20 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe15c00cd70 con 0x7fe16c075d20 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c1a1fa0 con 0x7fe16c075d20 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c018680 con 0x7fe16c075d20 2024-05-27T10:14:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.546+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c1a98c0 con 0x7fe16c075d20 2024-05-27T10:14:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.548+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16c077290 con 0x7fe16c075d20 2024-05-27T10:14:02.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.550+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe15c010430 con 0x7fe16c075d20 2024-05-27T10:14:02.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.553+0000 7fe168c00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1500b2900 0x7fe1500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:02.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.553+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fe15c0e2300 con 0x7fe16c075d20 2024-05-27T10:14:02.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.553+0000 7fe163e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1500b2900 0x7fe1500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.554+0000 7fe163e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1500b2900 0x7fe1500b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b2e0 tx=0x7fe154005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.555+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe15c0aa420 con 0x7fe16c075d20 2024-05-27T10:14:02.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.801+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe16c02e5e0 con 0x7fe16c075d20 2024-05-27T10:14:02.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:14:02.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:14:02.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:02 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.802+0000 7fe168c00700 1 -- 172.21.15.42:0/3774073552 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fe15c0a9d20 con 0x7fe16c075d20 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:14:02.803 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-27T10:14:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1500b2900 msgr2=0x7fe1500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1500b2900 0x7fe1500b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b2e0 tx=0x7fe154005fb0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 msgr2=0x7fe16c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 0x7fe16c19cde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00e9a0 tx=0x7fe15c00ecb0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 shutdown_connections 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c1109a0 0x7fe16c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe16c075d20 0x7fe16c19cde0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1500b2900 0x7fe1500b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 --2- 172.21.15.42:0/3774073552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe16c076670 0x7fe16c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 >> 172.21.15.42:0/3774073552 conn(0x7fe16c079d00 msgr2=0x7fe16c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 shutdown_connections 2024-05-27T10:14:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:02.805+0000 7fe170e00700 1 -- 172.21.15.42:0/3774073552 wait complete. 2024-05-27T10:14:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:14:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:14:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:02 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:14:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:14:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:02 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:03.872 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T10:14:03.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T10:14:03.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:03 smithi042 ceph-mon[31661]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:03.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:03 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:14:03.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:03 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3774073552' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T10:14:03.942 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:03 smithi143 ceph-mon[35545]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:03.942 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:03 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:14:03.942 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:03 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3774073552' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T10:14:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:03 smithi104 ceph-mon[34812]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:03 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:14:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:03 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3774073552' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T10:14:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:05 smithi042 ceph-mon[31661]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:05 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:05 smithi104 ceph-mon[34812]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:05 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:05 smithi143 ceph-mon[35545]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:05 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 -- 172.21.15.42:0/1932807706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c105e60 msgr2=0x7f5f8c10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 --2- 172.21.15.42:0/1932807706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c105e60 0x7f5f8c10e3b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009990 tx=0x7f5f74009ca0 comp rx=0 tx=0).stop 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 -- 172.21.15.42:0/1932807706 shutdown_connections 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 --2- 172.21.15.42:0/1932807706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 --2- 172.21.15.42:0/1932807706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f8c105510 0x7f5f8c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 --2- 172.21.15.42:0/1932807706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c105e60 0x7f5f8c10e3b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.438+0000 7f5f92400700 1 -- 172.21.15.42:0/1932807706 >> 172.21.15.42:0/1932807706 conn(0x7f5f8c0fede0 msgr2=0x7f5f8c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.439+0000 7f5f92400700 1 -- 172.21.15.42:0/1932807706 shutdown_connections 2024-05-27T10:14:07.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.439+0000 7f5f92400700 1 -- 172.21.15.42:0/1932807706 wait complete. 2024-05-27T10:14:07.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.440+0000 7f5f92400700 1 Processor -- start 2024-05-27T10:14:07.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.440+0000 7f5f92400700 1 -- start start 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.440+0000 7f5f92400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c105510 0x7f5f8c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f92400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f8c105e60 0x7f5f8c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f92400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f92400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c114340 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f92400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1141c0 con 0x7f5f8c105510 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f92400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1144c0 con 0x7f5f8c105e60 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c19f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43882/0 (socket says 172.21.15.42:43882) 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 -- 172.21.15.42:0/2456583440 learned_addr learned my addr 172.21.15.42:0/2456583440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8b400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c105510 0x7f5f8c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:07.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f83e00700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f8c105e60 0x7f5f8c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:07.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 -- 172.21.15.42:0/2456583440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f8c105e60 msgr2=0x7f5f8c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:07.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f8c105e60 0x7f5f8c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 -- 172.21.15.42:0/2456583440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c105510 msgr2=0x7f5f8c19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c105510 0x7f5f8c19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.441+0000 7f5f8be00700 1 -- 172.21.15.42:0/2456583440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c19fb30 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.442+0000 7f5f8be00700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c19f540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007ee0 tx=0x7f5f8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.442+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400dbd0 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.442+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f8400dd70 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.442+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c19fe20 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.442+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840058d0 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.442+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c1a0360 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.444+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c0512b0 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.448+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5f8401a020 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.449+0000 7f5f88c00700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5f700b2900 0x7f5f700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:07.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.450+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f5f840e1c60 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.450+0000 7f5f8b400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5f700b2900 0x7f5f700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:07.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.451+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5f84010070 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.451+0000 7f5f8b400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5f700b2900 0x7f5f700b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:07.712+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5f8c02e5e0 con 0x7f5f8c10e8f0 2024-05-27T10:14:07.713 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:07 smithi042 ceph-mon[31661]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:07.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:07 smithi104 ceph-mon[34812]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:08.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:07 smithi143 ceph-mon[35545]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:08.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.724+0000 7f5f88c00700 1 -- 172.21.15.42:0/2456583440 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f5f840a9d80 con 0x7f5f8c10e8f0 2024-05-27T10:14:08.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5f700b2900 msgr2=0x7f5f700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5f700b2900 0x7f5f700b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c005e50 comp rx=0 tx=0).stop 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 msgr2=0x7f5f8c19f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c19f540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007ee0 tx=0x7f5f8400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 shutdown_connections 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f8c10e8f0 0x7f5f8c19f540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5f700b2900 0x7f5f700b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8c105510 0x7f5f8c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 --2- 172.21.15.42:0/2456583440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f8c105e60 0x7f5f8c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.729+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 >> 172.21.15.42:0/2456583440 conn(0x7f5f8c0fede0 msgr2=0x7f5f8c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.730+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 shutdown_connections 2024-05-27T10:14:08.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:08.730+0000 7f5f92400700 1 -- 172.21.15.42:0/2456583440 wait complete. 2024-05-27T10:14:08.733 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-27T10:14:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:08 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2456583440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T10:14:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:14:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:14:08.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:08 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2456583440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T10:14:09.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:14:09.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:14:09.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:08 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2456583440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T10:14:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:14:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:14:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:14:09.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T10:14:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:09 smithi143 ceph-mon[35545]: pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:09 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2456583440' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T10:14:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:09 smithi143 ceph-mon[35545]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T10:14:10.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:09 smithi042 ceph-mon[31661]: pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:10.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:09 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2456583440' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T10:14:10.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:09 smithi042 ceph-mon[31661]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T10:14:10.146 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:09 smithi104 ceph-mon[34812]: pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:09 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2456583440' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T10:14:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:09 smithi104 ceph-mon[34812]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T10:14:10.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:10 smithi042 ceph-mon[31661]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T10:14:10.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:10 smithi042 ceph-mon[31661]: pgmap v379: 33 pgs: 4 active+clean, 5 creating+peering, 24 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:11.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:10 smithi143 ceph-mon[35545]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T10:14:11.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:10 smithi143 ceph-mon[35545]: pgmap v379: 33 pgs: 4 active+clean, 5 creating+peering, 24 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:11.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:10 smithi104 ceph-mon[34812]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T10:14:11.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:10 smithi104 ceph-mon[34812]: pgmap v379: 33 pgs: 4 active+clean, 5 creating+peering, 24 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:11.974 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:11 smithi042 ceph-mon[31661]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T10:14:11.974 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:11 smithi042 ceph-mon[31661]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T10:14:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:11 smithi143 ceph-mon[35545]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T10:14:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:11 smithi143 ceph-mon[35545]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T10:14:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:11 smithi104 ceph-mon[34812]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T10:14:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:11 smithi104 ceph-mon[34812]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T10:14:12.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 -- 172.21.15.42:0/2223246152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 msgr2=0x7f4b6410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:12.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 --2- 172.21.15.42:0/2223246152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b6410ad90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b0a0 tx=0x7f4b4c00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 -- 172.21.15.42:0/2223246152 shutdown_connections 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 --2- 172.21.15.42:0/2223246152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 0x7f4b64117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 --2- 172.21.15.42:0/2223246152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b64110960 0x7f4b64110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 --2- 172.21.15.42:0/2223246152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b6410ad90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 -- 172.21.15.42:0/2223246152 >> 172.21.15.42:0/2223246152 conn(0x7f4b64077c70 msgr2=0x7f4b640780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 -- 172.21.15.42:0/2223246152 shutdown_connections 2024-05-27T10:14:12.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.646+0000 7f4b6a400700 1 -- 172.21.15.42:0/2223246152 wait complete. 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.647+0000 7f4b6a400700 1 Processor -- start 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.647+0000 7f4b6a400700 1 -- start start 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b6a400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b641a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b6a400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 0x7f4b641a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b6a400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b64110960 0x7f4b641a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b6a400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6411a980 con 0x7f4b6410a980 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b6a400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6411a800 con 0x7f4b6410b2d0 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b6a400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6411ab00 con 0x7f4b64110960 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b63400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b641a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:12.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b63400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b641a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43902/0 (socket says 172.21.15.42:43902) 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.648+0000 7f4b63400700 1 -- 172.21.15.42:0/3102629829 learned_addr learned my addr 172.21.15.42:0/3102629829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b5aa00700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 0x7f4b641a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63e00700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b64110960 0x7f4b641a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63400700 1 -- 172.21.15.42:0/3102629829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b64110960 msgr2=0x7f4b641a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b64110960 0x7f4b641a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63400700 1 -- 172.21.15.42:0/3102629829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 msgr2=0x7f4b641a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 0x7f4b641a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63400700 1 -- 172.21.15.42:0/3102629829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b641a6280 con 0x7f4b6410a980 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b5aa00700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 0x7f4b641a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.649+0000 7f4b63400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b641a13b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d950 tx=0x7f4b5400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.650+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54016070 con 0x7f4b6410a980 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.650+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b5400de30 con 0x7f4b6410a980 2024-05-27T10:14:12.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.650+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54005240 con 0x7f4b6410a980 2024-05-27T10:14:12.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.650+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b641a6570 con 0x7f4b6410a980 2024-05-27T10:14:12.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.650+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641ade90 con 0x7f4b6410a980 2024-05-27T10:14:12.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.651+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6410c070 con 0x7f4b6410a980 2024-05-27T10:14:12.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.654+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4b540049a0 con 0x7f4b6410a980 2024-05-27T10:14:12.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.656+0000 7f4b60c00700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4b480b2900 0x7f4b480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:12.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.656+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f4b540e1a90 con 0x7f4b6410a980 2024-05-27T10:14:12.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.656+0000 7f4b5aa00700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4b480b2900 0x7f4b480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:12.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.657+0000 7f4b5aa00700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4b480b2900 0x7f4b480b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4b6407c9f0 tx=0x7f4b4c00b7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:12.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.659+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b5401f070 con 0x7f4b6410a980 2024-05-27T10:14:12.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:12.900+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4b6402e5e0 con 0x7f4b6410a980 2024-05-27T10:14:12.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:12 smithi042 ceph-mon[31661]: pgmap v381: 33 pgs: 20 active+clean, 5 creating+peering, 8 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:13.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:12 smithi143 ceph-mon[35545]: pgmap v381: 33 pgs: 20 active+clean, 5 creating+peering, 8 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:13.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:12 smithi104 ceph-mon[34812]: pgmap v381: 33 pgs: 20 active+clean, 5 creating+peering, 8 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.775+0000 7f4b60c00700 1 -- 172.21.15.42:0/3102629829 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f4b540a9a40 con 0x7f4b6410a980 2024-05-27T10:14:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.779+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4b480b2900 msgr2=0x7f4b480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.779+0000 7f4b6a400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4b480b2900 0x7f4b480b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4b6407c9f0 tx=0x7f4b4c00b7d0 comp rx=0 tx=0).stop 2024-05-27T10:14:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.779+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 msgr2=0x7f4b641a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.779+0000 7f4b6a400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b641a13b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d950 tx=0x7f4b5400dd10 comp rx=0 tx=0).stop 2024-05-27T10:14:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.780+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 shutdown_connections 2024-05-27T10:14:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.780+0000 7f4b6a400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b64110960 0x7f4b641a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.780+0000 7f4b6a400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b6410a980 0x7f4b641a13b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.780+0000 7f4b6a400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4b480b2900 0x7f4b480b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.780+0000 7f4b6a400700 1 --2- 172.21.15.42:0/3102629829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b6410b2d0 0x7f4b641a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.780+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 >> 172.21.15.42:0/3102629829 conn(0x7f4b64077c70 msgr2=0x7f4b6407b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.781+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 shutdown_connections 2024-05-27T10:14:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:13.781+0000 7f4b6a400700 1 -- 172.21.15.42:0/3102629829 wait complete. 2024-05-27T10:14:13.783 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-27T10:14:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:13 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3102629829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T10:14:14.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:13 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3102629829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T10:14:14.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:13 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3102629829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T10:14:14.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T10:14:15.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:14 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3102629829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T10:14:15.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:14 smithi042 ceph-mon[31661]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T10:14:15.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:14 smithi042 ceph-mon[31661]: pgmap v383: 97 pgs: 12 creating+peering, 52 unknown, 33 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:15.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:14 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3102629829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T10:14:15.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:14 smithi104 ceph-mon[34812]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T10:14:15.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:14 smithi104 ceph-mon[34812]: pgmap v383: 97 pgs: 12 creating+peering, 52 unknown, 33 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:15.285 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:14 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3102629829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T10:14:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:14 smithi143 ceph-mon[35545]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T10:14:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:14 smithi143 ceph-mon[35545]: pgmap v383: 97 pgs: 12 creating+peering, 52 unknown, 33 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:15.954 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:15 smithi042 ceph-mon[31661]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T10:14:16.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:15 smithi104 ceph-mon[34812]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T10:14:16.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:15 smithi143 ceph-mon[35545]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T10:14:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:16 smithi042 ceph-mon[31661]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T10:14:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:16 smithi042 ceph-mon[31661]: pgmap v386: 97 pgs: 12 creating+peering, 40 unknown, 45 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:17.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:16 smithi104 ceph-mon[34812]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T10:14:17.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:16 smithi104 ceph-mon[34812]: pgmap v386: 97 pgs: 12 creating+peering, 40 unknown, 45 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:17.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:16 smithi143 ceph-mon[35545]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T10:14:17.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:16 smithi143 ceph-mon[35545]: pgmap v386: 97 pgs: 12 creating+peering, 40 unknown, 45 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:17.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 -- 172.21.15.42:0/1348429292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 msgr2=0x7ff3a8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:17.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 --2- 172.21.15.42:0/1348429292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a8114e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff394009990 tx=0x7ff394009ca0 comp rx=0 tx=0).stop 2024-05-27T10:14:17.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 -- 172.21.15.42:0/1348429292 shutdown_connections 2024-05-27T10:14:17.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 --2- 172.21.15.42:0/1348429292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a8115360 0x7ff3a8117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:17.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 --2- 172.21.15.42:0/1348429292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8075b50 0x7ff3a8075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:17.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 --2- 172.21.15.42:0/1348429292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a8114e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:17.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 -- 172.21.15.42:0/1348429292 >> 172.21.15.42:0/1348429292 conn(0x7ff3a8100030 msgr2=0x7ff3a8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:17.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.817+0000 7ff3afc00700 1 -- 172.21.15.42:0/1348429292 shutdown_connections 2024-05-27T10:14:17.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.818+0000 7ff3afc00700 1 -- 172.21.15.42:0/1348429292 wait complete. 2024-05-27T10:14:17.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.819+0000 7ff3afc00700 1 Processor -- start 2024-05-27T10:14:17.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.819+0000 7ff3afc00700 1 -- start start 2024-05-27T10:14:17.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3afc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a8075b50 0x7ff3a81a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:17.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3afc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a81a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:17.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3afc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8115360 0x7ff3a81a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3afc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a811a960 con 0x7ff3a80764a0 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3afc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a811a7e0 con 0x7ff3a8115360 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3afc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a811aae0 con 0x7ff3a8075b50 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a81a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a81a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60084/0 (socket says 172.21.15.42:60084) 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.820+0000 7ff3a7e00700 1 -- 172.21.15.42:0/226019666 learned_addr learned my addr 172.21.15.42:0/226019666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3ad200700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a8075b50 0x7ff3a81a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:17.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3adc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8115360 0x7ff3a81a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:17.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3a7e00700 1 -- 172.21.15.42:0/226019666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a8075b50 msgr2=0x7ff3a81a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:17.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3a7e00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a8075b50 0x7ff3a81a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:17.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3a7e00700 1 -- 172.21.15.42:0/226019666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8115360 msgr2=0x7ff3a81a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:17.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3a7e00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8115360 0x7ff3a81a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:17.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3a7e00700 1 -- 172.21.15.42:0/226019666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a81a6240 con 0x7ff3a80764a0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.821+0000 7ff3adc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8115360 0x7ff3a81a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.822+0000 7ff3a7e00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a81a1910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b650 tx=0x7ff3940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.822+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394027070 con 0x7ff3a80764a0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.822+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff394022470 con 0x7ff3a80764a0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.822+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a81a64d0 con 0x7ff3a80764a0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.822+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394017650 con 0x7ff3a80764a0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.822+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a81a69b0 con 0x7ff3a80764a0 2024-05-27T10:14:17.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.823+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a80512b0 con 0x7ff3a80764a0 2024-05-27T10:14:17.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.827+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff394009d10 con 0x7ff3a80764a0 2024-05-27T10:14:17.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.828+0000 7ff3a5600700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7ff3900b2880 0x7ff3900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:17.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.828+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7ff3940e2f40 con 0x7ff3a80764a0 2024-05-27T10:14:17.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.829+0000 7ff3ad200700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7ff3900b2880 0x7ff3900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:17.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.829+0000 7ff3ad200700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7ff3900b2880 0x7ff3900b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009c60 tx=0x7ff39c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:17.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:17.832+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff394013070 con 0x7ff3a80764a0 2024-05-27T10:14:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.124+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff3a802e5e0 con 0x7ff3a80764a0 2024-05-27T10:14:18.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.181+0000 7ff3a5600700 1 -- 172.21.15.42:0/226019666 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff3940a4e20 con 0x7ff3a80764a0 2024-05-27T10:14:18.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.185+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7ff3900b2880 msgr2=0x7ff3900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:18.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.185+0000 7ff3afc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7ff3900b2880 0x7ff3900b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009c60 tx=0x7ff39c009260 comp rx=0 tx=0).stop 2024-05-27T10:14:18.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.185+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 msgr2=0x7ff3a81a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:18.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.185+0000 7ff3afc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a81a1910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b650 tx=0x7ff3940047b0 comp rx=0 tx=0).stop 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 shutdown_connections 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7ff3900b2880 0x7ff3900b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a8115360 0x7ff3a81a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a8075b50 0x7ff3a81a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 --2- 172.21.15.42:0/226019666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80764a0 0x7ff3a81a1910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 >> 172.21.15.42:0/226019666 conn(0x7ff3a8100030 msgr2=0x7ff3a8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 shutdown_connections 2024-05-27T10:14:18.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:18.186+0000 7ff3afc00700 1 -- 172.21.15.42:0/226019666 wait complete. 2024-05-27T10:14:18.188 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T10:14:18.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:18 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/226019666' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T10:14:18.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:18 smithi104 ceph-mon[34812]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T10:14:18.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:18 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/226019666' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T10:14:18.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:18 smithi143 ceph-mon[35545]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T10:14:18.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:18 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/226019666' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T10:14:18.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:18 smithi042 ceph-mon[31661]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T10:14:19.189 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:14:19.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:19 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/226019666' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T10:14:19.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:19 smithi143 ceph-mon[35545]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T10:14:19.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:19 smithi143 ceph-mon[35545]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T10:14:19.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:19 smithi143 ceph-mon[35545]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T10:14:19.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:19 smithi143 ceph-mon[35545]: pgmap v388: 97 pgs: 12 creating+peering, 8 unknown, 77 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:19.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:19 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/226019666' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T10:14:19.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:19 smithi042 ceph-mon[31661]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T10:14:19.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:19 smithi042 ceph-mon[31661]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T10:14:19.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:19 smithi042 ceph-mon[31661]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T10:14:19.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:19 smithi042 ceph-mon[31661]: pgmap v388: 97 pgs: 12 creating+peering, 8 unknown, 77 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:19 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/226019666' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T10:14:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:19 smithi104 ceph-mon[34812]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T10:14:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:19 smithi104 ceph-mon[34812]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T10:14:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:19 smithi104 ceph-mon[34812]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T10:14:19.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:19 smithi104 ceph-mon[34812]: pgmap v388: 97 pgs: 12 creating+peering, 8 unknown, 77 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:20 smithi042 ceph-mon[31661]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T10:14:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:20 smithi042 ceph-mon[31661]: Cluster is now healthy 2024-05-27T10:14:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:20 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] up:active 2024-05-27T10:14:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:20 smithi042 ceph-mon[31661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:20 smithi042 ceph-mon[31661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:20.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:20 smithi104 ceph-mon[34812]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T10:14:20.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:20 smithi104 ceph-mon[34812]: Cluster is now healthy 2024-05-27T10:14:20.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:20 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] up:active 2024-05-27T10:14:20.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:20 smithi104 ceph-mon[34812]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:20.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:20 smithi104 ceph-mon[34812]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:20.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:20 smithi143 ceph-mon[35545]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T10:14:20.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:20 smithi143 ceph-mon[35545]: Cluster is now healthy 2024-05-27T10:14:20.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:20 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] up:active 2024-05-27T10:14:20.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:20 smithi143 ceph-mon[35545]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:20.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:20 smithi143 ceph-mon[35545]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:20.948 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:14:20 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b[65158]: 2024-05-27T10:14:20.455+0000 7f81dfe00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T10:14:21.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:21 smithi042 ceph-mon[31661]: pgmap v389: 97 pgs: 97 active+clean; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-27T10:14:21.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:21 smithi143 ceph-mon[35545]: pgmap v389: 97 pgs: 97 active+clean; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-27T10:14:21.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:21 smithi104 ceph-mon[34812]: pgmap v389: 97 pgs: 97 active+clean; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-27T10:14:22.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 -- 172.21.15.42:0/603422446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0075b50 msgr2=0x7fedb0075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:22.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 --2- 172.21.15.42:0/603422446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0075b50 0x7fedb0075f60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feda000b0a0 tx=0x7feda000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 -- 172.21.15.42:0/603422446 shutdown_connections 2024-05-27T10:14:22.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 --2- 172.21.15.42:0/603422446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedb0115170 0x7fedb0117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 --2- 172.21.15.42:0/603422446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0075b50 0x7fedb0075f60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 --2- 172.21.15.42:0/603422446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fedb00764a0 0x7fedb0114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.090+0000 7fedafe00700 1 -- 172.21.15.42:0/603422446 >> 172.21.15.42:0/603422446 conn(0x7fedb0079d00 msgr2=0x7fedb007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.091+0000 7fedafe00700 1 -- 172.21.15.42:0/603422446 shutdown_connections 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.091+0000 7fedafe00700 1 -- 172.21.15.42:0/603422446 wait complete. 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.092+0000 7fedafe00700 1 Processor -- start 2024-05-27T10:14:22.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.092+0000 7fedafe00700 1 -- start start 2024-05-27T10:14:22.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedafe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fedb0075b50 0x7fedb01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:22.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedafe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedb00764a0 0x7fedb01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedafe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 0x7fedb01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedafe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fedb011a780 con 0x7fedb0115170 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedafe00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fedb011a600 con 0x7fedb00764a0 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedafe00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fedb011a900 con 0x7fedb0075b50 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.093+0000 7fedaf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 0x7fedb01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 0x7fedb01a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60104/0 (socket says 172.21.15.42:60104) 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 -- 172.21.15.42:0/2996738724 learned_addr learned my addr 172.21.15.42:0/2996738724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaea00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fedb0075b50 0x7fedb01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7feda6000700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedb00764a0 0x7fedb01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:22.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 -- 172.21.15.42:0/2996738724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fedb0075b50 msgr2=0x7fedb01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:22.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fedb0075b50 0x7fedb01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 -- 172.21.15.42:0/2996738724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedb00764a0 msgr2=0x7fedb01a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:22.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedb00764a0 0x7fedb01a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.094+0000 7fedaf400700 1 -- 172.21.15.42:0/2996738724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb01a6080 con 0x7fedb0115170 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.095+0000 7fedaf400700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 0x7fedb01a59e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feda800d980 tx=0x7feda800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.095+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8017070 con 0x7fedb0115170 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.095+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feda8005020 con 0x7fedb0115170 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.095+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb01a6310 con 0x7fedb0115170 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.095+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda800f980 con 0x7fedb0115170 2024-05-27T10:14:22.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.095+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb01a6800 con 0x7fedb0115170 2024-05-27T10:14:22.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.096+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb0077290 con 0x7fedb0115170 2024-05-27T10:14:22.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.101+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7feda8004a30 con 0x7fedb0115170 2024-05-27T10:14:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.102+0000 7feda7e00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fed940b2880 0x7fed940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.102+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7feda80e22e0 con 0x7fedb0115170 2024-05-27T10:14:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.102+0000 7fedaea00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fed940b2880 0x7fed940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.103+0000 7fedaea00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fed940b2880 0x7fed940b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feda0009fd0 tx=0x7feda0006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:22.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.105+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feda8020020 con 0x7fedb0115170 2024-05-27T10:14:22.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.346+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedb002e5e0 con 0x7fedb0115170 2024-05-27T10:14:22.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.351+0000 7feda7e00700 1 -- 172.21.15.42:0/2996738724 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22232 (secure 0 0 0) 0x7feda80aa0b0 con 0x7fedb0115170 2024-05-27T10:14:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:14:22.356 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:18.127326+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:14:22.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fed940b2880 msgr2=0x7fed940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:22.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fed940b2880 0x7fed940b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feda0009fd0 tx=0x7feda0006210 comp rx=0 tx=0).stop 2024-05-27T10:14:22.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 msgr2=0x7fedb01a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:22.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 0x7fedb01a59e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feda800d980 tx=0x7feda800dc90 comp rx=0 tx=0).stop 2024-05-27T10:14:22.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 shutdown_connections 2024-05-27T10:14:22.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedb0115170 0x7fedb01a59e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fedb0075b50 0x7fedb01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fed940b2880 0x7fed940b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 --2- 172.21.15.42:0/2996738724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedb00764a0 0x7fedb01a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:22.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.355+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 >> 172.21.15.42:0/2996738724 conn(0x7fedb0079d00 msgr2=0x7fedb0101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:22.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.356+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 shutdown_connections 2024-05-27T10:14:22.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:22.356+0000 7fedafe00700 1 -- 172.21.15.42:0/2996738724 wait complete. 2024-05-27T10:14:22.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:22 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2996738724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:14:22.740 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:22 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2996738724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:14:22.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:22 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2996738724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:14:23.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:14:23.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:23 smithi042 ceph-mon[31661]: pgmap v390: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-05-27T10:14:23.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:23 smithi104 ceph-mon[34812]: pgmap v390: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-05-27T10:14:23.786 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:23 smithi143 ceph-mon[35545]: pgmap v390: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-05-27T10:14:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:25 smithi042 ceph-mon[31661]: pgmap v391: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-27T10:14:25.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:25 smithi143 ceph-mon[35545]: pgmap v391: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-27T10:14:25.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:25 smithi104 ceph-mon[34812]: pgmap v391: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-27T10:14:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 -- 172.21.15.42:0/1506535535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10b2d0 msgr2=0x7f229c117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 --2- 172.21.15.42:0/1506535535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10b2d0 0x7f229c117800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f229400b0a0 tx=0x7f229400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 -- 172.21.15.42:0/1506535535 shutdown_connections 2024-05-27T10:14:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 --2- 172.21.15.42:0/1506535535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10b2d0 0x7f229c117800 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 --2- 172.21.15.42:0/1506535535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 0x7f229c110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 --2- 172.21.15.42:0/1506535535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f229c10a980 0x7f229c10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 -- 172.21.15.42:0/1506535535 >> 172.21.15.42:0/1506535535 conn(0x7f229c077c70 msgr2=0x7f229c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 -- 172.21.15.42:0/1506535535 shutdown_connections 2024-05-27T10:14:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.497+0000 7f22a3400700 1 -- 172.21.15.42:0/1506535535 wait complete. 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.498+0000 7f22a3400700 1 Processor -- start 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.498+0000 7f22a3400700 1 -- start start 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 0x7f229c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a3400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f229c10b2d0 0x7f229c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a3400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 0x7f229c1a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a3400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f229c11a930 con 0x7f229c10a980 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a3400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f229c11a7b0 con 0x7f229c110960 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a3400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f229c11aab0 con 0x7f229c10b2d0 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a0a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 0x7f229c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a0a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 0x7f229c1a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43950/0 (socket says 172.21.15.42:43950) 2024-05-27T10:14:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a0a00700 1 -- 172.21.15.42:0/1119899532 learned_addr learned my addr 172.21.15.42:0/1119899532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f229be00700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f229c10b2d0 0x7f229c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.499+0000 7f22a1400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 0x7f229c1a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a0a00700 1 -- 172.21.15.42:0/1119899532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f229c10b2d0 msgr2=0x7f229c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a0a00700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f229c10b2d0 0x7f229c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a0a00700 1 -- 172.21.15.42:0/1119899532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 msgr2=0x7f229c1a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a0a00700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 0x7f229c1a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a0a00700 1 -- 172.21.15.42:0/1119899532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c1a6230 con 0x7f229c10a980 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a1400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 0x7f229c1a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:14:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a0a00700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 0x7f229c1a1360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f229000d980 tx=0x7f229000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290017070 con 0x7f229c10a980 2024-05-27T10:14:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2290009dc0 con 0x7f229c10a980 2024-05-27T10:14:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000f3c0 con 0x7f229c10a980 2024-05-27T10:14:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.500+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c1a6520 con 0x7f229c10a980 2024-05-27T10:14:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.501+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1ade40 con 0x7f229c10a980 2024-05-27T10:14:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.502+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c10c070 con 0x7f229c10a980 2024-05-27T10:14:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.504+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2290009930 con 0x7f229c10a980 2024-05-27T10:14:26.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.507+0000 7f2299600700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f22840b2900 0x7f22840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:26.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.507+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f22900e1b70 con 0x7f229c10a980 2024-05-27T10:14:26.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.507+0000 7f229be00700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f22840b2900 0x7f22840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:26.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.508+0000 7f229be00700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f22840b2900 0x7f22840b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f229c109940 tx=0x7f228800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:26.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.510+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22900a99c0 con 0x7f229c10a980 2024-05-27T10:14:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.753+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f229c02e5e0 con 0x7f229c10a980 2024-05-27T10:14:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.756+0000 7f2299600700 1 -- 172.21.15.42:0/1119899532 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22232 (secure 0 0 0) 0x7f22900a92c0 con 0x7f229c10a980 2024-05-27T10:14:26.757 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:14:26.757 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:18.127326+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:14:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f22840b2900 msgr2=0x7f22840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f22840b2900 0x7f22840b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f229c109940 tx=0x7f228800b410 comp rx=0 tx=0).stop 2024-05-27T10:14:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 msgr2=0x7f229c1a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 0x7f229c1a1360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f229000d980 tx=0x7f229000dc90 comp rx=0 tx=0).stop 2024-05-27T10:14:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 shutdown_connections 2024-05-27T10:14:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f22840b2900 0x7f22840b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f229c110960 0x7f229c1a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f229c10a980 0x7f229c1a1360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 --2- 172.21.15.42:0/1119899532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f229c10b2d0 0x7f229c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:26.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.759+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 >> 172.21.15.42:0/1119899532 conn(0x7f229c077c70 msgr2=0x7f229c07b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:26.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.760+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 shutdown_connections 2024-05-27T10:14:26.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:26.760+0000 7f22a3400700 1 -- 172.21.15.42:0/1119899532 wait complete. 2024-05-27T10:14:27.747 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:27 smithi042 ceph-mon[31661]: pgmap v392: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T10:14:27.747 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:27 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1119899532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:14:27.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-27T10:14:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:27 smithi143 ceph-mon[35545]: pgmap v392: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T10:14:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:27 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1119899532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:14:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:27 smithi104 ceph-mon[34812]: pgmap v392: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T10:14:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:27 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1119899532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:14:28.241 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:29.748 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:29 smithi042 ceph-mon[31661]: pgmap v393: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T10:14:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:29 smithi143 ceph-mon[35545]: pgmap v393: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T10:14:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:29 smithi104 ceph-mon[34812]: pgmap v393: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T10:14:30.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 -- 172.21.15.42:0/3139480561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8075b50 msgr2=0x7f4bf8075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3139480561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8075b50 0x7f4bf8075f60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009990 tx=0x7f4be8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 -- 172.21.15.42:0/3139480561 shutdown_connections 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3139480561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf8115360 0x7f4bf8117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3139480561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8075b50 0x7f4bf8075f60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3139480561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80764a0 0x7f4bf8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 -- 172.21.15.42:0/3139480561 >> 172.21.15.42:0/3139480561 conn(0x7f4bf8100030 msgr2=0x7f4bf8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 -- 172.21.15.42:0/3139480561 shutdown_connections 2024-05-27T10:14:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.711+0000 7f4bfe400700 1 -- 172.21.15.42:0/3139480561 wait complete. 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.713+0000 7f4bfe400700 1 Processor -- start 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.713+0000 7f4bfe400700 1 -- start start 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.713+0000 7f4bfe400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf8075b50 0x7f4bf81a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.713+0000 7f4bfe400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf80764a0 0x7f4bf81a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bfe400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 0x7f4bf81a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bfe400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf811a960 con 0x7f4bf8115360 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bfe400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf811a7e0 con 0x7f4bf8075b50 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bfe400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf811aae0 con 0x7f4bf80764a0 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 0x7f4bf81a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 0x7f4bf81a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43978/0 (socket says 172.21.15.42:43978) 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf8075b50 0x7f4bf81a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 -- 172.21.15.42:0/3286103999 learned_addr learned my addr 172.21.15.42:0/3286103999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:30.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf6a00700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf80764a0 0x7f4bf81a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 -- 172.21.15.42:0/3286103999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf80764a0 msgr2=0x7f4bf81a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf80764a0 0x7f4bf81a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 -- 172.21.15.42:0/3286103999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf8075b50 msgr2=0x7f4bf81a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf8075b50 0x7f4bf81a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.714+0000 7f4bf7e00700 1 -- 172.21.15.42:0/3286103999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf81a6220 con 0x7f4bf8115360 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.715+0000 7f4bf7e00700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 0x7f4bf81a5b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0007ee0 tx=0x7f4bf000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.715+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000dbd0 con 0x7f4bf8115360 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.715+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bf000dd70 con 0x7f4bf8115360 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.715+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf81a6510 con 0x7f4bf8115360 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.715+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00058d0 con 0x7f4bf8115360 2024-05-27T10:14:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.715+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf81ade30 con 0x7f4bf8115360 2024-05-27T10:14:30.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.717+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf8077290 con 0x7f4bf8115360 2024-05-27T10:14:30.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.721+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4bf001a020 con 0x7f4bf8115360 2024-05-27T10:14:30.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.722+0000 7f4befe00700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:30.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.723+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f4bf00e1f90 con 0x7f4bf8115360 2024-05-27T10:14:30.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.723+0000 7f4bf7400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:30.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.724+0000 7f4bf7400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4be8006010 tx=0x7f4be800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:30.725+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bf0010070 con 0x7f4bf8115360 2024-05-27T10:14:31.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.023+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4bf802e5e0 con 0x7f4bf8115360 2024-05-27T10:14:31.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.520+0000 7f4befe00700 1 -- 172.21.15.42:0/3286103999 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f4bf00a9d60 con 0x7f4bf8115360 2024-05-27T10:14:31.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4bdc0b2880 msgr2=0x7f4bdc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:31.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4be8006010 tx=0x7f4be800b540 comp rx=0 tx=0).stop 2024-05-27T10:14:31.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 msgr2=0x7f4bf81a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:31.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 0x7f4bf81a5b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0007ee0 tx=0x7f4bf000d3b0 comp rx=0 tx=0).stop 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 shutdown_connections 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bf8115360 0x7f4bf81a5b80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf8075b50 0x7f4bf81a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 --2- 172.21.15.42:0/3286103999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf80764a0 0x7f4bf81a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.525+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 >> 172.21.15.42:0/3286103999 conn(0x7f4bf8100030 msgr2=0x7f4bf8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.526+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 shutdown_connections 2024-05-27T10:14:31.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:31.526+0000 7f4bfe400700 1 -- 172.21.15.42:0/3286103999 wait complete. 2024-05-27T10:14:31.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:31 smithi143 ceph-mon[35545]: pgmap v394: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-27T10:14:31.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:31 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3286103999' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T10:14:31.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:31 smithi042 ceph-mon[31661]: pgmap v394: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-27T10:14:31.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:31 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3286103999' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T10:14:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:31 smithi104 ceph-mon[34812]: pgmap v394: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-27T10:14:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:31 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3286103999' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T10:14:32.566 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-27T10:14:32.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-27T10:14:32.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:32 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3286103999' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T10:14:32.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:32 smithi143 ceph-mon[35545]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:32.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:32 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3286103999' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T10:14:32.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:32 smithi042 ceph-mon[31661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:32.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:32 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3286103999' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T10:14:32.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:32 smithi104 ceph-mon[34812]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:33.035 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:33.746 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:33 smithi042 ceph-mon[31661]: pgmap v395: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-27T10:14:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:33 smithi143 ceph-mon[35545]: pgmap v395: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-27T10:14:33.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:33 smithi104 ceph-mon[34812]: pgmap v395: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 -- 172.21.15.42:0/1897999441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6568108f10 msgr2=0x7f6568109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 --2- 172.21.15.42:0/1897999441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6568108f10 0x7f6568109320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f6554009cd0 comp rx=0 tx=0).stop 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 -- 172.21.15.42:0/1897999441 shutdown_connections 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 --2- 172.21.15.42:0/1897999441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568109860 0x7f65681111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 --2- 172.21.15.42:0/1897999441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65681045e0 0x7f65681049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 --2- 172.21.15.42:0/1897999441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6568108f10 0x7f6568109320 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:35.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.711+0000 7f656da00700 1 -- 172.21.15.42:0/1897999441 >> 172.21.15.42:0/1897999441 conn(0x7f6568100030 msgr2=0x7f6568102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:35.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.712+0000 7f656da00700 1 -- 172.21.15.42:0/1897999441 shutdown_connections 2024-05-27T10:14:35.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.712+0000 7f656da00700 1 -- 172.21.15.42:0/1897999441 wait complete. 2024-05-27T10:14:35.714 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:35 smithi042 ceph-mon[31661]: pgmap v396: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T10:14:35.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.713+0000 7f656da00700 1 Processor -- start 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.713+0000 7f656da00700 1 -- start start 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f656da00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 0x7f656819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f656da00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568108f10 0x7f656819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f656da00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6568109860 0x7f656819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f656da00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6568114370 con 0x7f65681045e0 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f656da00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65681141f0 con 0x7f6568108f10 2024-05-27T10:14:35.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f656da00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65681144f0 con 0x7f6568109860 2024-05-27T10:14:35.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f6566a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 0x7f656819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f6566a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 0x7f656819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42226/0 (socket says 172.21.15.42:42226) 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.714+0000 7f6566a00700 1 -- 172.21.15.42:0/3932341950 learned_addr learned my addr 172.21.15.42:0/3932341950 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566000700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568108f10 0x7f656819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6567400700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6568109860 0x7f656819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566a00700 1 -- 172.21.15.42:0/3932341950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6568109860 msgr2=0x7f656819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566a00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6568109860 0x7f656819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566a00700 1 -- 172.21.15.42:0/3932341950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568108f10 msgr2=0x7f656819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566a00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568108f10 0x7f656819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566a00700 1 -- 172.21.15.42:0/3932341950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656819fcb0 con 0x7f65681045e0 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.715+0000 7f6566a00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 0x7f656819ade0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f655800c920 tx=0x7f655800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.716+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558007b00 con 0x7f65681045e0 2024-05-27T10:14:35.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.716+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6558004cc0 con 0x7f65681045e0 2024-05-27T10:14:35.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.716+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580054f0 con 0x7f65681045e0 2024-05-27T10:14:35.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.716+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656819ffa0 con 0x7f65681045e0 2024-05-27T10:14:35.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.716+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65681a78c0 con 0x7f65681045e0 2024-05-27T10:14:35.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.717+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6568105a90 con 0x7f65681045e0 2024-05-27T10:14:35.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.720+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6558007ca0 con 0x7f65681045e0 2024-05-27T10:14:35.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.721+0000 7f655f400700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f654c0b2900 0x7f654c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:35.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.722+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f65580e2100 con 0x7f65681045e0 2024-05-27T10:14:35.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.722+0000 7f6566000700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f654c0b2900 0x7f654c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:35.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.723+0000 7f6566000700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f654c0b2900 0x7f654c0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6554009990 tx=0x7f6554005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:35.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:35.724+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6558014030 con 0x7f65681045e0 2024-05-27T10:14:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:35 smithi143 ceph-mon[35545]: pgmap v396: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T10:14:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:35 smithi104 ceph-mon[34812]: pgmap v396: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T10:14:36.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.019+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f656819c170 con 0x7f65681045e0 2024-05-27T10:14:36.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.560+0000 7f655f400700 1 -- 172.21.15.42:0/3932341950 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f656819c170 con 0x7f65681045e0 2024-05-27T10:14:36.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f654c0b2900 msgr2=0x7f654c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:36.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f654c0b2900 0x7f654c0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6554009990 tx=0x7f6554005fb0 comp rx=0 tx=0).stop 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 msgr2=0x7f656819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 0x7f656819ade0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f655800c920 tx=0x7f655800cce0 comp rx=0 tx=0).stop 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 shutdown_connections 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6568109860 0x7f656819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65681045e0 0x7f656819ade0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f654c0b2900 0x7f654c0b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 --2- 172.21.15.42:0/3932341950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568108f10 0x7f656819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.564+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 >> 172.21.15.42:0/3932341950 conn(0x7f6568100030 msgr2=0x7f656810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:36.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.565+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 shutdown_connections 2024-05-27T10:14:36.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:36.565+0000 7f656da00700 1 -- 172.21.15.42:0/3932341950 wait complete. 2024-05-27T10:14:36.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:36 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3932341950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-27T10:14:36.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:36 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3932341950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-27T10:14:36.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:36 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3932341950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-27T10:14:37.563 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-27T10:14:37.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: pgmap v397: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:37.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T10:14:37.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3932341950' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-27T10:14:37.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T10:14:37.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:37.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T10:14:37.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:37 smithi143 ceph-mon[35545]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T10:14:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: pgmap v397: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T10:14:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3932341950' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-27T10:14:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T10:14:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:37.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T10:14:37.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:37 smithi042 ceph-mon[31661]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T10:14:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: pgmap v397: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:14:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T10:14:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3932341950' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-27T10:14:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T10:14:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T10:14:37.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T10:14:37.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:37 smithi104 ceph-mon[34812]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T10:14:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:38.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:active 2024-05-27T10:14:38.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T10:14:38.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T10:14:38.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: Cluster is now healthy 2024-05-27T10:14:38.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T10:14:38.690 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T10:14:38.691 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:38 smithi042 ceph-mon[31661]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:active 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: Cluster is now healthy 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T10:14:38.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:38 smithi143 ceph-mon[35545]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:active 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: Cluster is now healthy 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T10:14:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:38 smithi104 ceph-mon[34812]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T10:14:39.814 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:39 smithi042 ceph-mon[31661]: pgmap v398: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 1 op/s 2024-05-27T10:14:39.814 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:39 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] up:active 2024-05-27T10:14:39.814 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:39 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T10:14:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:39 smithi104 ceph-mon[34812]: pgmap v398: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 1 op/s 2024-05-27T10:14:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:39 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] up:active 2024-05-27T10:14:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:39 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T10:14:40.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:39 smithi143 ceph-mon[35545]: pgmap v398: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 1 op/s 2024-05-27T10:14:40.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:39 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] up:active 2024-05-27T10:14:40.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:39 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 -- 172.21.15.42:0/2435924581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 msgr2=0x7fa4a8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2435924581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8109320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4940099c0 tx=0x7fa494009cd0 comp rx=0 tx=0).stop 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 -- 172.21.15.42:0/2435924581 shutdown_connections 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2435924581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4a8109860 0x7fa4a81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2435924581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a81045e0 0x7fa4a81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2435924581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8109320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 -- 172.21.15.42:0/2435924581 >> 172.21.15.42:0/2435924581 conn(0x7fa4a8100030 msgr2=0x7fa4a8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 -- 172.21.15.42:0/2435924581 shutdown_connections 2024-05-27T10:14:40.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.432+0000 7fa4adc00700 1 -- 172.21.15.42:0/2435924581 wait complete. 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 Processor -- start 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 -- start start 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a81045e0 0x7fa4a8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4a8109860 0x7fa4a8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a8114320 con 0x7fa4a8108f10 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81141a0 con 0x7fa4a81045e0 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.434+0000 7fa4adc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81144a0 con 0x7fa4a8109860 2024-05-27T10:14:40.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42234/0 (socket says 172.21.15.42:42234) 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 -- 172.21.15.42:0/2820189547 learned_addr learned my addr 172.21.15.42:0/2820189547 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6a00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a81045e0 0x7fa4a8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a7400700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4a8109860 0x7fa4a8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 -- 172.21.15.42:0/2820189547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4a8109860 msgr2=0x7fa4a8076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4a8109860 0x7fa4a8076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 -- 172.21.15.42:0/2820189547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a81045e0 msgr2=0x7fa4a8075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a81045e0 0x7fa4a8075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:40.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.435+0000 7fa4a6000700 1 -- 172.21.15.42:0/2820189547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a81a90e0 con 0x7fa4a8108f10 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.436+0000 7fa4a6000700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8075ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa49400b580 tx=0x7fa494004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.436+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49400f8a0 con 0x7fa4a8108f10 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.436+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a81a9370 con 0x7fa4a8108f10 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.436+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa494004830 con 0x7fa4a8108f10 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.436+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494005350 con 0x7fa4a8108f10 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.436+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a9760 con 0x7fa4a8108f10 2024-05-27T10:14:40.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.437+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a80512b0 con 0x7fa4a8108f10 2024-05-27T10:14:40.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.440+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa494029060 con 0x7fa4a8108f10 2024-05-27T10:14:40.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.441+0000 7fa49f400700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa48c0b2880 0x7fa48c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:40.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.442+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fa4940e2c80 con 0x7fa4a8108f10 2024-05-27T10:14:40.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.442+0000 7fa4a6a00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa48c0b2880 0x7fa48c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:40.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.443+0000 7fa4a6a00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa48c0b2880 0x7fa48c0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa498005950 tx=0x7fa49800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.446+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa494013070 con 0x7fa4a8108f10 2024-05-27T10:14:40.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:40.757+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa4a802e7a0 con 0x7fa4a8108f10 2024-05-27T10:14:40.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T10:14:40.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-27T10:14:40.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-27T10:14:40.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-27T10:14:40.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-27T10:14:40.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-27T10:14:40.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:40 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-27T10:14:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T10:14:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-27T10:14:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-27T10:14:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-27T10:14:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-27T10:14:40.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-27T10:14:40.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:40 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-27T10:14:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T10:14:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-27T10:14:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-27T10:14:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-27T10:14:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-27T10:14:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-27T10:14:41.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:40 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-27T10:14:41.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.620+0000 7fa49f400700 1 -- 172.21.15.42:0/2820189547 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fa4940aaae0 con 0x7fa4a8108f10 2024-05-27T10:14:41.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.625+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa48c0b2880 msgr2=0x7fa48c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:41.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.625+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa48c0b2880 0x7fa48c0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa498005950 tx=0x7fa49800b410 comp rx=0 tx=0).stop 2024-05-27T10:14:41.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.625+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 msgr2=0x7fa4a8075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:41.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.625+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8075ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa49400b580 tx=0x7fa494004790 comp rx=0 tx=0).stop 2024-05-27T10:14:41.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 shutdown_connections 2024-05-27T10:14:41.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4a8109860 0x7fa4a8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:41.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa48c0b2880 0x7fa48c0b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:41.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a81045e0 0x7fa4a8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:41.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 --2- 172.21.15.42:0/2820189547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a8108f10 0x7fa4a8075ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:41.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 >> 172.21.15.42:0/2820189547 conn(0x7fa4a8100030 msgr2=0x7fa4a8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:41.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 shutdown_connections 2024-05-27T10:14:41.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:41.626+0000 7fa4adc00700 1 -- 172.21.15.42:0/2820189547 wait complete. 2024-05-27T10:14:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: pgmap v399: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-27T10:14:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T10:14:41.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:41 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2820189547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T10:14:42.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: pgmap v399: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T10:14:42.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:41 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2820189547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T10:14:42.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: pgmap v399: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-27T10:14:42.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T10:14:42.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-27T10:14:42.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-27T10:14:42.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-27T10:14:42.063 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-27T10:14:42.063 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-27T10:14:42.063 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-27T10:14:42.063 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T10:14:42.063 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:41 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2820189547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T10:14:42.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-27T10:14:42.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:42 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2820189547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T10:14:42.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:42 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T10:14:42.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:42 smithi104 ceph-mon[34812]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T10:14:42.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:42 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:14:43.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:42 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2820189547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T10:14:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:42 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T10:14:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:42 smithi143 ceph-mon[35545]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T10:14:43.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:42 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:14:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:42 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2820189547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T10:14:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:42 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T10:14:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:42 smithi042 ceph-mon[31661]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T10:14:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:42 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:14:43.192 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:43.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:43 smithi104 ceph-mon[34812]: pgmap v402: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 3.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:43 smithi143 ceph-mon[35545]: pgmap v402: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 3.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:44.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:43 smithi042 ceph-mon[31661]: pgmap v402: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 3.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 -- 172.21.15.42:0/2650426392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 msgr2=0x7efdc0110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 --2- 172.21.15.42:0/2650426392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc0110da0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009990 tx=0x7efdb8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:14:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 -- 172.21.15.42:0/2650426392 shutdown_connections 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 --2- 172.21.15.42:0/2650426392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc0110da0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 --2- 172.21.15.42:0/2650426392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc0105450 0x7efdc0105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 --2- 172.21.15.42:0/2650426392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdc0105da0 0x7efdc010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.667+0000 7efdc8600700 1 -- 172.21.15.42:0/2650426392 >> 172.21.15.42:0/2650426392 conn(0x7efdc00fedb0 msgr2=0x7efdc01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.668+0000 7efdc8600700 1 -- 172.21.15.42:0/2650426392 shutdown_connections 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.668+0000 7efdc8600700 1 -- 172.21.15.42:0/2650426392 wait complete. 2024-05-27T10:14:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.669+0000 7efdc8600700 1 Processor -- start 2024-05-27T10:14:45.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.669+0000 7efdc8600700 1 -- start start 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc8600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc0105450 0x7efdc019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc8600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdc0105da0 0x7efdc019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc8600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc01a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc8600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0114320 con 0x7efdc010e8c0 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc8600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efdc01141a0 con 0x7efdc0105da0 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc8600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efdc01144a0 con 0x7efdc0105450 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc6600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc01a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc6600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc01a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40668/0 (socket says 172.21.15.42:40668) 2024-05-27T10:14:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc6600700 1 -- 172.21.15.42:0/3789468148 learned_addr learned my addr 172.21.15.42:0/3789468148 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc5c00700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc0105450 0x7efdc019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.670+0000 7efdc5200700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdc0105da0 0x7efdc019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdc6600700 1 -- 172.21.15.42:0/3789468148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc0105450 msgr2=0x7efdc019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdc6600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc0105450 0x7efdc019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdc6600700 1 -- 172.21.15.42:0/3789468148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdc0105da0 msgr2=0x7efdc019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdc6600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdc0105da0 0x7efdc019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdc6600700 1 -- 172.21.15.42:0/3789468148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc01a1ca0 con 0x7efdc010e8c0 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdc6600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc01a16b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c010 tx=0x7efdb8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8004830 con 0x7efdc010e8c0 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdb800f810 con 0x7efdc010e8c0 2024-05-27T10:14:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.671+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb80053a0 con 0x7efdc010e8c0 2024-05-27T10:14:45.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.672+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc01a1f30 con 0x7efdc010e8c0 2024-05-27T10:14:45.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.672+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc01a2360 con 0x7efdc010e8c0 2024-05-27T10:14:45.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.673+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc00512b0 con 0x7efdc010e8c0 2024-05-27T10:14:45.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.676+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efdb800f9c0 con 0x7efdc010e8c0 2024-05-27T10:14:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.677+0000 7efdbe000700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efda80b2880 0x7efda80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.677+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efdb80e3d90 con 0x7efdc010e8c0 2024-05-27T10:14:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.677+0000 7efdc5c00700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efda80b2880 0x7efda80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.678+0000 7efdc5c00700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efda80b2880 0x7efda80b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efdb400e270 tx=0x7efdb4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.680+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efdb8030070 con 0x7efdc010e8c0 2024-05-27T10:14:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:45 smithi042 ceph-mon[31661]: pgmap v403: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.0 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:45 smithi104 ceph-mon[34812]: pgmap v403: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.0 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:45.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:45.972+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7efdc01a9960 con 0x7efdc010e8c0 2024-05-27T10:14:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:45 smithi143 ceph-mon[35545]: pgmap v403: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.0 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:46.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.672+0000 7efdbe000700 1 -- 172.21.15.42:0/3789468148 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7efdb8013070 con 0x7efdc010e8c0 2024-05-27T10:14:46.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efda80b2880 msgr2=0x7efda80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:46.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efda80b2880 0x7efda80b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efdb400e270 tx=0x7efdb4009040 comp rx=0 tx=0).stop 2024-05-27T10:14:46.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 msgr2=0x7efdc01a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:46.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc01a16b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c010 tx=0x7efdb8009f90 comp rx=0 tx=0).stop 2024-05-27T10:14:46.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 shutdown_connections 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdc010e8c0 0x7efdc01a16b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc0105450 0x7efdc019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7efda80b2880 0x7efda80b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 --2- 172.21.15.42:0/3789468148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdc0105da0 0x7efdc019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 >> 172.21.15.42:0/3789468148 conn(0x7efdc00fedb0 msgr2=0x7efdc00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 shutdown_connections 2024-05-27T10:14:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:46.677+0000 7efdc8600700 1 -- 172.21.15.42:0/3789468148 wait complete. 2024-05-27T10:14:46.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:46 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3789468148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T10:14:47.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:46 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3789468148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T10:14:47.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:46 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3789468148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T10:14:47.641 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-27T10:14:47.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-27T10:14:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:47 smithi042 ceph-mon[31661]: pgmap v404: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:47.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:47 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3789468148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T10:14:47.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:47 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:14:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:47 smithi104 ceph-mon[34812]: pgmap v404: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:47 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3789468148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T10:14:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:47 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:14:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:47 smithi143 ceph-mon[35545]: pgmap v404: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:47 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3789468148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T10:14:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:47 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:14:48.112 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:49.927 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:49 smithi042 ceph-mon[31661]: pgmap v405: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:49.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:49 smithi104 ceph-mon[34812]: pgmap v405: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:50.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:49 smithi143 ceph-mon[35545]: pgmap v405: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:50.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 -- 172.21.15.42:0/488148253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec941045e0 msgr2=0x7fec941049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:50.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 --2- 172.21.15.42:0/488148253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec941045e0 0x7fec941049f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec840099c0 tx=0x7fec84009cd0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 -- 172.21.15.42:0/488148253 shutdown_connections 2024-05-27T10:14:50.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 --2- 172.21.15.42:0/488148253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec94109860 0x7fec941111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 --2- 172.21.15.42:0/488148253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec941045e0 0x7fec941049f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 --2- 172.21.15.42:0/488148253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec94108f10 0x7fec94109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 -- 172.21.15.42:0/488148253 >> 172.21.15.42:0/488148253 conn(0x7fec94100030 msgr2=0x7fec94102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 -- 172.21.15.42:0/488148253 shutdown_connections 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.582+0000 7fec99200700 1 -- 172.21.15.42:0/488148253 wait complete. 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.583+0000 7fec99200700 1 Processor -- start 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.583+0000 7fec99200700 1 -- start start 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec99200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec941045e0 0x7fec9419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec99200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 0x7fec9419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec99200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec94109860 0x7fec9419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec99200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fec94114340 con 0x7fec94108f10 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec99200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fec941141c0 con 0x7fec941045e0 2024-05-27T10:14:50.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec99200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec941144c0 con 0x7fec94109860 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec91400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 0x7fec9419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec91400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 0x7fec9419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40698/0 (socket says 172.21.15.42:40698) 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.584+0000 7fec91400700 1 -- 172.21.15.42:0/3072951126 learned_addr learned my addr 172.21.15.42:0/3072951126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91e00700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec941045e0 0x7fec9419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec92800700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec94109860 0x7fec9419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91400700 1 -- 172.21.15.42:0/3072951126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec94109860 msgr2=0x7fec9419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91400700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec94109860 0x7fec9419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91400700 1 -- 172.21.15.42:0/3072951126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec941045e0 msgr2=0x7fec9419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91400700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec941045e0 0x7fec9419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91400700 1 -- 172.21.15.42:0/3072951126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9419fc20 con 0x7fec94108f10 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91e00700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec941045e0 0x7fec9419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:14:50.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec91400700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 0x7fec9419b2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec8000d980 tx=0x7fec8000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80017070 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec80010460 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9419ff10 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.585+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8000de60 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.586+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941a7830 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.587+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec940512b0 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.590+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fec80010610 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.592+0000 7fec8aa00700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fec780b2880 0x7fec780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.592+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec800e1d60 con 0x7fec94108f10 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.592+0000 7fec91e00700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fec780b2880 0x7fec780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.593+0000 7fec91e00700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fec780b2880 0x7fec780b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec84009fd0 tx=0x7fec84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.593+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec800a9a50 con 0x7fec94108f10 2024-05-27T10:14:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.868+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fec9419c0b0 con 0x7fec780b2880 2024-05-27T10:14:50.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.940+0000 7fec8aa00700 1 -- 172.21.15.42:0/3072951126 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fec9419c0b0 con 0x7fec780b2880 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fec780b2880 msgr2=0x7fec780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fec780b2880 0x7fec780b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec84009fd0 tx=0x7fec84005fb0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 msgr2=0x7fec9419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 0x7fec9419b2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec8000d980 tx=0x7fec8000dd40 comp rx=0 tx=0).stop 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 shutdown_connections 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec94109860 0x7fec9419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fec780b2880 0x7fec780b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec941045e0 0x7fec9419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 --2- 172.21.15.42:0/3072951126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec94108f10 0x7fec9419b2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:50.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 >> 172.21.15.42:0/3072951126 conn(0x7fec94100030 msgr2=0x7fec94102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:50.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 shutdown_connections 2024-05-27T10:14:50.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:50.944+0000 7fec99200700 1 -- 172.21.15.42:0/3072951126 wait complete. 2024-05-27T10:14:51.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:14:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:14:50.877+0000 7f1905600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:14:51.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:14:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:14:50.877+0000 7f1905600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:14:51.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:14:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:14:50.877+0000 7f1905600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:14:51.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:14:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:14:50.877+0000 7f1905600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:14:51.345 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:14:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:14:50.877+0000 7f1905600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:14:51.855 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-27T10:14:51.940 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:51 smithi042 ceph-mon[31661]: pgmap v406: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:51.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:51 smithi104 ceph-mon[34812]: pgmap v406: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:52.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:51 smithi143 ceph-mon[35545]: pgmap v406: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:52.335 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:52.754 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:52 smithi042 ceph-mon[31661]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:14:53.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:52 smithi143 ceph-mon[35545]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:14:53.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:52 smithi104 ceph-mon[34812]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:14:53.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:53 smithi042 ceph-mon[31661]: pgmap v407: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 380 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:53.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:53 smithi042 ceph-mon[31661]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T10:14:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:53 smithi143 ceph-mon[35545]: pgmap v407: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 380 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:53 smithi143 ceph-mon[35545]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T10:14:54.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:53 smithi104 ceph-mon[34812]: pgmap v407: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 380 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:54.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:53 smithi104 ceph-mon[34812]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T10:14:54.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 -- 172.21.15.42:0/491846925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b4109860 msgr2=0x7fa7b41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:54.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 --2- 172.21.15.42:0/491846925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b4109860 0x7fa7b41111c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b0d0 tx=0x7fa7ac00b3e0 comp rx=0 tx=0).stop 2024-05-27T10:14:54.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 -- 172.21.15.42:0/491846925 shutdown_connections 2024-05-27T10:14:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 --2- 172.21.15.42:0/491846925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b4109860 0x7fa7b41111c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 --2- 172.21.15.42:0/491846925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b41045e0 0x7fa7b41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 --2- 172.21.15.42:0/491846925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4108f10 0x7fa7b4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 -- 172.21.15.42:0/491846925 >> 172.21.15.42:0/491846925 conn(0x7fa7b4100030 msgr2=0x7fa7b4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 -- 172.21.15.42:0/491846925 shutdown_connections 2024-05-27T10:14:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.856+0000 7fa7b9600700 1 -- 172.21.15.42:0/491846925 wait complete. 2024-05-27T10:14:54.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.857+0000 7fa7b9600700 1 Processor -- start 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.857+0000 7fa7b9600700 1 -- start start 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 0x7fa7b41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b9600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b4108f10 0x7fa7b41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b9600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4109860 0x7fa7b419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b9600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4114390 con 0x7fa7b41045e0 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b9600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4114210 con 0x7fa7b4109860 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b9600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4114510 con 0x7fa7b4108f10 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 0x7fa7b41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 0x7fa7b41a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36924/0 (socket says 172.21.15.42:36924) 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.858+0000 7fa7b2a00700 1 -- 172.21.15.42:0/4142824856 learned_addr learned my addr 172.21.15.42:0/4142824856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2000700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b4108f10 0x7fa7b41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:54.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b3400700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4109860 0x7fa7b419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:54.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2a00700 1 -- 172.21.15.42:0/4142824856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b4108f10 msgr2=0x7fa7b41a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:54.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2a00700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b4108f10 0x7fa7b41a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:54.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2a00700 1 -- 172.21.15.42:0/4142824856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4109860 msgr2=0x7fa7b419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:54.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2a00700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4109860 0x7fa7b419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:54.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2a00700 1 -- 172.21.15.42:0/4142824856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b419b140 con 0x7fa7b41045e0 2024-05-27T10:14:54.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b3400700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4109860 0x7fa7b419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b2a00700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 0x7fa7b41a0890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c9b0 tx=0x7fa7a400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4007b00 con 0x7fa7b41045e0 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.859+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b419b430 con 0x7fa7b41045e0 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.860+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7a4007ca0 con 0x7fa7b41045e0 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.860+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4018660 con 0x7fa7b41045e0 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.860+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a7740 con 0x7fa7b41045e0 2024-05-27T10:14:54.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.861+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b40512b0 con 0x7fa7b41045e0 2024-05-27T10:14:54.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.863+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa7a400f450 con 0x7fa7b41045e0 2024-05-27T10:14:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.865+0000 7fa7ab400700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa7980b2910 0x7fa7980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.865+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa7a40e24a0 con 0x7fa7b41045e0 2024-05-27T10:14:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.865+0000 7fa7b2000700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa7980b2910 0x7fa7980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.866+0000 7fa7b2000700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa7980b2910 0x7fa7980b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005fd0 tx=0x7fa7a000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:54.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:54.870+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7a4014030 con 0x7fa7b41045e0 2024-05-27T10:14:55.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.140+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa7b419bfc0 con 0x7fa7980b2910 2024-05-27T10:14:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.217+0000 7fa7ab400700 1 -- 172.21.15.42:0/4142824856 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa7b419bfc0 con 0x7fa7980b2910 2024-05-27T10:14:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.221+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa7980b2910 msgr2=0x7fa7980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.221+0000 7fa7b9600700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa7980b2910 0x7fa7980b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005fd0 tx=0x7fa7a000af20 comp rx=0 tx=0).stop 2024-05-27T10:14:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.221+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 msgr2=0x7fa7b41a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.221+0000 7fa7b9600700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 0x7fa7b41a0890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c9b0 tx=0x7fa7a400cd70 comp rx=0 tx=0).stop 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.221+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 shutdown_connections 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa7980b2910 0x7fa7980b4e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7b4109860 0x7fa7b419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b41045e0 0x7fa7b41a0890 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 --2- 172.21.15.42:0/4142824856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b4108f10 0x7fa7b41a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 >> 172.21.15.42:0/4142824856 conn(0x7fa7b4100030 msgr2=0x7fa7b4108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 shutdown_connections 2024-05-27T10:14:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:55.222+0000 7fa7b9600700 1 -- 172.21.15.42:0/4142824856 wait complete. 2024-05-27T10:14:56.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:55 smithi143 ceph-mon[35545]: pgmap v408: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:56.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:55 smithi042 ceph-mon[31661]: pgmap v408: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:56.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-27T10:14:56.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:55 smithi104 ceph-mon[34812]: pgmap v408: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:56.626 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:14:57.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:56 smithi042 ceph-mon[31661]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:14:57.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:56 smithi143 ceph-mon[35545]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:14:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:56 smithi104 ceph-mon[34812]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:14:58.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:57 smithi143 ceph-mon[35545]: pgmap v409: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:58.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:57 smithi042 ceph-mon[31661]: pgmap v409: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:58.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:57 smithi104 ceph-mon[34812]: pgmap v409: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:58.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:58 smithi042 ceph-mon[31661]: pgmap v410: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:59.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:58 smithi143 ceph-mon[35545]: pgmap v410: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:59.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 -- 172.21.15.42:0/1663562977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 msgr2=0x7f42bc1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:59.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 --2- 172.21.15.42:0/1663562977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc1049d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42b00099c0 tx=0x7f42b0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 -- 172.21.15.42:0/1663562977 shutdown_connections 2024-05-27T10:14:59.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 --2- 172.21.15.42:0/1663562977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc109840 0x7f42bc1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 --2- 172.21.15.42:0/1663562977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc1049d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 --2- 172.21.15.42:0/1663562977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc108ef0 0x7f42bc109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 -- 172.21.15.42:0/1663562977 >> 172.21.15.42:0/1663562977 conn(0x7f42bc0ffe90 msgr2=0x7f42bc1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:59.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 -- 172.21.15.42:0/1663562977 shutdown_connections 2024-05-27T10:14:59.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.190+0000 7f42c4000700 1 -- 172.21.15.42:0/1663562977 wait complete. 2024-05-27T10:14:59.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.192+0000 7f42c4000700 1 Processor -- start 2024-05-27T10:14:59.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.192+0000 7f42c4000700 1 -- start start 2024-05-27T10:14:59.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c4000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c4000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc108ef0 0x7f42bc077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c4000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc109840 0x7f42bc077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c4000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc114370 con 0x7f42bc1045c0 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c4000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1141f0 con 0x7f42bc108ef0 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c4000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1144f0 con 0x7f42bc109840 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc076bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36940/0 (socket says 172.21.15.42:36940) 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.193+0000 7f42c1600700 1 -- 172.21.15.42:0/3558411324 learned_addr learned my addr 172.21.15.42:0/3558411324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c0c00700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc108ef0 0x7f42bc077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c2000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc109840 0x7f42bc077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c1600700 1 -- 172.21.15.42:0/3558411324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc109840 msgr2=0x7f42bc077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:59.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c1600700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc109840 0x7f42bc077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c1600700 1 -- 172.21.15.42:0/3558411324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc108ef0 msgr2=0x7f42bc077100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:59.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c1600700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc108ef0 0x7f42bc077100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c1600700 1 -- 172.21.15.42:0/3558411324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1a9100 con 0x7f42bc1045c0 2024-05-27T10:14:59.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c2000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc109840 0x7f42bc077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:14:59.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.194+0000 7f42c1600700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc076bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006010 tx=0x7f42b00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:59.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.195+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0026070 con 0x7f42bc1045c0 2024-05-27T10:14:59.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.195+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42b000f460 con 0x7f42bc1045c0 2024-05-27T10:14:59.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.195+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0009e60 con 0x7f42bc1045c0 2024-05-27T10:14:59.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.195+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc1a9390 con 0x7f42bc1045c0 2024-05-27T10:14:59.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.195+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc1a9840 con 0x7f42bc1045c0 2024-05-27T10:14:59.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.196+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42bc0512b0 con 0x7f42bc1045c0 2024-05-27T10:14:59.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:58 smithi104 ceph-mon[34812]: pgmap v410: 97 pgs: 97 active+clean; 589 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-27T10:14:59.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.198+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f42b0029070 con 0x7f42bc1045c0 2024-05-27T10:14:59.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.200+0000 7f42ba000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42a40b2990 0x7f42a40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:14:59.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.200+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42b00e2c50 con 0x7f42bc1045c0 2024-05-27T10:14:59.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.200+0000 7f42c0c00700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42a40b2990 0x7f42a40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:14:59.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.201+0000 7f42c0c00700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42a40b2990 0x7f42a40b4ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42a8005950 tx=0x7f42a80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:14:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.205+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42b0013070 con 0x7f42bc1045c0 2024-05-27T10:14:59.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.481+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f42bc063bf0 con 0x7f42a40b2990 2024-05-27T10:14:59.501 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63 2024-05-27T10:14:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.501+0000 7f42ba000700 1 -- 172.21.15.42:0/3558411324 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f42bc063bf0 con 0x7f42a40b2990 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42a40b2990 msgr2=0x7f42a40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42a40b2990 0x7f42a40b4ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42a8005950 tx=0x7f42a80058e0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 msgr2=0x7f42bc076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc076bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006010 tx=0x7f42b00056a0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 shutdown_connections 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42bc109840 0x7f42bc077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42bc1045c0 0x7f42bc076bc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42a40b2990 0x7f42a40b4ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 --2- 172.21.15.42:0/3558411324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc108ef0 0x7f42bc077100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:14:59.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 >> 172.21.15.42:0/3558411324 conn(0x7f42bc0ffe90 msgr2=0x7f42bc1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:14:59.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.504+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 shutdown_connections 2024-05-27T10:14:59.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:14:59.505+0000 7f42c4000700 1 -- 172.21.15.42:0/3558411324 wait complete. 2024-05-27T10:15:00.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:14:59 smithi143 ceph-mon[35545]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:00.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:14:59 smithi042 ceph-mon[31661]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:00.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:14:59 smithi104 ceph-mon[34812]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:00.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-27T10:15:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:01.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:00 smithi042 ceph-mon[31661]: pgmap v411: 97 pgs: 97 active+clean; 603 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-27T10:15:01.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:00 smithi143 ceph-mon[35545]: pgmap v411: 97 pgs: 97 active+clean; 603 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-27T10:15:01.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:00 smithi104 ceph-mon[34812]: pgmap v411: 97 pgs: 97 active+clean; 603 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-27T10:15:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 -- 172.21.15.42:0/3502561915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80108550 msgr2=0x7f7f8010feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 --2- 172.21.15.42:0/3502561915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80108550 0x7f7f8010feb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b0d0 tx=0x7f7f7800b3e0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 -- 172.21.15.42:0/3502561915 shutdown_connections 2024-05-27T10:15:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 --2- 172.21.15.42:0/3502561915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80108550 0x7f7f8010feb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 --2- 172.21.15.42:0/3502561915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 --2- 172.21.15.42:0/3502561915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f80107c00 0x7f7f80108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 -- 172.21.15.42:0/3502561915 >> 172.21.15.42:0/3502561915 conn(0x7f7f80078ab0 msgr2=0x7f7f80078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 -- 172.21.15.42:0/3502561915 shutdown_connections 2024-05-27T10:15:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.317+0000 7f7f86800700 1 -- 172.21.15.42:0/3502561915 wait complete. 2024-05-27T10:15:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.319+0000 7f7f86800700 1 Processor -- start 2024-05-27T10:15:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.319+0000 7f7f86800700 1 -- start start 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f86800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f86800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80107c00 0x7f7f801a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f86800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f80108550 0x7f7f8019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f86800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801130a0 con 0x7f7f80107c00 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f86800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112f20 con 0x7f7f801032d0 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f86800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80113220 con 0x7f7f80108550 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f85400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f85400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:47738/0 (socket says 172.21.15.42:47738) 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f85400700 1 -- 172.21.15.42:0/2056318150 learned_addr learned my addr 172.21.15.42:0/2056318150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.320+0000 7f7f84a00700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80107c00 0x7f7f801a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f85e00700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f80108550 0x7f7f8019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:03.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f84a00700 1 -- 172.21.15.42:0/2056318150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f80108550 msgr2=0x7f7f8019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f84a00700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f80108550 0x7f7f8019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f84a00700 1 -- 172.21.15.42:0/2056318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 msgr2=0x7f7f801a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f84a00700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f84a00700 1 -- 172.21.15.42:0/2056318150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8019b020 con 0x7f7f80107c00 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f85400700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.321+0000 7f7f84a00700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80107c00 0x7f7f801a0c90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9c0 tx=0x7f7f6c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.322+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c004220 con 0x7f7f80107c00 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.322+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f6c0043c0 con 0x7f7f80107c00 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.322+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8019b310 con 0x7f7f80107c00 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.322+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00ce00 con 0x7f7f80107c00 2024-05-27T10:15:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.322+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801003f0 con 0x7f7f80107c00 2024-05-27T10:15:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.323+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f80104780 con 0x7f7f80107c00 2024-05-27T10:15:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.326+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7f6c004750 con 0x7f7f80107c00 2024-05-27T10:15:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.330+0000 7f7f7e000700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7f680b2990 0x7f7f680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.330+0000 7f7f85400700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7f680b2990 0x7f7f680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.330+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f6c0e2370 con 0x7f7f80107c00 2024-05-27T10:15:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.331+0000 7f7f85400700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7f680b2990 0x7f7f680b4ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f8019bab0 tx=0x7f7f74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:03.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.334+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f6c0a9f90 con 0x7f7f80107c00 2024-05-27T10:15:03.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:03 smithi042 ceph-mon[31661]: pgmap v412: 97 pgs: 97 active+clean; 603 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-27T10:15:03.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.607+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7f8019bea0 con 0x7f7f680b2990 2024-05-27T10:15:03.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.676+0000 7f7f7e000700 1 -- 172.21.15.42:0/2056318150 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f8019bea0 con 0x7f7f680b2990 2024-05-27T10:15:03.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7f680b2990 msgr2=0x7f7f680b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:03.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7f680b2990 0x7f7f680b4ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f8019bab0 tx=0x7f7f74009260 comp rx=0 tx=0).stop 2024-05-27T10:15:03.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80107c00 msgr2=0x7f7f801a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:03.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80107c00 0x7f7f801a0c90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9c0 tx=0x7f7f6c00ecd0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 shutdown_connections 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f80108550 0x7f7f8019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f7f680b2990 0x7f7f680b4ea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f801032d0 0x7f7f801a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 --2- 172.21.15.42:0/2056318150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f80107c00 0x7f7f801a0c90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 >> 172.21.15.42:0/2056318150 conn(0x7f7f80078ab0 msgr2=0x7f7f8006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 shutdown_connections 2024-05-27T10:15:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:03.679+0000 7f7f86800700 1 -- 172.21.15.42:0/2056318150 wait complete. 2024-05-27T10:15:03.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:03 smithi143 ceph-mon[35545]: pgmap v412: 97 pgs: 97 active+clean; 603 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-27T10:15:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:03 smithi104 ceph-mon[34812]: pgmap v412: 97 pgs: 97 active+clean; 603 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-27T10:15:04.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-27T10:15:04.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:04 smithi143 ceph-mon[35545]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:04.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:04 smithi042 ceph-mon[31661]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:04.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:04 smithi104 ceph-mon[34812]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:05.058 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:05.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:05 smithi143 ceph-mon[35545]: pgmap v413: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-05-27T10:15:05.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:05 smithi042 ceph-mon[31661]: pgmap v413: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-05-27T10:15:05.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:05 smithi104 ceph-mon[34812]: pgmap v413: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-05-27T10:15:07.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 -- 172.21.15.42:0/3577407231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd87410e8c0 msgr2=0x7fd874110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 --2- 172.21.15.42:0/3577407231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd87410e8c0 0x7fd874110da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009990 tx=0x7fd86c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 -- 172.21.15.42:0/3577407231 shutdown_connections 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 --2- 172.21.15.42:0/3577407231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd87410e8c0 0x7fd874110da0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 --2- 172.21.15.42:0/3577407231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd874105450 0x7fd874105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 --2- 172.21.15.42:0/3577407231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd874105da0 0x7fd87410e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 -- 172.21.15.42:0/3577407231 >> 172.21.15.42:0/3577407231 conn(0x7fd8740fedb0 msgr2=0x7fd8741011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 -- 172.21.15.42:0/3577407231 shutdown_connections 2024-05-27T10:15:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.520+0000 7fd87b400700 1 -- 172.21.15.42:0/3577407231 wait complete. 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.521+0000 7fd87b400700 1 Processor -- start 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.521+0000 7fd87b400700 1 -- start start 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd87b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 0x7fd87419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd87b400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd874105da0 0x7fd87419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd87b400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd87410e8c0 0x7fd8741a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd87b400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741142d0 con 0x7fd874105450 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd87b400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd874114150 con 0x7fd874105da0 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd87b400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd874114450 con 0x7fd87410e8c0 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd878a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 0x7fd87419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd878a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 0x7fd87419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44552/0 (socket says 172.21.15.42:44552) 2024-05-27T10:15:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.522+0000 7fd878a00700 1 -- 172.21.15.42:0/2819455992 learned_addr learned my addr 172.21.15.42:0/2819455992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd873e00700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd874105da0 0x7fd87419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd879400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd87410e8c0 0x7fd8741a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd878a00700 1 -- 172.21.15.42:0/2819455992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd87410e8c0 msgr2=0x7fd8741a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd878a00700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd87410e8c0 0x7fd8741a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd878a00700 1 -- 172.21.15.42:0/2819455992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd874105da0 msgr2=0x7fd87419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd878a00700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd874105da0 0x7fd87419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd878a00700 1 -- 172.21.15.42:0/2819455992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741a1d00 con 0x7fd874105450 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.523+0000 7fd878a00700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 0x7fd87419cee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd868007e20 tx=0x7fd86800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.524+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868016020 con 0x7fd874105450 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.524+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86800ae00 con 0x7fd874105450 2024-05-27T10:15:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.524+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868011670 con 0x7fd874105450 2024-05-27T10:15:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.524+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741a1ff0 con 0x7fd874105450 2024-05-27T10:15:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.524+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741a24e0 con 0x7fd874105450 2024-05-27T10:15:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.525+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8740512b0 con 0x7fd874105450 2024-05-27T10:15:07.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.528+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd868019020 con 0x7fd874105450 2024-05-27T10:15:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.529+0000 7fd871600700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd85c0b2990 0x7fd85c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.529+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd8680e23e0 con 0x7fd874105450 2024-05-27T10:15:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.530+0000 7fd873e00700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd85c0b2990 0x7fd85c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:07.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.530+0000 7fd873e00700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd85c0b2990 0x7fd85c0b4ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd860005950 tx=0x7fd8600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.532+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8680aa080 con 0x7fd874105450 2024-05-27T10:15:07.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.805+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd874063bf0 con 0x7fd85c0b2990 2024-05-27T10:15:07.806 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:07 smithi042 ceph-mon[31661]: pgmap v414: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-05-27T10:15:07.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:07 smithi143 ceph-mon[35545]: pgmap v414: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-05-27T10:15:07.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.812+0000 7fd871600700 1 -- 172.21.15.42:0/2819455992 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd874063bf0 con 0x7fd85c0b2990 2024-05-27T10:15:07.812 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_1/e5e61405-4131-4d26-b6c1-edbb0a5a7783 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd85c0b2990 msgr2=0x7fd85c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd85c0b2990 0x7fd85c0b4ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd860005950 tx=0x7fd8600058e0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 msgr2=0x7fd87419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 0x7fd87419cee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd868007e20 tx=0x7fd86800a340 comp rx=0 tx=0).stop 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 shutdown_connections 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd87410e8c0 0x7fd8741a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd874105450 0x7fd87419cee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd85c0b2990 0x7fd85c0b4ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 --2- 172.21.15.42:0/2819455992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd874105da0 0x7fd87419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 >> 172.21.15.42:0/2819455992 conn(0x7fd8740fedb0 msgr2=0x7fd874103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 shutdown_connections 2024-05-27T10:15:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:07.815+0000 7fd87b400700 1 -- 172.21.15.42:0/2819455992 wait complete. 2024-05-27T10:15:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:07 smithi104 ceph-mon[34812]: pgmap v414: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-05-27T10:15:08.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:08 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:15:08.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:15:08.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:08 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:15:08.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:08 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:15:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:09.805 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:09 smithi042 ceph-mon[31661]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:09.805 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:09 smithi042 ceph-mon[31661]: pgmap v415: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-27T10:15:09.805 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.806 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.806 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.806 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:09 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:09 smithi143 ceph-mon[35545]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:09 smithi143 ceph-mon[35545]: pgmap v415: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-27T10:15:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:09 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:09 smithi104 ceph-mon[34812]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:15:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:09 smithi104 ceph-mon[34812]: pgmap v415: 97 pgs: 97 active+clean; 622 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-27T10:15:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:09 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:11.650 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:11 smithi042 ceph-mon[31661]: pgmap v416: 97 pgs: 97 active+clean; 633 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-27T10:15:11.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:11 smithi143 ceph-mon[35545]: pgmap v416: 97 pgs: 97 active+clean; 633 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-27T10:15:11.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:11 smithi104 ceph-mon[34812]: pgmap v416: 97 pgs: 97 active+clean; 633 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-27T10:15:12.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.270+0000 7f0633000700 1 -- 172.21.15.42:0/696690431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c1045e0 msgr2=0x7f062c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:12.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.270+0000 7f0633000700 1 --2- 172.21.15.42:0/696690431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c1045e0 0x7f062c1049f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f061c0099c0 tx=0x7f061c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 -- 172.21.15.42:0/696690431 shutdown_connections 2024-05-27T10:15:12.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 --2- 172.21.15.42:0/696690431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c109860 0x7f062c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 --2- 172.21.15.42:0/696690431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c1045e0 0x7f062c1049f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 --2- 172.21.15.42:0/696690431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f062c108f10 0x7f062c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 -- 172.21.15.42:0/696690431 >> 172.21.15.42:0/696690431 conn(0x7f062c100030 msgr2=0x7f062c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:12.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 -- 172.21.15.42:0/696690431 shutdown_connections 2024-05-27T10:15:12.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.271+0000 7f0633000700 1 -- 172.21.15.42:0/696690431 wait complete. 2024-05-27T10:15:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.272+0000 7f0633000700 1 Processor -- start 2024-05-27T10:15:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 -- start start 2024-05-27T10:15:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c1045e0 0x7f062c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 0x7f062c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f062c109860 0x7f062c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f062c114390 con 0x7f062c108f10 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f062c114210 con 0x7f062c109860 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f0633000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f062c114510 con 0x7f062c1045e0 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.273+0000 7f062b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 0x7f062c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 0x7f062c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44576/0 (socket says 172.21.15.42:44576) 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 -- 172.21.15.42:0/2211689852 learned_addr learned my addr 172.21.15.42:0/2211689852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062be00700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c1045e0 0x7f062c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f0631000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f062c109860 0x7f062c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 -- 172.21.15.42:0/2211689852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c1045e0 msgr2=0x7f062c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c1045e0 0x7f062c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 -- 172.21.15.42:0/2211689852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f062c109860 msgr2=0x7f062c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f062c109860 0x7f062c1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.274+0000 7f062b400700 1 -- 172.21.15.42:0/2211689852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c1a1ef0 con 0x7f062c108f10 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.275+0000 7f062b400700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 0x7f062c19d5c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f061800d980 tx=0x7f061800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:12.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.275+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618017070 con 0x7f062c108f10 2024-05-27T10:15:12.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.275+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c1a21e0 con 0x7f062c108f10 2024-05-27T10:15:12.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.275+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f061800fd20 con 0x7f062c108f10 2024-05-27T10:15:12.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.275+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061800e3b0 con 0x7f062c108f10 2024-05-27T10:15:12.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.275+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c1a9b00 con 0x7f062c108f10 2024-05-27T10:15:12.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.276+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c0512b0 con 0x7f062c108f10 2024-05-27T10:15:12.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.280+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f061800f890 con 0x7f062c108f10 2024-05-27T10:15:12.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.282+0000 7f0628c00700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f06100b2990 0x7f06100b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:12.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.282+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f06180e1d90 con 0x7f062c108f10 2024-05-27T10:15:12.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.282+0000 7f062be00700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f06100b2990 0x7f06100b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:12.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.283+0000 7f062be00700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f06100b2990 0x7f06100b4ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f061c000c00 tx=0x7f061c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:12.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.284+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06180a99a0 con 0x7f062c108f10 2024-05-27T10:15:12.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.597+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f062c1a9cd0 con 0x7f062c108f10 2024-05-27T10:15:12.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.598+0000 7f0628c00700 1 -- 172.21.15.42:0/2211689852 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f06180a92a0 con 0x7f062c108f10 2024-05-27T10:15:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:12.598 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:14:45.975282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f06100b2990 msgr2=0x7f06100b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f06100b2990 0x7f06100b4ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f061c000c00 tx=0x7f061c01a040 comp rx=0 tx=0).stop 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 msgr2=0x7f062c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 0x7f062c19d5c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f061800d980 tx=0x7f061800dc90 comp rx=0 tx=0).stop 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 shutdown_connections 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f06100b2990 0x7f06100b4ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f062c109860 0x7f062c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c1045e0 0x7f062c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 --2- 172.21.15.42:0/2211689852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f062c108f10 0x7f062c19d5c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.601+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 >> 172.21.15.42:0/2211689852 conn(0x7f062c100030 msgr2=0x7f062c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.602+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 shutdown_connections 2024-05-27T10:15:12.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:12.602+0000 7f0633000700 1 -- 172.21.15.42:0/2211689852 wait complete. 2024-05-27T10:15:12.603 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:15:13.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:12 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:12 smithi143 ceph-mon[35545]: pgmap v417: 97 pgs: 97 active+clean; 633 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-27T10:15:13.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:12 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2211689852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:15:13.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:12 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:12 smithi042 ceph-mon[31661]: pgmap v417: 97 pgs: 97 active+clean; 633 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-27T10:15:13.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:12 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2211689852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:15:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:12 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:12 smithi104 ceph-mon[34812]: pgmap v417: 97 pgs: 97 active+clean; 633 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-27T10:15:13.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:12 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2211689852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:15:13.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:15:14.011 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:14.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:14.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:13 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:14.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:13 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:14.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:14.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:13 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:15:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:15:15.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:15.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:15 smithi143 ceph-mon[35545]: pgmap v418: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-27T10:15:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:15:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:15:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:15.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:15 smithi042 ceph-mon[31661]: pgmap v418: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-27T10:15:15.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:15:15.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:15:15.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:15:15.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:15 smithi104 ceph-mon[34812]: pgmap v418: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-27T10:15:16.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 -- 172.21.15.42:0/2316271160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 msgr2=0x7fc6e0106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 --2- 172.21.15.42:0/2316271160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e0106a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00099c0 tx=0x7fc6d0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 -- 172.21.15.42:0/2316271160 shutdown_connections 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 --2- 172.21.15.42:0/2316271160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6e010b890 0x7fc6e01131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 --2- 172.21.15.42:0/2316271160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e0106a20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 --2- 172.21.15.42:0/2316271160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6e010af40 0x7fc6e010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 -- 172.21.15.42:0/2316271160 >> 172.21.15.42:0/2316271160 conn(0x7fc6e0102020 msgr2=0x7fc6e0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 -- 172.21.15.42:0/2316271160 shutdown_connections 2024-05-27T10:15:16.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.488+0000 7fc6df200700 1 -- 172.21.15.42:0/2316271160 wait complete. 2024-05-27T10:15:16.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.490+0000 7fc6df200700 1 Processor -- start 2024-05-27T10:15:16.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.490+0000 7fc6df200700 1 -- start start 2024-05-27T10:15:16.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6df200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6df200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6e010af40 0x7fc6e019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6df200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6e010b890 0x7fc6e01a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6df200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01163c0 con 0x7fc6e0106610 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6df200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0116240 con 0x7fc6e010b890 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6df200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0116540 con 0x7fc6e010af40 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43888/0 (socket says 172.21.15.42:43888) 2024-05-27T10:15:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 -- 172.21.15.42:0/1212015837 learned_addr learned my addr 172.21.15.42:0/1212015837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dd400700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6e010af40 0x7fc6e019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6de800700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6e010b890 0x7fc6e01a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 -- 172.21.15.42:0/1212015837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6e010af40 msgr2=0x7fc6e019d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6e010af40 0x7fc6e019d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 -- 172.21.15.42:0/1212015837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6e010b890 msgr2=0x7fc6e01a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6e010b890 0x7fc6e01a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.491+0000 7fc6dde00700 1 -- 172.21.15.42:0/1212015837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e01a1ca0 con 0x7fc6e0106610 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.492+0000 7fc6dde00700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e019cdd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0005910 tx=0x7fc6d0004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.492+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0026070 con 0x7fc6e0106610 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.492+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6d000f460 con 0x7fc6e0106610 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.492+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0009e60 con 0x7fc6e0106610 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.492+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e01a1f30 con 0x7fc6e0106610 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.492+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e01a98b0 con 0x7fc6e0106610 2024-05-27T10:15:16.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.494+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e00512b0 con 0x7fc6e0106610 2024-05-27T10:15:16.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.496+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc6d0029070 con 0x7fc6e0106610 2024-05-27T10:15:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.498+0000 7fc6d6a00700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6c40b2990 0x7fc6c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.498+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc6d00e2ed0 con 0x7fc6e0106610 2024-05-27T10:15:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.498+0000 7fc6dd400700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6c40b2990 0x7fc6c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.499+0000 7fc6dd400700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6c40b2990 0x7fc6c40b4ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0109880 tx=0x7fc6cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:16.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.501+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6d005edf0 con 0x7fc6e0106610 2024-05-27T10:15:16.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.738+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6e002e5e0 con 0x7fc6e0106610 2024-05-27T10:15:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.740+0000 7fc6d6a00700 1 -- 172.21.15.42:0/1212015837 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22562 (secure 0 0 0) 0x7fc6d0013070 con 0x7fc6e0106610 2024-05-27T10:15:16.740 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:16.740 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:41.618756+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:15:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.743+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6c40b2990 msgr2=0x7fc6c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.743+0000 7fc6df200700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6c40b2990 0x7fc6c40b4ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0109880 tx=0x7fc6cc009260 comp rx=0 tx=0).stop 2024-05-27T10:15:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.743+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 msgr2=0x7fc6e019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e019cdd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0005910 tx=0x7fc6d0004690 comp rx=0 tx=0).stop 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 shutdown_connections 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc6c40b2990 0x7fc6c40b4ea0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6e010b890 0x7fc6e01a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e0106610 0x7fc6e019cdd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 --2- 172.21.15.42:0/1212015837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6e010af40 0x7fc6e019d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 >> 172.21.15.42:0/1212015837 conn(0x7fc6e0102020 msgr2=0x7fc6e010a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 shutdown_connections 2024-05-27T10:15:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:16.744+0000 7fc6df200700 1 -- 172.21.15.42:0/1212015837 wait complete. 2024-05-27T10:15:17.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:17 smithi042 ceph-mon[31661]: pgmap v419: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-27T10:15:17.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:17 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1212015837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:17.724 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T10:15:17.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:15:17.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:17 smithi143 ceph-mon[35545]: pgmap v419: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-27T10:15:17.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:17 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1212015837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:17.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:17 smithi104 ceph-mon[34812]: pgmap v419: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-27T10:15:17.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:17 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1212015837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:18.181 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:19.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:19 smithi042 ceph-mon[31661]: pgmap v420: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-27T10:15:19.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:19 smithi143 ceph-mon[35545]: pgmap v420: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-27T10:15:19.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:19 smithi104 ceph-mon[34812]: pgmap v420: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-27T10:15:20.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 -- 172.21.15.42:0/2547483241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 msgr2=0x7f87141038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:20.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 --2- 172.21.15.42:0/2547483241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87141038c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f87040099c0 tx=0x7f8704009cd0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 -- 172.21.15.42:0/2547483241 shutdown_connections 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 --2- 172.21.15.42:0/2547483241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 0x7f8714110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 --2- 172.21.15.42:0/2547483241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87141038c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 --2- 172.21.15.42:0/2547483241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8714107de0 0x7f87141081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 -- 172.21.15.42:0/2547483241 >> 172.21.15.42:0/2547483241 conn(0x7f87140fede0 msgr2=0x7f8714101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 -- 172.21.15.42:0/2547483241 shutdown_connections 2024-05-27T10:15:20.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.559+0000 7f8719e00700 1 -- 172.21.15.42:0/2547483241 wait complete. 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.560+0000 7f8719e00700 1 Processor -- start 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 -- start start 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87140ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8714107de0 0x7f87140ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 0x7f87140ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8714113300 con 0x7f87141034b0 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8714113180 con 0x7f8714108730 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8719e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8714113480 con 0x7f8714107de0 2024-05-27T10:15:20.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.561+0000 7f8712a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87140ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:20.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87140ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43904/0 (socket says 172.21.15.42:43904) 2024-05-27T10:15:20.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 -- 172.21.15.42:0/3320437632 learned_addr learned my addr 172.21.15.42:0/3320437632 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f870be00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8714107de0 0x7f87140ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8713400700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 0x7f87140ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 -- 172.21.15.42:0/3320437632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8714107de0 msgr2=0x7f87140ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8714107de0 0x7f87140ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 -- 172.21.15.42:0/3320437632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 msgr2=0x7f87140ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 0x7f87140ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8712a00700 1 -- 172.21.15.42:0/3320437632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87141a9180 con 0x7f87141034b0 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.562+0000 7f8713400700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 0x7f87140ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.563+0000 7f8712a00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87140ff2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8704000c00 tx=0x7f870400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.563+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704026070 con 0x7f87141034b0 2024-05-27T10:15:20.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.563+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87040054c0 con 0x7f87141034b0 2024-05-27T10:15:20.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.563+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87141a9410 con 0x7f87141034b0 2024-05-27T10:15:20.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.563+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004c20 con 0x7f87141034b0 2024-05-27T10:15:20.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.563+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87141a98c0 con 0x7f87141034b0 2024-05-27T10:15:20.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.564+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87140512b0 con 0x7f87141034b0 2024-05-27T10:15:20.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.567+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8704029070 con 0x7f87141034b0 2024-05-27T10:15:20.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.568+0000 7f8718a00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f86f80b2990 0x7f86f80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:20.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.568+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f87040e32f0 con 0x7f87141034b0 2024-05-27T10:15:20.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.568+0000 7f870be00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f86f80b2990 0x7f86f80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:20.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.569+0000 7f870be00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f86f80b2990 0x7f86f80b4ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86fc006fd0 tx=0x7f86fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:20.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.571+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8704030080 con 0x7f87141034b0 2024-05-27T10:15:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.800+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8714063bf0 con 0x7f86f80b2990 2024-05-27T10:15:20.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.829+0000 7f8718a00700 1 -- 172.21.15.42:0/3320437632 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258929 (secure 0 0 0) 0x7f8714063bf0 con 0x7f86f80b2990 2024-05-27T10:15:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f86f80b2990 msgr2=0x7f86f80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f86f80b2990 0x7f86f80b4ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86fc006fd0 tx=0x7f86fc008040 comp rx=0 tx=0).stop 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 msgr2=0x7f87140ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87140ff2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8704000c00 tx=0x7f870400f740 comp rx=0 tx=0).stop 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 shutdown_connections 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f86f80b2990 0x7f86f80b4ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8714108730 0x7f87140ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141034b0 0x7f87140ff2c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 --2- 172.21.15.42:0/3320437632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8714107de0 0x7f87140ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.834+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 >> 172.21.15.42:0/3320437632 conn(0x7f87140fede0 msgr2=0x7f8714106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.835+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 shutdown_connections 2024-05-27T10:15:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:20.835+0000 7f8719e00700 1 -- 172.21.15.42:0/3320437632 wait complete. 2024-05-27T10:15:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:15:21.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:21 smithi042 ceph-mon[31661]: pgmap v421: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-27T10:15:21.779 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-05-27T10:15:20.418690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653650,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":333,"num_read_kb":263,"num_write":291,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359928,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581896,"statfs":{"total":1151940427776,"available":1151571861504,"internally_reserved":0,"allocated":11010048,"data_stored":7090998,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16455,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":58,"num_write":10,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005504"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639743+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:14:41.639743+0000","last_peered":"2024-05-27T10:14:41.639743+0000","last_clean":"2024-05-27T10:14:41.639743+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:14:41.639743+0000","last_undegraded":"2024-05-27T10:14:41.639743+0000","last_fullsized":"2024-05-27T10:14:41.639743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455418+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:14:42.455418+0000","last_peered":"2024-05-27T10:14:42.455418+0000","last_clean":"2024-05-27T10:14:42.455418+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:14:42.455418+0000","last_undegraded":"2024-05-27T10:14:42.455418+0000","last_fullsized":"2024-05-27T10:14:42.455418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454454+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:14:42.454454+0000","last_peered":"2024-05-27T10:14:42.454454+0000","last_clean":"2024-05-27T10:14:42.454454+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:14:42.454454+0000","last_undegraded":"2024-05-27T10:14:42.454454+0000","last_fullsized":"2024-05-27T10:14:42.454454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.656369+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:14:41.656369+0000","last_peered":"2024-05-27T10:14:41.656369+0000","last_clean":"2024-05-27T10:14:41.656369+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:14:41.656369+0000","last_undegraded":"2024-05-27T10:14:41.656369+0000","last_fullsized":"2024-05-27T10:14:41.656369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643072+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:14:41.643072+0000","last_peered":"2024-05-27T10:14:41.643072+0000","last_clean":"2024-05-27T10:14:41.643072+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:14:41.643072+0000","last_undegraded":"2024-05-27T10:14:41.643072+0000","last_fullsized":"2024-05-27T10:14:41.643072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.651541+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:14:41.651541+0000","last_peered":"2024-05-27T10:14:41.651541+0000","last_clean":"2024-05-27T10:14:41.651541+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:14:41.651541+0000","last_undegraded":"2024-05-27T10:14:41.651541+0000","last_fullsized":"2024-05-27T10:14:41.651541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642785+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:14:41.642785+0000","last_peered":"2024-05-27T10:14:41.642785+0000","last_clean":"2024-05-27T10:14:41.642785+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:14:41.642785+0000","last_undegraded":"2024-05-27T10:14:41.642785+0000","last_fullsized":"2024-05-27T10:14:41.642785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644594+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:14:41.644594+0000","last_peered":"2024-05-27T10:14:41.644594+0000","last_clean":"2024-05-27T10:14:41.644594+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:14:41.644594+0000","last_undegraded":"2024-05-27T10:14:41.644594+0000","last_fullsized":"2024-05-27T10:14:41.644594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.055248+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:14:42.055248+0000","last_peered":"2024-05-27T10:14:42.055248+0000","last_clean":"2024-05-27T10:14:42.055248+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:14:42.055248+0000","last_undegraded":"2024-05-27T10:14:42.055248+0000","last_fullsized":"2024-05-27T10:14:42.055248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.641192+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:14:41.641192+0000","last_peered":"2024-05-27T10:14:41.641192+0000","last_clean":"2024-05-27T10:14:41.641192+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:14:41.641192+0000","last_undegraded":"2024-05-27T10:14:41.641192+0000","last_fullsized":"2024-05-27T10:14:41.641192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642828+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:14:41.642828+0000","last_peered":"2024-05-27T10:14:41.642828+0000","last_clean":"2024-05-27T10:14:41.642828+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:14:41.642828+0000","last_undegraded":"2024-05-27T10:14:41.642828+0000","last_fullsized":"2024-05-27T10:14:41.642828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652571+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:14:41.652571+0000","last_peered":"2024-05-27T10:14:41.652571+0000","last_clean":"2024-05-27T10:14:41.652571+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:14:41.652571+0000","last_undegraded":"2024-05-27T10:14:41.652571+0000","last_fullsized":"2024-05-27T10:14:41.652571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637240+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:14:41.637240+0000","last_peered":"2024-05-27T10:14:41.637240+0000","last_clean":"2024-05-27T10:14:41.637240+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:14:41.637240+0000","last_undegraded":"2024-05-27T10:14:41.637240+0000","last_fullsized":"2024-05-27T10:14:41.637240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454308+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:14:42.454308+0000","last_peered":"2024-05-27T10:14:42.454308+0000","last_clean":"2024-05-27T10:14:42.454308+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:14:42.454308+0000","last_undegraded":"2024-05-27T10:14:42.454308+0000","last_fullsized":"2024-05-27T10:14:42.454308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454292+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:14:42.454292+0000","last_peered":"2024-05-27T10:14:42.454292+0000","last_clean":"2024-05-27T10:14:42.454292+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:14:42.454292+0000","last_undegraded":"2024-05-27T10:14:42.454292+0000","last_fullsized":"2024-05-27T10:14:42.454292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053684+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:14:42.053684+0000","last_peered":"2024-05-27T10:14:42.053684+0000","last_clean":"2024-05-27T10:14:42.053684+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:14:42.053684+0000","last_undegraded":"2024-05-27T10:14:42.053684+0000","last_fullsized":"2024-05-27T10:14:42.053684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640169+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:14:41.640169+0000","last_peered":"2024-05-27T10:14:41.640169+0000","last_clean":"2024-05-27T10:14:41.640169+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:14:41.640169+0000","last_undegraded":"2024-05-27T10:14:41.640169+0000","last_fullsized":"2024-05-27T10:14:41.640169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.647542+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:14:41.647542+0000","last_peered":"2024-05-27T10:14:41.647542+0000","last_clean":"2024-05-27T10:14:41.647542+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:14:41.647542+0000","last_undegraded":"2024-05-27T10:14:41.647542+0000","last_fullsized":"2024-05-27T10:14:41.647542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644789+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:14:41.644789+0000","last_peered":"2024-05-27T10:14:41.644789+0000","last_clean":"2024-05-27T10:14:41.644789+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:14:41.644789+0000","last_undegraded":"2024-05-27T10:14:41.644789+0000","last_fullsized":"2024-05-27T10:14:41.644789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.649244+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:14:41.649244+0000","last_peered":"2024-05-27T10:14:41.649244+0000","last_clean":"2024-05-27T10:14:41.649244+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:14:41.649244+0000","last_undegraded":"2024-05-27T10:14:41.649244+0000","last_fullsized":"2024-05-27T10:14:41.649244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646972+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:14:41.646972+0000","last_peered":"2024-05-27T10:14:41.646972+0000","last_clean":"2024-05-27T10:14:41.646972+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:14:41.646972+0000","last_undegraded":"2024-05-27T10:14:41.646972+0000","last_fullsized":"2024-05-27T10:14:41.646972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648077+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:14:41.648077+0000","last_peered":"2024-05-27T10:14:41.648077+0000","last_clean":"2024-05-27T10:14:41.648077+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:14:41.648077+0000","last_undegraded":"2024-05-27T10:14:41.648077+0000","last_fullsized":"2024-05-27T10:14:41.648077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455634+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:14:42.455634+0000","last_peered":"2024-05-27T10:14:42.455634+0000","last_clean":"2024-05-27T10:14:42.455634+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:14:42.455634+0000","last_undegraded":"2024-05-27T10:14:42.455634+0000","last_fullsized":"2024-05-27T10:14:42.455634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644487+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:14:41.644487+0000","last_peered":"2024-05-27T10:14:41.644487+0000","last_clean":"2024-05-27T10:14:41.644487+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:14:41.644487+0000","last_undegraded":"2024-05-27T10:14:41.644487+0000","last_fullsized":"2024-05-27T10:14:41.644487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053677+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:14:42.053677+0000","last_peered":"2024-05-27T10:14:42.053677+0000","last_clean":"2024-05-27T10:14:42.053677+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:14:42.053677+0000","last_undegraded":"2024-05-27T10:14:42.053677+0000","last_fullsized":"2024-05-27T10:14:42.053677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643944+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:14:41.643944+0000","last_peered":"2024-05-27T10:14:41.643944+0000","last_clean":"2024-05-27T10:14:41.643944+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:14:41.643944+0000","last_undegraded":"2024-05-27T10:14:41.643944+0000","last_fullsized":"2024-05-27T10:14:41.643944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653394+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:14:41.653394+0000","last_peered":"2024-05-27T10:14:41.653394+0000","last_clean":"2024-05-27T10:14:41.653394+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:14:41.653394+0000","last_undegraded":"2024-05-27T10:14:41.653394+0000","last_fullsized":"2024-05-27T10:14:41.653394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:16.770874+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:15:16.770874+0000","last_peered":"2024-05-27T10:15:16.770874+0000","last_clean":"2024-05-27T10:15:16.770874+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:15:16.770874+0000","last_undegraded":"2024-05-27T10:15:16.770874+0000","last_fullsized":"2024-05-27T10:15:16.770874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643280+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:14:41.643280+0000","last_peered":"2024-05-27T10:14:41.643280+0000","last_clean":"2024-05-27T10:14:41.643280+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:14:41.643280+0000","last_undegraded":"2024-05-27T10:14:41.643280+0000","last_fullsized":"2024-05-27T10:14:41.643280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640164+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:14:41.640164+0000","last_peered":"2024-05-27T10:14:41.640164+0000","last_clean":"2024-05-27T10:14:41.640164+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:14:41.640164+0000","last_undegraded":"2024-05-27T10:14:41.640164+0000","last_fullsized":"2024-05-27T10:14:41.640164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652993+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:14:41.652993+0000","last_peered":"2024-05-27T10:14:41.652993+0000","last_clean":"2024-05-27T10:14:41.652993+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:14:41.652993+0000","last_undegraded":"2024-05-27T10:14:41.652993+0000","last_fullsized":"2024-05-27T10:14:41.652993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636434+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:14:41.636434+0000","last_peered":"2024-05-27T10:14:41.636434+0000","last_clean":"2024-05-27T10:14:41.636434+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:14:41.636434+0000","last_undegraded":"2024-05-27T10:14:41.636434+0000","last_fullsized":"2024-05-27T10:14:41.636434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644566+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:14:41.644566+0000","last_peered":"2024-05-27T10:14:41.644566+0000","last_clean":"2024-05-27T10:14:41.644566+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:14:41.644566+0000","last_undegraded":"2024-05-27T10:14:41.644566+0000","last_fullsized":"2024-05-27T10:14:41.644566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652467+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:14:41.652467+0000","last_peered":"2024-05-27T10:14:41.652467+0000","last_clean":"2024-05-27T10:14:41.652467+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:14:41.652467+0000","last_undegraded":"2024-05-27T10:14:41.652467+0000","last_fullsized":"2024-05-27T10:14:41.652467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.647637+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:14:41.647637+0000","last_peered":"2024-05-27T10:14:41.647637+0000","last_clean":"2024-05-27T10:14:41.647637+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:14:41.647637+0000","last_undegraded":"2024-05-27T10:14:41.647637+0000","last_fullsized":"2024-05-27T10:14:41.647637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:17.772624+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:15:17.772624+0000","last_peered":"2024-05-27T10:15:17.772624+0000","last_clean":"2024-05-27T10:15:17.772624+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:15:17.772624+0000","last_undegraded":"2024-05-27T10:15:17.772624+0000","last_fullsized":"2024-05-27T10:15:17.772624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.054888+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:14:42.054888+0000","last_peered":"2024-05-27T10:14:42.054888+0000","last_clean":"2024-05-27T10:14:42.054888+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:14:42.054888+0000","last_undegraded":"2024-05-27T10:14:42.054888+0000","last_fullsized":"2024-05-27T10:14:42.054888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642885+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:14:41.642885+0000","last_peered":"2024-05-27T10:14:41.642885+0000","last_clean":"2024-05-27T10:14:41.642885+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:14:41.642885+0000","last_undegraded":"2024-05-27T10:14:41.642885+0000","last_fullsized":"2024-05-27T10:14:41.642885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455826+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:14:42.455826+0000","last_peered":"2024-05-27T10:14:42.455826+0000","last_clean":"2024-05-27T10:14:42.455826+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:14:42.455826+0000","last_undegraded":"2024-05-27T10:14:42.455826+0000","last_fullsized":"2024-05-27T10:14:42.455826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653643+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:14:41.653643+0000","last_peered":"2024-05-27T10:14:41.653643+0000","last_clean":"2024-05-27T10:14:41.653643+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:14:41.653643+0000","last_undegraded":"2024-05-27T10:14:41.653643+0000","last_fullsized":"2024-05-27T10:14:41.653643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053851+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:14:42.053851+0000","last_peered":"2024-05-27T10:14:42.053851+0000","last_clean":"2024-05-27T10:14:42.053851+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:14:42.053851+0000","last_undegraded":"2024-05-27T10:14:42.053851+0000","last_fullsized":"2024-05-27T10:14:42.053851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:18.783232+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:15:18.783232+0000","last_peered":"2024-05-27T10:15:18.783232+0000","last_clean":"2024-05-27T10:15:18.783232+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:15:18.783232+0000","last_undegraded":"2024-05-27T10:15:18.783232+0000","last_fullsized":"2024-05-27T10:15:18.783232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58656,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":85,"num_read_kb":99,"num_write":29,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643929+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:14:41.643929+0000","last_peered":"2024-05-27T10:14:41.643929+0000","last_clean":"2024-05-27T10:14:41.643929+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:14:41.643929+0000","last_undegraded":"2024-05-27T10:14:41.643929+0000","last_fullsized":"2024-05-27T10:14:41.643929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644588+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:14:41.644588+0000","last_peered":"2024-05-27T10:14:41.644588+0000","last_clean":"2024-05-27T10:14:41.644588+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:14:41.644588+0000","last_undegraded":"2024-05-27T10:14:41.644588+0000","last_fullsized":"2024-05-27T10:14:41.644588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636683+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:14:41.636683+0000","last_peered":"2024-05-27T10:14:41.636683+0000","last_clean":"2024-05-27T10:14:41.636683+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:14:41.636683+0000","last_undegraded":"2024-05-27T10:14:41.636683+0000","last_fullsized":"2024-05-27T10:14:41.636683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652771+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:14:41.652771+0000","last_peered":"2024-05-27T10:14:41.652771+0000","last_clean":"2024-05-27T10:14:41.652771+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:14:41.652771+0000","last_undegraded":"2024-05-27T10:14:41.652771+0000","last_fullsized":"2024-05-27T10:14:41.652771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642835+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:14:41.642835+0000","last_peered":"2024-05-27T10:14:41.642835+0000","last_clean":"2024-05-27T10:14:41.642835+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:14:41.642835+0000","last_undegraded":"2024-05-27T10:14:41.642835+0000","last_fullsized":"2024-05-27T10:14:41.642835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:14.765486+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:15:14.765486+0000","last_peered":"2024-05-27T10:15:14.765486+0000","last_clean":"2024-05-27T10:15:14.765486+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:15:14.765486+0000","last_undegraded":"2024-05-27T10:15:14.765486+0000","last_fullsized":"2024-05-27T10:15:14.765486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.654514+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:14:41.654514+0000","last_peered":"2024-05-27T10:14:41.654514+0000","last_clean":"2024-05-27T10:14:41.654514+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:14:41.654514+0000","last_undegraded":"2024-05-27T10:14:41.654514+0000","last_fullsized":"2024-05-27T10:14:41.654514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642435+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:14:41.642435+0000","last_peered":"2024-05-27T10:14:41.642435+0000","last_clean":"2024-05-27T10:14:41.642435+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:14:41.642435+0000","last_undegraded":"2024-05-27T10:14:41.642435+0000","last_fullsized":"2024-05-27T10:14:41.642435+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653301+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:14:41.653301+0000","last_peered":"2024-05-27T10:14:41.653301+0000","last_clean":"2024-05-27T10:14:41.653301+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:14:41.653301+0000","last_undegraded":"2024-05-27T10:14:41.653301+0000","last_fullsized":"2024-05-27T10:14:41.653301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646830+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:14:41.646830+0000","last_peered":"2024-05-27T10:14:41.646830+0000","last_clean":"2024-05-27T10:14:41.646830+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:14:41.646830+0000","last_undegraded":"2024-05-27T10:14:41.646830+0000","last_fullsized":"2024-05-27T10:14:41.646830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639338+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:14:41.639338+0000","last_peered":"2024-05-27T10:14:41.639338+0000","last_clean":"2024-05-27T10:14:41.639338+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:14:41.639338+0000","last_undegraded":"2024-05-27T10:14:41.639338+0000","last_fullsized":"2024-05-27T10:14:41.639338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646777+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:14:41.646777+0000","last_peered":"2024-05-27T10:14:41.646777+0000","last_clean":"2024-05-27T10:14:41.646777+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:14:41.646777+0000","last_undegraded":"2024-05-27T10:14:41.646777+0000","last_fullsized":"2024-05-27T10:14:41.646777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.641212+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:14:41.641212+0000","last_peered":"2024-05-27T10:14:41.641212+0000","last_clean":"2024-05-27T10:14:41.641212+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:14:41.641212+0000","last_undegraded":"2024-05-27T10:14:41.641212+0000","last_fullsized":"2024-05-27T10:14:41.641212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642492+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:14:41.642492+0000","last_peered":"2024-05-27T10:14:41.642492+0000","last_clean":"2024-05-27T10:14:41.642492+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:14:41.642492+0000","last_undegraded":"2024-05-27T10:14:41.642492+0000","last_fullsized":"2024-05-27T10:14:41.642492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640332+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:14:41.640332+0000","last_peered":"2024-05-27T10:14:41.640332+0000","last_clean":"2024-05-27T10:14:41.640332+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:14:41.640332+0000","last_undegraded":"2024-05-27T10:14:41.640332+0000","last_fullsized":"2024-05-27T10:14:41.640332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652987+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:14:41.652987+0000","last_peered":"2024-05-27T10:14:41.652987+0000","last_clean":"2024-05-27T10:14:41.652987+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:14:41.652987+0000","last_undegraded":"2024-05-27T10:14:41.652987+0000","last_fullsized":"2024-05-27T10:14:41.652987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:16.769539+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:15:16.769539+0000","last_peered":"2024-05-27T10:15:16.769539+0000","last_clean":"2024-05-27T10:15:16.769539+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:15:16.769539+0000","last_undegraded":"2024-05-27T10:15:16.769539+0000","last_fullsized":"2024-05-27T10:15:16.769539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645152+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:14:41.645152+0000","last_peered":"2024-05-27T10:14:41.645152+0000","last_clean":"2024-05-27T10:14:41.645152+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:14:41.645152+0000","last_undegraded":"2024-05-27T10:14:41.645152+0000","last_fullsized":"2024-05-27T10:14:41.645152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.654480+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:14:41.654480+0000","last_peered":"2024-05-27T10:14:41.654480+0000","last_clean":"2024-05-27T10:14:41.654480+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:14:41.654480+0000","last_undegraded":"2024-05-27T10:14:41.654480+0000","last_fullsized":"2024-05-27T10:14:41.654480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":99,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643011+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:14:41.643011+0000","last_peered":"2024-05-27T10:14:41.643011+0000","last_clean":"2024-05-27T10:14:41.643011+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:14:41.643011+0000","last_undegraded":"2024-05-27T10:14:41.643011+0000","last_fullsized":"2024-05-27T10:14:41.643011+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636152+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:14:41.636152+0000","last_peered":"2024-05-27T10:14:41.636152+0000","last_clean":"2024-05-27T10:14:41.636152+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:14:41.636152+0000","last_undegraded":"2024-05-27T10:14:41.636152+0000","last_fullsized":"2024-05-27T10:14:41.636152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643273+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:14:41.643273+0000","last_peered":"2024-05-27T10:14:41.643273+0000","last_clean":"2024-05-27T10:14:41.643273+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:14:41.643273+0000","last_undegraded":"2024-05-27T10:14:41.643273+0000","last_fullsized":"2024-05-27T10:14:41.643273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644150+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:14:41.644150+0000","last_peered":"2024-05-27T10:14:41.644150+0000","last_clean":"2024-05-27T10:14:41.644150+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:14:41.644150+0000","last_undegraded":"2024-05-27T10:14:41.644150+0000","last_fullsized":"2024-05-27T10:14:41.644150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637007+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:14:41.637007+0000","last_peered":"2024-05-27T10:14:41.637007+0000","last_clean":"2024-05-27T10:14:41.637007+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:14:41.637007+0000","last_undegraded":"2024-05-27T10:14:41.637007+0000","last_fullsized":"2024-05-27T10:14:41.637007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:09.209320+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:15:09.209320+0000","last_peered":"2024-05-27T10:15:09.209320+0000","last_clean":"2024-05-27T10:15:09.209320+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:15:09.209320+0000","last_undegraded":"2024-05-27T10:15:09.209320+0000","last_fullsized":"2024-05-27T10:15:09.209320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646942+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:14:41.646942+0000","last_peered":"2024-05-27T10:14:41.646942+0000","last_clean":"2024-05-27T10:14:41.646942+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:14:41.646942+0000","last_undegraded":"2024-05-27T10:14:41.646942+0000","last_fullsized":"2024-05-27T10:14:41.646942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652545+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:14:41.652545+0000","last_peered":"2024-05-27T10:14:41.652545+0000","last_clean":"2024-05-27T10:14:41.652545+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:14:41.652545+0000","last_undegraded":"2024-05-27T10:14:41.652545+0000","last_fullsized":"2024-05-27T10:14:41.652545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455434+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:14:42.455434+0000","last_peered":"2024-05-27T10:14:42.455434+0000","last_clean":"2024-05-27T10:14:42.455434+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:14:42.455434+0000","last_undegraded":"2024-05-27T10:14:42.455434+0000","last_fullsized":"2024-05-27T10:14:42.455434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053163+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:14:42.053163+0000","last_peered":"2024-05-27T10:14:42.053163+0000","last_clean":"2024-05-27T10:14:42.053163+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:14:42.053163+0000","last_undegraded":"2024-05-27T10:14:42.053163+0000","last_fullsized":"2024-05-27T10:14:42.053163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642586+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:14:41.642586+0000","last_peered":"2024-05-27T10:14:41.642586+0000","last_clean":"2024-05-27T10:14:41.642586+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:14:41.642586+0000","last_undegraded":"2024-05-27T10:14:41.642586+0000","last_fullsized":"2024-05-27T10:14:41.642586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645409+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:14:41.645409+0000","last_peered":"2024-05-27T10:14:41.645409+0000","last_clean":"2024-05-27T10:14:41.645409+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:14:41.645409+0000","last_undegraded":"2024-05-27T10:14:41.645409+0000","last_fullsized":"2024-05-27T10:14:41.645409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.638987+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:14:41.638987+0000","last_peered":"2024-05-27T10:14:41.638987+0000","last_clean":"2024-05-27T10:14:41.638987+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:14:41.638987+0000","last_undegraded":"2024-05-27T10:14:41.638987+0000","last_fullsized":"2024-05-27T10:14:41.638987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.659057+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:14:41.659057+0000","last_peered":"2024-05-27T10:14:41.659057+0000","last_clean":"2024-05-27T10:14:41.659057+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:14:41.659057+0000","last_undegraded":"2024-05-27T10:14:41.659057+0000","last_fullsized":"2024-05-27T10:14:41.659057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646308+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:14:41.646308+0000","last_peered":"2024-05-27T10:14:41.646308+0000","last_clean":"2024-05-27T10:14:41.646308+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:14:41.646308+0000","last_undegraded":"2024-05-27T10:14:41.646308+0000","last_fullsized":"2024-05-27T10:14:41.646308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.663013+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:14:41.663013+0000","last_peered":"2024-05-27T10:14:41.663013+0000","last_clean":"2024-05-27T10:14:41.663013+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:14:41.663013+0000","last_undegraded":"2024-05-27T10:14:41.663013+0000","last_fullsized":"2024-05-27T10:14:41.663013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642553+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:14:41.642553+0000","last_peered":"2024-05-27T10:14:41.642553+0000","last_clean":"2024-05-27T10:14:41.642553+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:14:41.642553+0000","last_undegraded":"2024-05-27T10:14:41.642553+0000","last_fullsized":"2024-05-27T10:14:41.642553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644620+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:14:41.644620+0000","last_peered":"2024-05-27T10:14:41.644620+0000","last_clean":"2024-05-27T10:14:41.644620+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:14:41.644620+0000","last_undegraded":"2024-05-27T10:14:41.644620+0000","last_fullsized":"2024-05-27T10:14:41.644620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642991+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:14:41.642991+0000","last_peered":"2024-05-27T10:14:41.642991+0000","last_clean":"2024-05-27T10:14:41.642991+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:14:41.642991+0000","last_undegraded":"2024-05-27T10:14:41.642991+0000","last_fullsized":"2024-05-27T10:14:41.642991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653579+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:14:41.653579+0000","last_peered":"2024-05-27T10:14:41.653579+0000","last_clean":"2024-05-27T10:14:41.653579+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:14:41.653579+0000","last_undegraded":"2024-05-27T10:14:41.653579+0000","last_fullsized":"2024-05-27T10:14:41.653579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642782+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:14:41.642782+0000","last_peered":"2024-05-27T10:14:41.642782+0000","last_clean":"2024-05-27T10:14:41.642782+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:14:41.642782+0000","last_undegraded":"2024-05-27T10:14:41.642782+0000","last_fullsized":"2024-05-27T10:14:41.642782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637216+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:14:41.637216+0000","last_peered":"2024-05-27T10:14:41.637216+0000","last_clean":"2024-05-27T10:14:41.637216+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:14:41.637216+0000","last_undegraded":"2024-05-27T10:14:41.637216+0000","last_fullsized":"2024-05-27T10:14:41.637216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643381+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:14:41.643381+0000","last_peered":"2024-05-27T10:14:41.643381+0000","last_clean":"2024-05-27T10:14:41.643381+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:14:41.643381+0000","last_undegraded":"2024-05-27T10:14:41.643381+0000","last_fullsized":"2024-05-27T10:14:41.643381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639128+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:14:41.639128+0000","last_peered":"2024-05-27T10:14:41.639128+0000","last_clean":"2024-05-27T10:14:41.639128+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:14:41.639128+0000","last_undegraded":"2024-05-27T10:14:41.639128+0000","last_fullsized":"2024-05-27T10:14:41.639128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643457+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:14:41.643457+0000","last_peered":"2024-05-27T10:14:41.643457+0000","last_clean":"2024-05-27T10:14:41.643457+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:14:41.643457+0000","last_undegraded":"2024-05-27T10:14:41.643457+0000","last_fullsized":"2024-05-27T10:14:41.643457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643352+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:14:41.643352+0000","last_peered":"2024-05-27T10:14:41.643352+0000","last_clean":"2024-05-27T10:14:41.643352+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:14:41.643352+0000","last_undegraded":"2024-05-27T10:14:41.643352+0000","last_fullsized":"2024-05-27T10:14:41.643352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454848+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:14:42.454848+0000","last_peered":"2024-05-27T10:14:42.454848+0000","last_clean":"2024-05-27T10:14:42.454848+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:14:42.454848+0000","last_undegraded":"2024-05-27T10:14:42.454848+0000","last_fullsized":"2024-05-27T10:14:42.454848+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637191+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:14:41.637191+0000","last_peered":"2024-05-27T10:14:41.637191+0000","last_clean":"2024-05-27T10:14:41.637191+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:14:41.637191+0000","last_undegraded":"2024-05-27T10:14:41.637191+0000","last_fullsized":"2024-05-27T10:14:41.637191+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640158+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:14:41.640158+0000","last_peered":"2024-05-27T10:14:41.640158+0000","last_clean":"2024-05-27T10:14:41.640158+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:14:41.640158+0000","last_undegraded":"2024-05-27T10:14:41.640158+0000","last_fullsized":"2024-05-27T10:14:41.640158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645293+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:14:41.645293+0000","last_peered":"2024-05-27T10:14:41.645293+0000","last_clean":"2024-05-27T10:14:41.645293+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:14:41.645293+0000","last_undegraded":"2024-05-27T10:14:41.645293+0000","last_fullsized":"2024-05-27T10:14:41.645293+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.657723+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:14:41.657723+0000","last_peered":"2024-05-27T10:14:41.657723+0000","last_clean":"2024-05-27T10:14:41.657723+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:14:41.657723+0000","last_undegraded":"2024-05-27T10:14:41.657723+0000","last_fullsized":"2024-05-27T10:14:41.657723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652483+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:14:41.652483+0000","last_peered":"2024-05-27T10:14:41.652483+0000","last_clean":"2024-05-27T10:14:41.652483+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:14:41.652483+0000","last_undegraded":"2024-05-27T10:14:41.652483+0000","last_fullsized":"2024-05-27T10:14:41.652483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645178+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:14:41.645178+0000","last_peered":"2024-05-27T10:14:41.645178+0000","last_clean":"2024-05-27T10:14:41.645178+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:14:41.645178+0000","last_undegraded":"2024-05-27T10:14:41.645178+0000","last_fullsized":"2024-05-27T10:14:41.645178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053963+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:14:42.053963+0000","last_peered":"2024-05-27T10:14:42.053963+0000","last_clean":"2024-05-27T10:14:42.053963+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:14:42.053963+0000","last_undegraded":"2024-05-27T10:14:42.053963+0000","last_fullsized":"2024-05-27T10:14:42.053963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637018+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:14:41.637018+0000","last_peered":"2024-05-27T10:14:41.637018+0000","last_clean":"2024-05-27T10:14:41.637018+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:14:41.637018+0000","last_undegraded":"2024-05-27T10:14:41.637018+0000","last_fullsized":"2024-05-27T10:14:41.637018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639344+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:14:41.639344+0000","last_peered":"2024-05-27T10:14:41.639344+0000","last_clean":"2024-05-27T10:14:41.639344+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:14:41.639344+0000","last_undegraded":"2024-05-27T10:14:41.639344+0000","last_fullsized":"2024-05-27T10:14:41.639344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63048,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":252,"num_read_kb":194,"num_write":166,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678048,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":428086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874280,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1078614,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037807,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":428322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":428242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":1020563,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593798,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":427711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":486160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":428982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":487901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":428022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953558,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":428539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790030,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":1019856,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:15:21.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:15:21.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:21 smithi143 ceph-mon[35545]: pgmap v421: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-27T10:15:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:21 smithi104 ceph-mon[34812]: pgmap v421: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-27T10:15:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:22.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:22 smithi042 ceph-mon[31661]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:15:22.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:22 smithi143 ceph-mon[35545]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:15:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:22 smithi104 ceph-mon[34812]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:15:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:23 smithi143 ceph-mon[35545]: pgmap v422: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:15:23.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:23 smithi042 ceph-mon[31661]: pgmap v422: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:15:23.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:23 smithi104 ceph-mon[34812]: pgmap v422: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:15:24.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 -- 172.21.15.42:0/2507932989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f010b2d0 msgr2=0x7fa0f0117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/2507932989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f010b2d0 0x7fa0f0117800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b0a0 tx=0x7fa0e800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 -- 172.21.15.42:0/2507932989 shutdown_connections 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/2507932989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f010b2d0 0x7fa0f0117800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/2507932989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0f0110960 0x7fa0f0110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/2507932989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f010a980 0x7fa0f010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 -- 172.21.15.42:0/2507932989 >> 172.21.15.42:0/2507932989 conn(0x7fa0f0077c70 msgr2=0x7fa0f00780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 -- 172.21.15.42:0/2507932989 shutdown_connections 2024-05-27T10:15:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.869+0000 7fa0f4a00700 1 -- 172.21.15.42:0/2507932989 wait complete. 2024-05-27T10:15:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.870+0000 7fa0f4a00700 1 Processor -- start 2024-05-27T10:15:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 -- start start 2024-05-27T10:15:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0f010a980 0x7fa0f01a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f010b2d0 0x7fa0f01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 0x7fa0f01a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f011a930 con 0x7fa0f0110960 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f011a7b0 con 0x7fa0f010a980 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0f4a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f011aab0 con 0x7fa0f010b2d0 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0ee800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 0x7fa0f01a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0ee800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 0x7fa0f01a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46054/0 (socket says 172.21.15.42:46054) 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.871+0000 7fa0ee800700 1 -- 172.21.15.42:0/1852887323 learned_addr learned my addr 172.21.15.42:0/1852887323 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ede00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0f010a980 0x7fa0f01a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ed400700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f010b2d0 0x7fa0f01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ee800700 1 -- 172.21.15.42:0/1852887323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f010b2d0 msgr2=0x7fa0f01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ee800700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f010b2d0 0x7fa0f01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ee800700 1 -- 172.21.15.42:0/1852887323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0f010a980 msgr2=0x7fa0f01a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ee800700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0f010a980 0x7fa0f01a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ee800700 1 -- 172.21.15.42:0/1852887323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f01a61e0 con 0x7fa0f0110960 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.872+0000 7fa0ee800700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 0x7fa0f01a5bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80062a0 tx=0x7fa0e8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.873+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8021070 con 0x7fa0f0110960 2024-05-27T10:15:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.873+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e8007d00 con 0x7fa0f0110960 2024-05-27T10:15:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.873+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8005280 con 0x7fa0f0110960 2024-05-27T10:15:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.873+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f01a6470 con 0x7fa0f0110960 2024-05-27T10:15:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.873+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f01a6900 con 0x7fa0f0110960 2024-05-27T10:15:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.874+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f00512b0 con 0x7fa0f0110960 2024-05-27T10:15:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.877+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa0e8024070 con 0x7fa0f0110960 2024-05-27T10:15:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.879+0000 7fa0e6a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa0d40b2990 0x7fa0d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.879+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa0e800f020 con 0x7fa0f0110960 2024-05-27T10:15:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.879+0000 7fa0ede00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa0d40b2990 0x7fa0d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.880+0000 7fa0ede00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa0d40b2990 0x7fa0d40b4ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005fd0 tx=0x7fa0e000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:24.882+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0e802b020 con 0x7fa0f0110960 2024-05-27T10:15:25.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.119+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0f01a2720 con 0x7fa0d40b2990 2024-05-27T10:15:25.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.143+0000 7fa0e6a00700 1 -- 172.21.15.42:0/1852887323 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258867 (secure 0 0 0) 0x7fa0f01a2720 con 0x7fa0d40b2990 2024-05-27T10:15:25.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:25.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa0d40b2990 msgr2=0x7fa0d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa0d40b2990 0x7fa0d40b4ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005fd0 tx=0x7fa0e000af20 comp rx=0 tx=0).stop 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 msgr2=0x7fa0f01a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 0x7fa0f01a5bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80062a0 tx=0x7fa0e8004ed0 comp rx=0 tx=0).stop 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 shutdown_connections 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f0110960 0x7fa0f01a5bf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa0d40b2990 0x7fa0d40b4ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:25.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0f010a980 0x7fa0f01a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:25.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 --2- 172.21.15.42:0/1852887323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f010b2d0 0x7fa0f01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:25.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.149+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 >> 172.21.15.42:0/1852887323 conn(0x7fa0f0077c70 msgr2=0x7fa0f007b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:25.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.150+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 shutdown_connections 2024-05-27T10:15:25.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:25.150+0000 7fa0f4a00700 1 -- 172.21.15.42:0/1852887323 wait complete. 2024-05-27T10:15:25.157 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:15:25.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:25 smithi143 ceph-mon[35545]: pgmap v423: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-27T10:15:25.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:25 smithi042 ceph-mon[31661]: pgmap v423: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-27T10:15:25.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:25 smithi104 ceph-mon[34812]: pgmap v423: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-27T10:15:26.117 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-27T10:15:24.420772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653650,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":363,"num_read_kb":278,"num_write":291,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359928,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581896,"statfs":{"total":1151940427776,"available":1151571861504,"internally_reserved":0,"allocated":11010048,"data_stored":7090998,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5395,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":52,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005663"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639743+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:14:41.639743+0000","last_peered":"2024-05-27T10:14:41.639743+0000","last_clean":"2024-05-27T10:14:41.639743+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:14:41.639743+0000","last_undegraded":"2024-05-27T10:14:41.639743+0000","last_fullsized":"2024-05-27T10:14:41.639743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455418+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:14:42.455418+0000","last_peered":"2024-05-27T10:14:42.455418+0000","last_clean":"2024-05-27T10:14:42.455418+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:14:42.455418+0000","last_undegraded":"2024-05-27T10:14:42.455418+0000","last_fullsized":"2024-05-27T10:14:42.455418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454454+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:14:42.454454+0000","last_peered":"2024-05-27T10:14:42.454454+0000","last_clean":"2024-05-27T10:14:42.454454+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:14:42.454454+0000","last_undegraded":"2024-05-27T10:14:42.454454+0000","last_fullsized":"2024-05-27T10:14:42.454454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.656369+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:14:41.656369+0000","last_peered":"2024-05-27T10:14:41.656369+0000","last_clean":"2024-05-27T10:14:41.656369+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:14:41.656369+0000","last_undegraded":"2024-05-27T10:14:41.656369+0000","last_fullsized":"2024-05-27T10:14:41.656369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643072+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:14:41.643072+0000","last_peered":"2024-05-27T10:14:41.643072+0000","last_clean":"2024-05-27T10:14:41.643072+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:14:41.643072+0000","last_undegraded":"2024-05-27T10:14:41.643072+0000","last_fullsized":"2024-05-27T10:14:41.643072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.651541+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:14:41.651541+0000","last_peered":"2024-05-27T10:14:41.651541+0000","last_clean":"2024-05-27T10:14:41.651541+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:14:41.651541+0000","last_undegraded":"2024-05-27T10:14:41.651541+0000","last_fullsized":"2024-05-27T10:14:41.651541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642785+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:14:41.642785+0000","last_peered":"2024-05-27T10:14:41.642785+0000","last_clean":"2024-05-27T10:14:41.642785+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:14:41.642785+0000","last_undegraded":"2024-05-27T10:14:41.642785+0000","last_fullsized":"2024-05-27T10:14:41.642785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644594+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:14:41.644594+0000","last_peered":"2024-05-27T10:14:41.644594+0000","last_clean":"2024-05-27T10:14:41.644594+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:14:41.644594+0000","last_undegraded":"2024-05-27T10:14:41.644594+0000","last_fullsized":"2024-05-27T10:14:41.644594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.055248+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:14:42.055248+0000","last_peered":"2024-05-27T10:14:42.055248+0000","last_clean":"2024-05-27T10:14:42.055248+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:14:42.055248+0000","last_undegraded":"2024-05-27T10:14:42.055248+0000","last_fullsized":"2024-05-27T10:14:42.055248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.641192+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:14:41.641192+0000","last_peered":"2024-05-27T10:14:41.641192+0000","last_clean":"2024-05-27T10:14:41.641192+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:14:41.641192+0000","last_undegraded":"2024-05-27T10:14:41.641192+0000","last_fullsized":"2024-05-27T10:14:41.641192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642828+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:14:41.642828+0000","last_peered":"2024-05-27T10:14:41.642828+0000","last_clean":"2024-05-27T10:14:41.642828+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:14:41.642828+0000","last_undegraded":"2024-05-27T10:14:41.642828+0000","last_fullsized":"2024-05-27T10:14:41.642828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652571+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:14:41.652571+0000","last_peered":"2024-05-27T10:14:41.652571+0000","last_clean":"2024-05-27T10:14:41.652571+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:14:41.652571+0000","last_undegraded":"2024-05-27T10:14:41.652571+0000","last_fullsized":"2024-05-27T10:14:41.652571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637240+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:14:41.637240+0000","last_peered":"2024-05-27T10:14:41.637240+0000","last_clean":"2024-05-27T10:14:41.637240+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:14:41.637240+0000","last_undegraded":"2024-05-27T10:14:41.637240+0000","last_fullsized":"2024-05-27T10:14:41.637240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454308+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:14:42.454308+0000","last_peered":"2024-05-27T10:14:42.454308+0000","last_clean":"2024-05-27T10:14:42.454308+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:14:42.454308+0000","last_undegraded":"2024-05-27T10:14:42.454308+0000","last_fullsized":"2024-05-27T10:14:42.454308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454292+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:14:42.454292+0000","last_peered":"2024-05-27T10:14:42.454292+0000","last_clean":"2024-05-27T10:14:42.454292+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:14:42.454292+0000","last_undegraded":"2024-05-27T10:14:42.454292+0000","last_fullsized":"2024-05-27T10:14:42.454292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053684+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:14:42.053684+0000","last_peered":"2024-05-27T10:14:42.053684+0000","last_clean":"2024-05-27T10:14:42.053684+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:14:42.053684+0000","last_undegraded":"2024-05-27T10:14:42.053684+0000","last_fullsized":"2024-05-27T10:14:42.053684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640169+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:14:41.640169+0000","last_peered":"2024-05-27T10:14:41.640169+0000","last_clean":"2024-05-27T10:14:41.640169+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:14:41.640169+0000","last_undegraded":"2024-05-27T10:14:41.640169+0000","last_fullsized":"2024-05-27T10:14:41.640169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.647542+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:14:41.647542+0000","last_peered":"2024-05-27T10:14:41.647542+0000","last_clean":"2024-05-27T10:14:41.647542+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:14:41.647542+0000","last_undegraded":"2024-05-27T10:14:41.647542+0000","last_fullsized":"2024-05-27T10:14:41.647542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644789+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:14:41.644789+0000","last_peered":"2024-05-27T10:14:41.644789+0000","last_clean":"2024-05-27T10:14:41.644789+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:14:41.644789+0000","last_undegraded":"2024-05-27T10:14:41.644789+0000","last_fullsized":"2024-05-27T10:14:41.644789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.649244+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:14:41.649244+0000","last_peered":"2024-05-27T10:14:41.649244+0000","last_clean":"2024-05-27T10:14:41.649244+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:14:41.649244+0000","last_undegraded":"2024-05-27T10:14:41.649244+0000","last_fullsized":"2024-05-27T10:14:41.649244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646972+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:14:41.646972+0000","last_peered":"2024-05-27T10:14:41.646972+0000","last_clean":"2024-05-27T10:14:41.646972+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:14:41.646972+0000","last_undegraded":"2024-05-27T10:14:41.646972+0000","last_fullsized":"2024-05-27T10:14:41.646972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648077+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:14:41.648077+0000","last_peered":"2024-05-27T10:14:41.648077+0000","last_clean":"2024-05-27T10:14:41.648077+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:14:41.648077+0000","last_undegraded":"2024-05-27T10:14:41.648077+0000","last_fullsized":"2024-05-27T10:14:41.648077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455634+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:14:42.455634+0000","last_peered":"2024-05-27T10:14:42.455634+0000","last_clean":"2024-05-27T10:14:42.455634+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:14:42.455634+0000","last_undegraded":"2024-05-27T10:14:42.455634+0000","last_fullsized":"2024-05-27T10:14:42.455634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644487+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:14:41.644487+0000","last_peered":"2024-05-27T10:14:41.644487+0000","last_clean":"2024-05-27T10:14:41.644487+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:14:41.644487+0000","last_undegraded":"2024-05-27T10:14:41.644487+0000","last_fullsized":"2024-05-27T10:14:41.644487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053677+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:14:42.053677+0000","last_peered":"2024-05-27T10:14:42.053677+0000","last_clean":"2024-05-27T10:14:42.053677+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:14:42.053677+0000","last_undegraded":"2024-05-27T10:14:42.053677+0000","last_fullsized":"2024-05-27T10:14:42.053677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643944+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:14:41.643944+0000","last_peered":"2024-05-27T10:14:41.643944+0000","last_clean":"2024-05-27T10:14:41.643944+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:14:41.643944+0000","last_undegraded":"2024-05-27T10:14:41.643944+0000","last_fullsized":"2024-05-27T10:14:41.643944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653394+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:14:41.653394+0000","last_peered":"2024-05-27T10:14:41.653394+0000","last_clean":"2024-05-27T10:14:41.653394+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:14:41.653394+0000","last_undegraded":"2024-05-27T10:14:41.653394+0000","last_fullsized":"2024-05-27T10:14:41.653394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:21.786391+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:15:21.786391+0000","last_peered":"2024-05-27T10:15:21.786391+0000","last_clean":"2024-05-27T10:15:21.786391+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:15:21.786391+0000","last_undegraded":"2024-05-27T10:15:21.786391+0000","last_fullsized":"2024-05-27T10:15:21.786391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643280+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:14:41.643280+0000","last_peered":"2024-05-27T10:14:41.643280+0000","last_clean":"2024-05-27T10:14:41.643280+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:14:41.643280+0000","last_undegraded":"2024-05-27T10:14:41.643280+0000","last_fullsized":"2024-05-27T10:14:41.643280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640164+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:14:41.640164+0000","last_peered":"2024-05-27T10:14:41.640164+0000","last_clean":"2024-05-27T10:14:41.640164+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:14:41.640164+0000","last_undegraded":"2024-05-27T10:14:41.640164+0000","last_fullsized":"2024-05-27T10:14:41.640164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652993+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:14:41.652993+0000","last_peered":"2024-05-27T10:14:41.652993+0000","last_clean":"2024-05-27T10:14:41.652993+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:14:41.652993+0000","last_undegraded":"2024-05-27T10:14:41.652993+0000","last_fullsized":"2024-05-27T10:14:41.652993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636434+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:14:41.636434+0000","last_peered":"2024-05-27T10:14:41.636434+0000","last_clean":"2024-05-27T10:14:41.636434+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:14:41.636434+0000","last_undegraded":"2024-05-27T10:14:41.636434+0000","last_fullsized":"2024-05-27T10:14:41.636434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644566+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:14:41.644566+0000","last_peered":"2024-05-27T10:14:41.644566+0000","last_clean":"2024-05-27T10:14:41.644566+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:14:41.644566+0000","last_undegraded":"2024-05-27T10:14:41.644566+0000","last_fullsized":"2024-05-27T10:14:41.644566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652467+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:14:41.652467+0000","last_peered":"2024-05-27T10:14:41.652467+0000","last_clean":"2024-05-27T10:14:41.652467+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:14:41.652467+0000","last_undegraded":"2024-05-27T10:14:41.652467+0000","last_fullsized":"2024-05-27T10:14:41.652467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.647637+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:14:41.647637+0000","last_peered":"2024-05-27T10:14:41.647637+0000","last_clean":"2024-05-27T10:14:41.647637+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:14:41.647637+0000","last_undegraded":"2024-05-27T10:14:41.647637+0000","last_fullsized":"2024-05-27T10:14:41.647637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:22.788221+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:15:22.788221+0000","last_peered":"2024-05-27T10:15:22.788221+0000","last_clean":"2024-05-27T10:15:22.788221+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:15:22.788221+0000","last_undegraded":"2024-05-27T10:15:22.788221+0000","last_fullsized":"2024-05-27T10:15:22.788221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.054888+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:14:42.054888+0000","last_peered":"2024-05-27T10:14:42.054888+0000","last_clean":"2024-05-27T10:14:42.054888+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:14:42.054888+0000","last_undegraded":"2024-05-27T10:14:42.054888+0000","last_fullsized":"2024-05-27T10:14:42.054888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642885+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:14:41.642885+0000","last_peered":"2024-05-27T10:14:41.642885+0000","last_clean":"2024-05-27T10:14:41.642885+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:14:41.642885+0000","last_undegraded":"2024-05-27T10:14:41.642885+0000","last_fullsized":"2024-05-27T10:14:41.642885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455826+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:14:42.455826+0000","last_peered":"2024-05-27T10:14:42.455826+0000","last_clean":"2024-05-27T10:14:42.455826+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:14:42.455826+0000","last_undegraded":"2024-05-27T10:14:42.455826+0000","last_fullsized":"2024-05-27T10:14:42.455826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653643+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:14:41.653643+0000","last_peered":"2024-05-27T10:14:41.653643+0000","last_clean":"2024-05-27T10:14:41.653643+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:14:41.653643+0000","last_undegraded":"2024-05-27T10:14:41.653643+0000","last_fullsized":"2024-05-27T10:14:41.653643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053851+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:14:42.053851+0000","last_peered":"2024-05-27T10:14:42.053851+0000","last_clean":"2024-05-27T10:14:42.053851+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:14:42.053851+0000","last_undegraded":"2024-05-27T10:14:42.053851+0000","last_fullsized":"2024-05-27T10:14:42.053851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":149,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:23.797767+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:15:23.797767+0000","last_peered":"2024-05-27T10:15:23.797767+0000","last_clean":"2024-05-27T10:15:23.797767+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:15:23.797767+0000","last_undegraded":"2024-05-27T10:15:23.797767+0000","last_fullsized":"2024-05-27T10:15:23.797767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58656,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":95,"num_read_kb":104,"num_write":29,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643929+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:14:41.643929+0000","last_peered":"2024-05-27T10:14:41.643929+0000","last_clean":"2024-05-27T10:14:41.643929+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:14:41.643929+0000","last_undegraded":"2024-05-27T10:14:41.643929+0000","last_fullsized":"2024-05-27T10:14:41.643929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644588+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:14:41.644588+0000","last_peered":"2024-05-27T10:14:41.644588+0000","last_clean":"2024-05-27T10:14:41.644588+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:14:41.644588+0000","last_undegraded":"2024-05-27T10:14:41.644588+0000","last_fullsized":"2024-05-27T10:14:41.644588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636683+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:14:41.636683+0000","last_peered":"2024-05-27T10:14:41.636683+0000","last_clean":"2024-05-27T10:14:41.636683+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:14:41.636683+0000","last_undegraded":"2024-05-27T10:14:41.636683+0000","last_fullsized":"2024-05-27T10:14:41.636683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652771+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:14:41.652771+0000","last_peered":"2024-05-27T10:14:41.652771+0000","last_clean":"2024-05-27T10:14:41.652771+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:14:41.652771+0000","last_undegraded":"2024-05-27T10:14:41.652771+0000","last_fullsized":"2024-05-27T10:14:41.652771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642835+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:14:41.642835+0000","last_peered":"2024-05-27T10:14:41.642835+0000","last_clean":"2024-05-27T10:14:41.642835+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:14:41.642835+0000","last_undegraded":"2024-05-27T10:14:41.642835+0000","last_fullsized":"2024-05-27T10:14:41.642835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":73,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:19.779095+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:15:19.779095+0000","last_peered":"2024-05-27T10:15:19.779095+0000","last_clean":"2024-05-27T10:15:19.779095+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:15:19.779095+0000","last_undegraded":"2024-05-27T10:15:19.779095+0000","last_fullsized":"2024-05-27T10:15:19.779095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.654514+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:14:41.654514+0000","last_peered":"2024-05-27T10:14:41.654514+0000","last_clean":"2024-05-27T10:14:41.654514+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:14:41.654514+0000","last_undegraded":"2024-05-27T10:14:41.654514+0000","last_fullsized":"2024-05-27T10:14:41.654514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642435+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:14:41.642435+0000","last_peered":"2024-05-27T10:14:41.642435+0000","last_clean":"2024-05-27T10:14:41.642435+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:14:41.642435+0000","last_undegraded":"2024-05-27T10:14:41.642435+0000","last_fullsized":"2024-05-27T10:14:41.642435+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653301+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:14:41.653301+0000","last_peered":"2024-05-27T10:14:41.653301+0000","last_clean":"2024-05-27T10:14:41.653301+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:14:41.653301+0000","last_undegraded":"2024-05-27T10:14:41.653301+0000","last_fullsized":"2024-05-27T10:14:41.653301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646830+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:14:41.646830+0000","last_peered":"2024-05-27T10:14:41.646830+0000","last_clean":"2024-05-27T10:14:41.646830+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:14:41.646830+0000","last_undegraded":"2024-05-27T10:14:41.646830+0000","last_fullsized":"2024-05-27T10:14:41.646830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639338+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:14:41.639338+0000","last_peered":"2024-05-27T10:14:41.639338+0000","last_clean":"2024-05-27T10:14:41.639338+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:14:41.639338+0000","last_undegraded":"2024-05-27T10:14:41.639338+0000","last_fullsized":"2024-05-27T10:14:41.639338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646777+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:14:41.646777+0000","last_peered":"2024-05-27T10:14:41.646777+0000","last_clean":"2024-05-27T10:14:41.646777+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:14:41.646777+0000","last_undegraded":"2024-05-27T10:14:41.646777+0000","last_fullsized":"2024-05-27T10:14:41.646777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.641212+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:14:41.641212+0000","last_peered":"2024-05-27T10:14:41.641212+0000","last_clean":"2024-05-27T10:14:41.641212+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:14:41.641212+0000","last_undegraded":"2024-05-27T10:14:41.641212+0000","last_fullsized":"2024-05-27T10:14:41.641212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642492+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:14:41.642492+0000","last_peered":"2024-05-27T10:14:41.642492+0000","last_clean":"2024-05-27T10:14:41.642492+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:14:41.642492+0000","last_undegraded":"2024-05-27T10:14:41.642492+0000","last_fullsized":"2024-05-27T10:14:41.642492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640332+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:14:41.640332+0000","last_peered":"2024-05-27T10:14:41.640332+0000","last_clean":"2024-05-27T10:14:41.640332+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:14:41.640332+0000","last_undegraded":"2024-05-27T10:14:41.640332+0000","last_fullsized":"2024-05-27T10:14:41.640332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652987+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:14:41.652987+0000","last_peered":"2024-05-27T10:14:41.652987+0000","last_clean":"2024-05-27T10:14:41.652987+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:14:41.652987+0000","last_undegraded":"2024-05-27T10:14:41.652987+0000","last_fullsized":"2024-05-27T10:14:41.652987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:21.784124+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:15:21.784124+0000","last_peered":"2024-05-27T10:15:21.784124+0000","last_clean":"2024-05-27T10:15:21.784124+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:15:21.784124+0000","last_undegraded":"2024-05-27T10:15:21.784124+0000","last_fullsized":"2024-05-27T10:15:21.784124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645152+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:14:41.645152+0000","last_peered":"2024-05-27T10:14:41.645152+0000","last_clean":"2024-05-27T10:14:41.645152+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:14:41.645152+0000","last_undegraded":"2024-05-27T10:14:41.645152+0000","last_fullsized":"2024-05-27T10:14:41.645152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.654480+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:14:41.654480+0000","last_peered":"2024-05-27T10:14:41.654480+0000","last_clean":"2024-05-27T10:14:41.654480+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:14:41.654480+0000","last_undegraded":"2024-05-27T10:14:41.654480+0000","last_fullsized":"2024-05-27T10:14:41.654480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":99,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643011+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:14:41.643011+0000","last_peered":"2024-05-27T10:14:41.643011+0000","last_clean":"2024-05-27T10:14:41.643011+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:14:41.643011+0000","last_undegraded":"2024-05-27T10:14:41.643011+0000","last_fullsized":"2024-05-27T10:14:41.643011+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636152+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:14:41.636152+0000","last_peered":"2024-05-27T10:14:41.636152+0000","last_clean":"2024-05-27T10:14:41.636152+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:14:41.636152+0000","last_undegraded":"2024-05-27T10:14:41.636152+0000","last_fullsized":"2024-05-27T10:14:41.636152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643273+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:14:41.643273+0000","last_peered":"2024-05-27T10:14:41.643273+0000","last_clean":"2024-05-27T10:14:41.643273+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:14:41.643273+0000","last_undegraded":"2024-05-27T10:14:41.643273+0000","last_fullsized":"2024-05-27T10:14:41.643273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644150+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:14:41.644150+0000","last_peered":"2024-05-27T10:14:41.644150+0000","last_clean":"2024-05-27T10:14:41.644150+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:14:41.644150+0000","last_undegraded":"2024-05-27T10:14:41.644150+0000","last_fullsized":"2024-05-27T10:14:41.644150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637007+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:14:41.637007+0000","last_peered":"2024-05-27T10:14:41.637007+0000","last_clean":"2024-05-27T10:14:41.637007+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:14:41.637007+0000","last_undegraded":"2024-05-27T10:14:41.637007+0000","last_fullsized":"2024-05-27T10:14:41.637007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:15:09.209320+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:15:09.209320+0000","last_peered":"2024-05-27T10:15:09.209320+0000","last_clean":"2024-05-27T10:15:09.209320+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:15:09.209320+0000","last_undegraded":"2024-05-27T10:15:09.209320+0000","last_fullsized":"2024-05-27T10:15:09.209320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646942+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:14:41.646942+0000","last_peered":"2024-05-27T10:14:41.646942+0000","last_clean":"2024-05-27T10:14:41.646942+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:14:41.646942+0000","last_undegraded":"2024-05-27T10:14:41.646942+0000","last_fullsized":"2024-05-27T10:14:41.646942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652545+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:14:41.652545+0000","last_peered":"2024-05-27T10:14:41.652545+0000","last_clean":"2024-05-27T10:14:41.652545+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:14:41.652545+0000","last_undegraded":"2024-05-27T10:14:41.652545+0000","last_fullsized":"2024-05-27T10:14:41.652545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.455434+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:14:42.455434+0000","last_peered":"2024-05-27T10:14:42.455434+0000","last_clean":"2024-05-27T10:14:42.455434+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:14:42.455434+0000","last_undegraded":"2024-05-27T10:14:42.455434+0000","last_fullsized":"2024-05-27T10:14:42.455434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053163+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:14:42.053163+0000","last_peered":"2024-05-27T10:14:42.053163+0000","last_clean":"2024-05-27T10:14:42.053163+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:14:42.053163+0000","last_undegraded":"2024-05-27T10:14:42.053163+0000","last_fullsized":"2024-05-27T10:14:42.053163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642586+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:14:41.642586+0000","last_peered":"2024-05-27T10:14:41.642586+0000","last_clean":"2024-05-27T10:14:41.642586+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:14:41.642586+0000","last_undegraded":"2024-05-27T10:14:41.642586+0000","last_fullsized":"2024-05-27T10:14:41.642586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645409+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:14:41.645409+0000","last_peered":"2024-05-27T10:14:41.645409+0000","last_clean":"2024-05-27T10:14:41.645409+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:14:41.645409+0000","last_undegraded":"2024-05-27T10:14:41.645409+0000","last_fullsized":"2024-05-27T10:14:41.645409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.638987+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:14:41.638987+0000","last_peered":"2024-05-27T10:14:41.638987+0000","last_clean":"2024-05-27T10:14:41.638987+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:14:41.638987+0000","last_undegraded":"2024-05-27T10:14:41.638987+0000","last_fullsized":"2024-05-27T10:14:41.638987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.659057+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:14:41.659057+0000","last_peered":"2024-05-27T10:14:41.659057+0000","last_clean":"2024-05-27T10:14:41.659057+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:14:41.659057+0000","last_undegraded":"2024-05-27T10:14:41.659057+0000","last_fullsized":"2024-05-27T10:14:41.659057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646308+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:14:41.646308+0000","last_peered":"2024-05-27T10:14:41.646308+0000","last_clean":"2024-05-27T10:14:41.646308+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:14:41.646308+0000","last_undegraded":"2024-05-27T10:14:41.646308+0000","last_fullsized":"2024-05-27T10:14:41.646308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.663013+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:14:41.663013+0000","last_peered":"2024-05-27T10:14:41.663013+0000","last_clean":"2024-05-27T10:14:41.663013+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:14:41.663013+0000","last_undegraded":"2024-05-27T10:14:41.663013+0000","last_fullsized":"2024-05-27T10:14:41.663013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642553+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:14:41.642553+0000","last_peered":"2024-05-27T10:14:41.642553+0000","last_clean":"2024-05-27T10:14:41.642553+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:14:41.642553+0000","last_undegraded":"2024-05-27T10:14:41.642553+0000","last_fullsized":"2024-05-27T10:14:41.642553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644620+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:14:41.644620+0000","last_peered":"2024-05-27T10:14:41.644620+0000","last_clean":"2024-05-27T10:14:41.644620+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:14:41.644620+0000","last_undegraded":"2024-05-27T10:14:41.644620+0000","last_fullsized":"2024-05-27T10:14:41.644620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642991+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:14:41.642991+0000","last_peered":"2024-05-27T10:14:41.642991+0000","last_clean":"2024-05-27T10:14:41.642991+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:14:41.642991+0000","last_undegraded":"2024-05-27T10:14:41.642991+0000","last_fullsized":"2024-05-27T10:14:41.642991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.653579+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:14:41.653579+0000","last_peered":"2024-05-27T10:14:41.653579+0000","last_clean":"2024-05-27T10:14:41.653579+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:14:41.653579+0000","last_undegraded":"2024-05-27T10:14:41.653579+0000","last_fullsized":"2024-05-27T10:14:41.653579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642782+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:14:41.642782+0000","last_peered":"2024-05-27T10:14:41.642782+0000","last_clean":"2024-05-27T10:14:41.642782+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:14:41.642782+0000","last_undegraded":"2024-05-27T10:14:41.642782+0000","last_fullsized":"2024-05-27T10:14:41.642782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637216+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:14:41.637216+0000","last_peered":"2024-05-27T10:14:41.637216+0000","last_clean":"2024-05-27T10:14:41.637216+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:14:41.637216+0000","last_undegraded":"2024-05-27T10:14:41.637216+0000","last_fullsized":"2024-05-27T10:14:41.637216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643381+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:14:41.643381+0000","last_peered":"2024-05-27T10:14:41.643381+0000","last_clean":"2024-05-27T10:14:41.643381+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:14:41.643381+0000","last_undegraded":"2024-05-27T10:14:41.643381+0000","last_fullsized":"2024-05-27T10:14:41.643381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639128+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:14:41.639128+0000","last_peered":"2024-05-27T10:14:41.639128+0000","last_clean":"2024-05-27T10:14:41.639128+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:14:41.639128+0000","last_undegraded":"2024-05-27T10:14:41.639128+0000","last_fullsized":"2024-05-27T10:14:41.639128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643457+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:14:41.643457+0000","last_peered":"2024-05-27T10:14:41.643457+0000","last_clean":"2024-05-27T10:14:41.643457+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:14:41.643457+0000","last_undegraded":"2024-05-27T10:14:41.643457+0000","last_fullsized":"2024-05-27T10:14:41.643457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643352+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:14:41.643352+0000","last_peered":"2024-05-27T10:14:41.643352+0000","last_clean":"2024-05-27T10:14:41.643352+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:14:41.643352+0000","last_undegraded":"2024-05-27T10:14:41.643352+0000","last_fullsized":"2024-05-27T10:14:41.643352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.454848+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:14:42.454848+0000","last_peered":"2024-05-27T10:14:42.454848+0000","last_clean":"2024-05-27T10:14:42.454848+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:14:42.454848+0000","last_undegraded":"2024-05-27T10:14:42.454848+0000","last_fullsized":"2024-05-27T10:14:42.454848+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637191+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:14:41.637191+0000","last_peered":"2024-05-27T10:14:41.637191+0000","last_clean":"2024-05-27T10:14:41.637191+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:14:41.637191+0000","last_undegraded":"2024-05-27T10:14:41.637191+0000","last_fullsized":"2024-05-27T10:14:41.637191+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640158+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:14:41.640158+0000","last_peered":"2024-05-27T10:14:41.640158+0000","last_clean":"2024-05-27T10:14:41.640158+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:14:41.640158+0000","last_undegraded":"2024-05-27T10:14:41.640158+0000","last_fullsized":"2024-05-27T10:14:41.640158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645293+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:14:41.645293+0000","last_peered":"2024-05-27T10:14:41.645293+0000","last_clean":"2024-05-27T10:14:41.645293+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:14:41.645293+0000","last_undegraded":"2024-05-27T10:14:41.645293+0000","last_fullsized":"2024-05-27T10:14:41.645293+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.657723+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:14:41.657723+0000","last_peered":"2024-05-27T10:14:41.657723+0000","last_clean":"2024-05-27T10:14:41.657723+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:14:41.657723+0000","last_undegraded":"2024-05-27T10:14:41.657723+0000","last_fullsized":"2024-05-27T10:14:41.657723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652483+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:14:41.652483+0000","last_peered":"2024-05-27T10:14:41.652483+0000","last_clean":"2024-05-27T10:14:41.652483+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:14:41.652483+0000","last_undegraded":"2024-05-27T10:14:41.652483+0000","last_fullsized":"2024-05-27T10:14:41.652483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645178+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:14:41.645178+0000","last_peered":"2024-05-27T10:14:41.645178+0000","last_clean":"2024-05-27T10:14:41.645178+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:14:41.645178+0000","last_undegraded":"2024-05-27T10:14:41.645178+0000","last_fullsized":"2024-05-27T10:14:41.645178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053963+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:14:42.053963+0000","last_peered":"2024-05-27T10:14:42.053963+0000","last_clean":"2024-05-27T10:14:42.053963+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:14:42.053963+0000","last_undegraded":"2024-05-27T10:14:42.053963+0000","last_fullsized":"2024-05-27T10:14:42.053963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637018+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:14:41.637018+0000","last_peered":"2024-05-27T10:14:41.637018+0000","last_clean":"2024-05-27T10:14:41.637018+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:14:41.637018+0000","last_undegraded":"2024-05-27T10:14:41.637018+0000","last_fullsized":"2024-05-27T10:14:41.637018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.639344+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:14:41.639344+0000","last_peered":"2024-05-27T10:14:41.639344+0000","last_clean":"2024-05-27T10:14:41.639344+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:14:41.639344+0000","last_undegraded":"2024-05-27T10:14:41.639344+0000","last_fullsized":"2024-05-27T10:14:41.639344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63048,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":282,"num_read_kb":209,"num_write":166,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678049,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":428086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1078614,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037808,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":428322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":428242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":1020563,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":427711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":486160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":428982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":487901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":428022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":428539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790031,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":1019856,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:15:26.121 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T10:15:26.122 INFO:tasks.cephadm:Setting up client nodes... 2024-05-27T10:15:26.122 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T10:15:26.592 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:26.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:26 smithi143 ceph-mon[35545]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:15:26.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:26 smithi042 ceph-mon[31661]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:15:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:26 smithi104 ceph-mon[34812]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:15:27.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:27 smithi042 ceph-mon[31661]: pgmap v424: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-27T10:15:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:27 smithi143 ceph-mon[35545]: pgmap v424: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-27T10:15:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:27 smithi104 ceph-mon[34812]: pgmap v424: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-27T10:15:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 -- 172.21.15.42:0/3092636225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0760e0 msgr2=0x7f85fc076570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 --2- 172.21.15.42:0/3092636225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0760e0 0x7f85fc076570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009990 tx=0x7f85e4009ca0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 -- 172.21.15.42:0/3092636225 shutdown_connections 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 --2- 172.21.15.42:0/3092636225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85fc10bff0 0x7f85fc10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 --2- 172.21.15.42:0/3092636225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85fc0776d0 0x7f85fc075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 --2- 172.21.15.42:0/3092636225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0760e0 0x7f85fc076570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 -- 172.21.15.42:0/3092636225 >> 172.21.15.42:0/3092636225 conn(0x7f85fc0ff7a0 msgr2=0x7f85fc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 -- 172.21.15.42:0/3092636225 shutdown_connections 2024-05-27T10:15:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.046+0000 7f8601200700 1 -- 172.21.15.42:0/3092636225 wait complete. 2024-05-27T10:15:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.047+0000 7f8601200700 1 Processor -- start 2024-05-27T10:15:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 -- start start 2024-05-27T10:15:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85fc0760e0 0x7f85fc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 0x7f85fc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85fc10bff0 0x7f85fc1a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc111c50 con 0x7f85fc0776d0 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc111ad0 con 0x7f85fc0760e0 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f8601200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc111dd0 con 0x7f85fc10bff0 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f85f2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 0x7f85fc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f85f2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 0x7f85fc19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46066/0 (socket says 172.21.15.42:46066) 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.048+0000 7f85f2a00700 1 -- 172.21.15.42:0/1432592896 learned_addr learned my addr 172.21.15.42:0/1432592896 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85fb400700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85fc0760e0 0x7f85fc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85fbe00700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85fc10bff0 0x7f85fc1a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85f2a00700 1 -- 172.21.15.42:0/1432592896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85fc10bff0 msgr2=0x7f85fc1a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85f2a00700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85fc10bff0 0x7f85fc1a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85f2a00700 1 -- 172.21.15.42:0/1432592896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85fc0760e0 msgr2=0x7f85fc19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85f2a00700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85fc0760e0 0x7f85fc19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85f2a00700 1 -- 172.21.15.42:0/1432592896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc1a43b0 con 0x7f85fc0776d0 2024-05-27T10:15:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.049+0000 7f85f2a00700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 0x7f85fc19cd90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85e40098e0 tx=0x7f85e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.050+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4004830 con 0x7f85fc0776d0 2024-05-27T10:15:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.050+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85e4005020 con 0x7f85fc0776d0 2024-05-27T10:15:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.050+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4003d30 con 0x7f85fc0776d0 2024-05-27T10:15:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.050+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc1a4640 con 0x7f85fc0776d0 2024-05-27T10:15:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.050+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc1a4a30 con 0x7f85fc0776d0 2024-05-27T10:15:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.051+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85fc0512b0 con 0x7f85fc0776d0 2024-05-27T10:15:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.054+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f85e40054d0 con 0x7f85fc0776d0 2024-05-27T10:15:29.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.055+0000 7f85f8c00700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f85e00b2910 0x7f85e00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:29.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.056+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f85e40e3890 con 0x7f85fc0776d0 2024-05-27T10:15:29.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.056+0000 7f85fb400700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f85e00b2910 0x7f85e00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.056+0000 7f85fb400700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f85e00b2910 0x7f85e00b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f85ec005950 tx=0x7f85ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:29.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.058+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85e4030020 con 0x7f85fc0776d0 2024-05-27T10:15:29.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.387+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f85fc0643c0 con 0x7f85fc0776d0 2024-05-27T10:15:29.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.394+0000 7f85f8c00700 1 -- 172.21.15.42:0/1432592896 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v34) ==== 170+0+59 (secure 0 0 0) 0x7f85e4013070 con 0x7f85fc0776d0 2024-05-27T10:15:29.394 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-27T10:15:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBBXVRmij4wFxAAlqt3EqNWRyLZ1rG/l6ZawQ== 2024-05-27T10:15:29.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f85e00b2910 msgr2=0x7f85e00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:29.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f85e00b2910 0x7f85e00b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f85ec005950 tx=0x7f85ec00b410 comp rx=0 tx=0).stop 2024-05-27T10:15:29.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 msgr2=0x7f85fc19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:29.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 0x7f85fc19cd90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85e40098e0 tx=0x7f85e40047b0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 shutdown_connections 2024-05-27T10:15:29.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85fc10bff0 0x7f85fc1a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f85e00b2910 0x7f85e00b4e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85fc0760e0 0x7f85fc19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 --2- 172.21.15.42:0/1432592896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85fc0776d0 0x7f85fc19cd90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:29.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 >> 172.21.15.42:0/1432592896 conn(0x7f85fc0ff7a0 msgr2=0x7f85fc107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:29.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 shutdown_connections 2024-05-27T10:15:29.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:29.397+0000 7f8601200700 1 -- 172.21.15.42:0/1432592896 wait complete. 2024-05-27T10:15:29.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:29 smithi042 ceph-mon[31661]: pgmap v425: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-27T10:15:29.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:29 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1432592896' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T10:15:29.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:29 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1432592896' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T10:15:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:29 smithi143 ceph-mon[35545]: pgmap v425: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-27T10:15:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:29 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1432592896' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T10:15:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:29 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1432592896' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T10:15:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:29 smithi104 ceph-mon[34812]: pgmap v425: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-27T10:15:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:29 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1432592896' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T10:15:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:29 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1432592896' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T10:15:30.247 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:15:30.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-27T10:15:30.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T10:15:30.374 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T10:15:30.374 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-27T10:15:30.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-27T10:15:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:31.575 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:31 smithi042 ceph-mon[31661]: pgmap v426: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-27T10:15:31.811 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:31 smithi143 ceph-mon[35545]: pgmap v426: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-27T10:15:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:31 smithi104 ceph-mon[34812]: pgmap v426: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-27T10:15:33.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 -- 172.21.15.42:0/3546126656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 msgr2=0x7f74a810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:33.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 --2- 172.21.15.42:0/3546126656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a810abc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7490009990 tx=0x7f7490009ca0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 -- 172.21.15.42:0/3546126656 shutdown_connections 2024-05-27T10:15:33.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 --2- 172.21.15.42:0/3546126656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a810b100 0x7f74a8117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 --2- 172.21.15.42:0/3546126656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74a810f6b0 0x7f74a810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 --2- 172.21.15.42:0/3546126656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a810abc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.297+0000 7f74ada00700 1 -- 172.21.15.42:0/3546126656 >> 172.21.15.42:0/3546126656 conn(0x7f74a8076af0 msgr2=0x7f74a8076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:33.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.298+0000 7f74ada00700 1 -- 172.21.15.42:0/3546126656 shutdown_connections 2024-05-27T10:15:33.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.298+0000 7f74ada00700 1 -- 172.21.15.42:0/3546126656 wait complete. 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.299+0000 7f74ada00700 1 Processor -- start 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.299+0000 7f74ada00700 1 -- start start 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74ada00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a81a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74ada00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a810b100 0x7f74a81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74ada00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74a810f6b0 0x7f74a81a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74ada00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74a811a740 con 0x7f74a810a7b0 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74ada00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74a811a5c0 con 0x7f74a810f6b0 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74ada00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f74a811a8c0 con 0x7f74a810b100 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a81a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a81a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60078/0 (socket says 172.21.15.42:60078) 2024-05-27T10:15:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.300+0000 7f74a7e00700 1 -- 172.21.15.42:0/4166258109 learned_addr learned my addr 172.21.15.42:0/4166258109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:33.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f749f400700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a810b100 0x7f74a81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a7e00700 1 -- 172.21.15.42:0/4166258109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a810b100 msgr2=0x7f74a81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a7e00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a810b100 0x7f74a81a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a7e00700 1 -- 172.21.15.42:0/4166258109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74a810f6b0 msgr2=0x7f74a81a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74ad000700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74a810f6b0 0x7f74a81a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a7e00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74a810f6b0 0x7f74a81a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a7e00700 1 -- 172.21.15.42:0/4166258109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a81a5f70 con 0x7f74a810a7b0 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a7e00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a81a1150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f749800d950 tx=0x7f749800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.301+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498016070 con 0x7f74a810a7b0 2024-05-27T10:15:33.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.302+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7498010460 con 0x7f74a810a7b0 2024-05-27T10:15:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.302+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a81a6260 con 0x7f74a810a7b0 2024-05-27T10:15:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.302+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749800f620 con 0x7f74a810a7b0 2024-05-27T10:15:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.302+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81a67a0 con 0x7f74a810a7b0 2024-05-27T10:15:33.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.303+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a80512b0 con 0x7f74a810a7b0 2024-05-27T10:15:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.307+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7498019070 con 0x7f74a810a7b0 2024-05-27T10:15:33.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.309+0000 7f74a5600700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748c0b2910 0x7f748c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:33.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.309+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f74980e1f70 con 0x7f74a810a7b0 2024-05-27T10:15:33.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.309+0000 7f749f400700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748c0b2910 0x7f748c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:33.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.310+0000 7f749f400700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748c0b2910 0x7f748c0b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7490003760 tx=0x7f7490005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:33.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.311+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74980a9b00 con 0x7f74a810a7b0 2024-05-27T10:15:33.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:33 smithi042 ceph-mon[31661]: pgmap v427: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:33.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.615+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f74a802e5e0 con 0x7f74a810a7b0 2024-05-27T10:15:33.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.622+0000 7f74a5600700 1 -- 172.21.15.42:0/4166258109 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288647 (secure 0 0 0) 0x7f74980a9400 con 0x7f74a810a7b0 2024-05-27T10:15:33.630 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748c0b2910 msgr2=0x7f748c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748c0b2910 0x7f748c0b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7490003760 tx=0x7f7490005fb0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 msgr2=0x7f74a81a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a81a1150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f749800d950 tx=0x7f749800dd10 comp rx=0 tx=0).stop 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 shutdown_connections 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f748c0b2910 0x7f748c0b4e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74a810f6b0 0x7f74a81a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74a810a7b0 0x7f74a81a1150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 --2- 172.21.15.42:0/4166258109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a810b100 0x7f74a81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.628+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 >> 172.21.15.42:0/4166258109 conn(0x7f74a8076af0 msgr2=0x7f74a8108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.629+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 shutdown_connections 2024-05-27T10:15:33.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:33.629+0000 7f74ada00700 1 -- 172.21.15.42:0/4166258109 wait complete. 2024-05-27T10:15:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:33 smithi143 ceph-mon[35545]: pgmap v427: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:33.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:33 smithi104 ceph-mon[34812]: pgmap v427: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:34.564 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":2062230330},{"type":"v1","addr":"172.21.15.42:6801","nonce":2062230330}]},"active_addr":"172.21.15.42:6801/2062230330","active_change":"2024-05-27T10:03:40.179637+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2667703980}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1953650070}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2970531755}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1310078254}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1511498983}]}]} 2024-05-27T10:15:34.571 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-27T10:15:34.571 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-27T10:15:34.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:15:34.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:34 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4166258109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T10:15:34.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:34 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4166258109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T10:15:34.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:34 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4166258109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T10:15:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:35 smithi042 ceph-mon[31661]: pgmap v428: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:35.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:35 smithi104 ceph-mon[34812]: pgmap v428: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:36.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:35 smithi143 ceph-mon[35545]: pgmap v428: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:37.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.495+0000 7f21bde00700 1 -- 172.21.15.42:0/2719926356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b8106ec0 msgr2=0x7f21b8113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.495+0000 7f21bde00700 1 --2- 172.21.15.42:0/2719926356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b8106ec0 0x7f21b8113350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21b000b0a0 tx=0x7f21b000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 -- 172.21.15.42:0/2719926356 shutdown_connections 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 --2- 172.21.15.42:0/2719926356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b8106ec0 0x7f21b8113350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 --2- 172.21.15.42:0/2719926356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b810c5b0 0x7f21b810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 --2- 172.21.15.42:0/2719926356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106570 0x7f21b8106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 -- 172.21.15.42:0/2719926356 >> 172.21.15.42:0/2719926356 conn(0x7f21b8102070 msgr2=0x7f21b81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 -- 172.21.15.42:0/2719926356 shutdown_connections 2024-05-27T10:15:37.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.496+0000 7f21bde00700 1 -- 172.21.15.42:0/2719926356 wait complete. 2024-05-27T10:15:37.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.497+0000 7f21bde00700 1 Processor -- start 2024-05-27T10:15:37.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.497+0000 7f21bde00700 1 -- start start 2024-05-27T10:15:37.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21bde00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b8106570 0x7f21b819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:37.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21bde00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 0x7f21b819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:37.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21bde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b810c5b0 0x7f21b81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:37.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21bde00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8116600 con 0x7f21b810c5b0 2024-05-27T10:15:37.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21bde00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8116480 con 0x7f21b8106ec0 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21bde00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8116780 con 0x7f21b8106570 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21ae000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 0x7f21b819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21b7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b810c5b0 0x7f21b81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21ae000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 0x7f21b819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38308/0 (socket says 172.21.15.42:38308) 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21ae000700 1 -- 172.21.15.42:0/1180517633 learned_addr learned my addr 172.21.15.42:0/1180517633 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.498+0000 7f21b6a00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b8106570 0x7f21b819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21ae000700 1 -- 172.21.15.42:0/1180517633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b8106570 msgr2=0x7f21b819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21ae000700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b8106570 0x7f21b819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21ae000700 1 -- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b810c5b0 msgr2=0x7f21b81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21ae000700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b810c5b0 0x7f21b81a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21ae000700 1 -- 172.21.15.42:0/1180517633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b81a1d50 con 0x7f21b8106ec0 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21b7400700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b810c5b0 0x7f21b81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21ae000700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 0x7f21b819d4d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c9b0 tx=0x7f21a000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.499+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0007aa0 con 0x7f21b8106ec0 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.500+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21a000ce50 con 0x7f21b8106ec0 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.500+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b81a1fe0 con 0x7f21b8106ec0 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.500+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0018400 con 0x7f21b8106ec0 2024-05-27T10:15:37.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.500+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b81a2520 con 0x7f21b8106ec0 2024-05-27T10:15:37.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.501+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b80512b0 con 0x7f21b8106ec0 2024-05-27T10:15:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.504+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f21a000f450 con 0x7f21b8106ec0 2024-05-27T10:15:37.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.506+0000 7f21bca00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f219c0b2910 0x7f219c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:37.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.506+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f21a00e2650 con 0x7f21b8106ec0 2024-05-27T10:15:37.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.506+0000 7f21b6a00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f219c0b2910 0x7f219c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:37.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.507+0000 7f21b6a00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f219c0b2910 0x7f219c0b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21a8005950 tx=0x7f21a800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:37.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.521+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21a0014030 con 0x7f21b8106ec0 2024-05-27T10:15:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.766+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21b802e5e0 con 0x7f21b8106ec0 2024-05-27T10:15:37.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:37 smithi042 ceph-mon[31661]: pgmap v429: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:15:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.772+0000 7f21bca00700 1 -- 172.21.15.42:0/1180517633 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22562 (secure 0 0 0) 0x7f21a00aa1e0 con 0x7f21b8106ec0 2024-05-27T10:15:37.773 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:37.773 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:41.618756+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:15:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f219c0b2910 msgr2=0x7f219c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:37.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f219c0b2910 0x7f219c0b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21a8005950 tx=0x7f21a800a300 comp rx=0 tx=0).stop 2024-05-27T10:15:37.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 msgr2=0x7f21b819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:37.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 0x7f21b819d4d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c9b0 tx=0x7f21a000ccc0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 shutdown_connections 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b810c5b0 0x7f21b81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b8106570 0x7f21b819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f219c0b2910 0x7f219c0b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 --2- 172.21.15.42:0/1180517633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21b8106ec0 0x7f21b819d4d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 >> 172.21.15.42:0/1180517633 conn(0x7f21b8102070 msgr2=0x7f21b8104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 shutdown_connections 2024-05-27T10:15:37.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:37.776+0000 7f21bde00700 1 -- 172.21.15.42:0/1180517633 wait complete. 2024-05-27T10:15:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:37 smithi104 ceph-mon[34812]: pgmap v429: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:15:38.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:37 smithi143 ceph-mon[35545]: pgmap v429: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:15:38.745 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-27T10:15:38.746 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:15:38.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:38 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1180517633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:38.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:38 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1180517633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:39.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:38 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1180517633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:39.215 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:39.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:39 smithi042 ceph-mon[31661]: pgmap v430: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:15:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:39 smithi104 ceph-mon[34812]: pgmap v430: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:15:40.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:39 smithi143 ceph-mon[35545]: pgmap v430: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:15:41.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 -- 172.21.15.42:0/3301128496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b0106610 msgr2=0x7f61b0106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3301128496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b0106610 0x7f61b0106a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f61a40099c0 tx=0x7f61a4009cd0 comp rx=0 tx=0).stop 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 -- 172.21.15.42:0/3301128496 shutdown_connections 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3301128496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010b890 0x7f61b01131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3301128496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b0106610 0x7f61b0106a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3301128496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 0x7f61b010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 -- 172.21.15.42:0/3301128496 >> 172.21.15.42:0/3301128496 conn(0x7f61b0102020 msgr2=0x7f61b0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 -- 172.21.15.42:0/3301128496 shutdown_connections 2024-05-27T10:15:41.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.747+0000 7f61b6a00700 1 -- 172.21.15.42:0/3301128496 wait complete. 2024-05-27T10:15:41.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.748+0000 7f61b6a00700 1 Processor -- start 2024-05-27T10:15:41.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 -- start start 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 0x7f61b019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 0x7f61b019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b01a16a0 0x7f61b01a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01163c0 con 0x7f61b010b890 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0116240 con 0x7f61b010af40 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b6a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0116540 con 0x7f61b01a16a0 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.749+0000 7f61b4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 0x7f61b019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 0x7f61b019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60108/0 (socket says 172.21.15.42:60108) 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 -- 172.21.15.42:0/3786662780 learned_addr learned my addr 172.21.15.42:0/3786662780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b5600700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 0x7f61b019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b6000700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b01a16a0 0x7f61b01a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 -- 172.21.15.42:0/3786662780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b01a16a0 msgr2=0x7f61b01a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b01a16a0 0x7f61b01a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 -- 172.21.15.42:0/3786662780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 msgr2=0x7f61b019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 0x7f61b019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b4c00700 1 -- 172.21.15.42:0/3786662780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b01a21f0 con 0x7f61b010b890 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.750+0000 7f61b5600700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 0x7f61b019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.751+0000 7f61b4c00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 0x7f61b019d4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f619c00d950 tx=0x7f619c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.751+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c016070 con 0x7f61b010b890 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.751+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f619c00de30 con 0x7f61b010b890 2024-05-27T10:15:41.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.751+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c005240 con 0x7f61b010b890 2024-05-27T10:15:41.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.751+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b01a24e0 con 0x7f61b010b890 2024-05-27T10:15:41.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.751+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01a9c70 con 0x7f61b010b890 2024-05-27T10:15:41.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.752+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b00512b0 con 0x7f61b010b890 2024-05-27T10:15:41.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.755+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f619c0049a0 con 0x7f61b010b890 2024-05-27T10:15:41.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.756+0000 7f61ae000700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61980b2990 0x7f61980b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:41.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.757+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f619c0e1f40 con 0x7f61b010b890 2024-05-27T10:15:41.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.757+0000 7f61b5600700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61980b2990 0x7f61980b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:41.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.758+0000 7f61b5600700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61980b2990 0x7f61980b4ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f61a4000c00 tx=0x7f61a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:41.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:41.759+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f619c01f070 con 0x7f61b010b890 2024-05-27T10:15:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:41 smithi042 ceph-mon[31661]: pgmap v431: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:15:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:41 smithi104 ceph-mon[34812]: pgmap v431: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:15:42.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.002+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61b002e7a0 con 0x7f61b010b890 2024-05-27T10:15:42.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.007+0000 7f61ae000700 1 -- 172.21.15.42:0/3786662780 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22562 (secure 0 0 0) 0x7f619c0a9ad0 con 0x7f61b010b890 2024-05-27T10:15:42.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:15:42.008 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:41.618756+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.010+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61980b2990 msgr2=0x7f61980b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.010+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61980b2990 0x7f61980b4ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f61a4000c00 tx=0x7f61a401a040 comp rx=0 tx=0).stop 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.010+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 msgr2=0x7f61b019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.010+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 0x7f61b019d4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f619c00d950 tx=0x7f619c00dd10 comp rx=0 tx=0).stop 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 shutdown_connections 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b01a16a0 0x7f61b01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61980b2990 0x7f61980b4ea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010af40 0x7f61b019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 --2- 172.21.15.42:0/3786662780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010b890 0x7f61b019d4d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 >> 172.21.15.42:0/3786662780 conn(0x7f61b0102020 msgr2=0x7f61b0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 shutdown_connections 2024-05-27T10:15:42.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:42.011+0000 7f61b6a00700 1 -- 172.21.15.42:0/3786662780 wait complete. 2024-05-27T10:15:42.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:41 smithi143 ceph-mon[35545]: pgmap v431: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:15:42.863 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:42 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3786662780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:42.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-27T10:15:42.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-27T10:15:42.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-27T10:15:42.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-27T10:15:42.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-27T10:15:42.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-27T10:15:42.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-27T10:15:42.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-27T10:15:42.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-27T10:15:42.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-27T10:15:42.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-27T10:15:42.937 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-27T10:15:42.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:42 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3786662780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:43.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:42 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3786662780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:15:43.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:43 smithi104 ceph-mon[34812]: pgmap v432: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:43.992 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:43 smithi143 ceph-mon[35545]: pgmap v432: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:44.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:43 smithi042 ceph-mon[31661]: pgmap v432: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:44.187 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.218 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.222 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:15:45.673 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:45 smithi042 ceph-mon[31661]: pgmap v433: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:45 smithi104 ceph-mon[34812]: pgmap v433: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:45 smithi143 ceph-mon[35545]: pgmap v433: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:47 smithi042 ceph-mon[31661]: pgmap v434: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:15:47.917 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:47 smithi104 ceph-mon[34812]: pgmap v434: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:15:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:47 smithi143 ceph-mon[35545]: pgmap v434: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:15:49.925 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:49 smithi042 ceph-mon[31661]: pgmap v435: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:15:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:49 smithi104 ceph-mon[34812]: pgmap v435: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:15:50.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:49 smithi143 ceph-mon[35545]: pgmap v435: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:15:51.662 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:51 smithi042 ceph-mon[31661]: pgmap v436: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:15:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:51 smithi104 ceph-mon[34812]: pgmap v436: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:15:52.062 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:51 smithi143 ceph-mon[35545]: pgmap v436: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:15:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:53 smithi104 ceph-mon[34812]: pgmap v437: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:54.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:53 smithi042 ceph-mon[31661]: pgmap v437: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:53 smithi143 ceph-mon[35545]: pgmap v437: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 -- 172.21.15.42:0/1014837845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410af40 msgr2=0x7fc29410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 --2- 172.21.15.42:0/1014837845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410af40 0x7fc29410b350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0099c0 tx=0x7fc27c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 -- 172.21.15.42:0/1014837845 shutdown_connections 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 --2- 172.21.15.42:0/1014837845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29410b890 0x7fc2941131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 --2- 172.21.15.42:0/1014837845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc294106610 0x7fc294106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 --2- 172.21.15.42:0/1014837845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410af40 0x7fc29410b350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 -- 172.21.15.42:0/1014837845 >> 172.21.15.42:0/1014837845 conn(0x7fc294102020 msgr2=0x7fc294104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 -- 172.21.15.42:0/1014837845 shutdown_connections 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.206+0000 7fc293200700 1 -- 172.21.15.42:0/1014837845 wait complete. 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.207+0000 7fc293200700 1 Processor -- start 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.207+0000 7fc293200700 1 -- start start 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.208+0000 7fc293200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc294106610 0x7fc29419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.208+0000 7fc293200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc29410af40 0x7fc29419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.208+0000 7fc293200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410b890 0x7fc2941a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.208+0000 7fc293200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941163a0 con 0x7fc29410b890 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.208+0000 7fc293200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc294116220 con 0x7fc29410af40 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.208+0000 7fc293200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc294116520 con 0x7fc294106610 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.209+0000 7fc291e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc294106610 0x7fc29419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.209+0000 7fc291400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc29410af40 0x7fc29419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.209+0000 7fc291400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc29410af40 0x7fc29419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:43856/0 (socket says 172.21.15.42:43856) 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.209+0000 7fc291400700 1 -- 172.21.15.42:0/1245802999 learned_addr learned my addr 172.21.15.42:0/1245802999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.209+0000 7fc292800700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410b890 0x7fc2941a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.210+0000 7fc291400700 1 -- 172.21.15.42:0/1245802999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc294106610 msgr2=0x7fc29419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.210+0000 7fc291400700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc294106610 0x7fc29419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.210+0000 7fc291400700 1 -- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410b890 msgr2=0x7fc2941a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.210+0000 7fc291400700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc29410b890 0x7fc2941a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.210+0000 7fc291400700 1 -- 172.21.15.42:0/1245802999 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2941a1ce0 con 0x7fc29410af40 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.211+0000 7fc291400700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc29410af40 0x7fc29419d3b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b580 tx=0x7fc27c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.211+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c009e60 con 0x7fc29410af40 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.211+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2941a1f70 con 0x7fc29410af40 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.212+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc27c0047c0 con 0x7fc29410af40 2024-05-27T10:15:55.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.212+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c003e60 con 0x7fc29410af40 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.212+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2941a2440 con 0x7fc29410af40 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.213+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc294063e80 con 0x7fc29410af40 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.218+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc27c004c90 con 0x7fc29410af40 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.219+0000 7fc28aa00700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc2780b2990 0x7fc2780b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.219+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc27c0e32f0 con 0x7fc29410af40 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.220+0000 7fc291e00700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc2780b2990 0x7fc2780b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.220+0000 7fc28aa00700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] conn(0x7fc2780bc4a0 0x7fc2780be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.220+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 --> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2780bf060 con 0x7fc2780bc4a0 2024-05-27T10:15:55.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.220+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc27c0e3740 con 0x7fc29410af40 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.220+0000 7fc292800700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] conn(0x7fc2780bc4a0 0x7fc2780be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.220+0000 7fc291e00700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc2780b2990 0x7fc2780b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009550 tx=0x7fc28c0091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.221+0000 7fc292800700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] conn(0x7fc2780bc4a0 0x7fc2780be990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.222+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== osd.6 v2:172.21.15.42:6818/2880152722 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2780bf060 con 0x7fc2780bc4a0 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.241+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 --> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc2940642b0 con 0x7fc2780bc4a0 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.241+0000 7fc28aa00700 1 -- 172.21.15.42:0/1245802999 <== osd.6 v2:172.21.15.42:6818/2880152722 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc2940642b0 con 0x7fc2780bc4a0 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] conn(0x7fc2780bc4a0 msgr2=0x7fc2780be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] conn(0x7fc2780bc4a0 0x7fc2780be990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc2780b2990 msgr2=0x7fc2780b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc2780b2990 0x7fc2780b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009550 tx=0x7fc28c0091f0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc29410af40 msgr2=0x7fc29419d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:15:55.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 --2- 172.21.15.42:0/1245802999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc29410af40 0x7fc29419d3b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b580 tx=0x7fc27c0045e0 comp rx=0 tx=0).stop 2024-05-27T10:15:55.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc292800700 1 -- 172.21.15.42:0/1245802999 reap_dead start 2024-05-27T10:15:55.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 shutdown_connections 2024-05-27T10:15:55.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 >> 172.21.15.42:0/1245802999 conn(0x7fc294102020 msgr2=0x7fc29410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:15:55.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 shutdown_connections 2024-05-27T10:15:55.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:15:55.242+0000 7fc293200700 1 -- 172.21.15.42:0/1245802999 wait complete. 2024-05-27T10:15:55.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:55 smithi104 ceph-mon[34812]: pgmap v438: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:56.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:55 smithi143 ceph-mon[35545]: pgmap v438: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:56.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:55 smithi042 ceph-mon[31661]: pgmap v438: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:15:57.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:57 smithi104 ceph-mon[34812]: pgmap v439: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T10:15:58.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:57 smithi143 ceph-mon[35545]: pgmap v439: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T10:15:58.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:57 smithi042 ceph-mon[31661]: pgmap v439: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T10:15:59.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:15:59 smithi104 ceph-mon[34812]: pgmap v440: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:00.034 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:15:59 smithi042 ceph-mon[31661]: pgmap v440: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:00.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:15:59 smithi143 ceph-mon[35545]: pgmap v440: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:01 smithi042 ceph-mon[31661]: pgmap v441: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:01.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:01 smithi104 ceph-mon[34812]: pgmap v441: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:02.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:01 smithi143 ceph-mon[35545]: pgmap v441: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:02.379 INFO:teuthology.orchestra.run.smithi042.stdout:184683593806 2024-05-27T10:16:02.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 -- 172.21.15.42:0/1000088183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4075ad0 msgr2=0x7f5fd4075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 --2- 172.21.15.42:0/1000088183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4075ad0 0x7f5fd4075ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b0a0 tx=0x7f5fcc00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 -- 172.21.15.42:0/1000088183 shutdown_connections 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 --2- 172.21.15.42:0/1000088183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4110a20 0x7f5fd4112e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 --2- 172.21.15.42:0/1000088183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4075ad0 0x7f5fd4075ee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 --2- 172.21.15.42:0/1000088183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fd4076420 0x7f5fd41104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 -- 172.21.15.42:0/1000088183 >> 172.21.15.42:0/1000088183 conn(0x7f5fd4079d00 msgr2=0x7f5fd407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 -- 172.21.15.42:0/1000088183 shutdown_connections 2024-05-27T10:16:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.454+0000 7f5fdae00700 1 -- 172.21.15.42:0/1000088183 wait complete. 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 Processor -- start 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 -- start start 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4075ad0 0x7f5fd419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fd4076420 0x7f5fd419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4110a20 0x7f5fd41a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd41163a0 con 0x7f5fd4110a20 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4116220 con 0x7f5fd4076420 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.456+0000 7f5fdae00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4116520 con 0x7f5fd4075ad0 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4110a20 0x7f5fd41a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4110a20 0x7f5fd41a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36188/0 (socket says 172.21.15.42:36188) 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 -- 172.21.15.42:0/869903800 learned_addr learned my addr 172.21.15.42:0/869903800 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fd9000700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fd4076420 0x7f5fd419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:02.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fd9a00700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4075ad0 0x7f5fd419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 -- 172.21.15.42:0/869903800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4075ad0 msgr2=0x7f5fd419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4075ad0 0x7f5fd419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 -- 172.21.15.42:0/869903800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fd4076420 msgr2=0x7f5fd419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fd4076420 0x7f5fd419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.457+0000 7f5fda400700 1 -- 172.21.15.42:0/869903800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd41a1c20 con 0x7f5fd4110a20 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.458+0000 7f5fda400700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4110a20 0x7f5fd41a1630 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c900 tx=0x7f5fc400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.458+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4007b00 con 0x7f5fd4110a20 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.458+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc4007ca0 con 0x7f5fd4110a20 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.458+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4018660 con 0x7f5fd4110a20 2024-05-27T10:16:02.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.458+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd41a1f10 con 0x7f5fd4110a20 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.458+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd41a2400 con 0x7f5fd4110a20 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.459+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5fd4063f00 con 0x7f5fd4110a20 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.462+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5fc400f450 con 0x7f5fd4110a20 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.463+0000 7f5fd2000700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5fb80b2990 0x7f5fb80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.464+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5fc40e2390 con 0x7f5fd4110a20 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.464+0000 7f5fd9a00700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5fb80b2990 0x7f5fb80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.464+0000 7f5fd2000700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] conn(0x7f5fb80bc4a0 0x7f5fb80be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.464+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 --> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fc4012ec0 con 0x7f5fb80bc4a0 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.464+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5fc40e2740 con 0x7f5fd4110a20 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.464+0000 7f5fd9000700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] conn(0x7f5fb80bc4a0 0x7f5fb80be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.465+0000 7f5fd9a00700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5fb80b2990 0x7f5fb80b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009fd0 tx=0x7f5fcc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.465+0000 7f5fd9000700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] conn(0x7f5fb80bc4a0 0x7f5fb80be990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.466+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== osd.0 v2:172.21.15.42:6802/3193665133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5fc4012ec0 con 0x7f5fb80bc4a0 2024-05-27T10:16:02.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.487+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 --> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5fd4064380 con 0x7f5fb80bc4a0 2024-05-27T10:16:02.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.488+0000 7f5fd2000700 1 -- 172.21.15.42:0/869903800 <== osd.0 v2:172.21.15.42:6802/3193665133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5fd4064380 con 0x7f5fb80bc4a0 2024-05-27T10:16:02.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.488+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] conn(0x7f5fb80bc4a0 msgr2=0x7f5fb80be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:02.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.488+0000 7f5fdae00700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] conn(0x7f5fb80bc4a0 0x7f5fb80be990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5fb80b2990 msgr2=0x7f5fb80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:02.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5fb80b2990 0x7f5fb80b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009fd0 tx=0x7f5fcc00b7b0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4110a20 msgr2=0x7f5fd41a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:02.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 --2- 172.21.15.42:0/869903800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fd4110a20 0x7f5fd41a1630 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c900 tx=0x7f5fc400ccc0 comp rx=0 tx=0).stop 2024-05-27T10:16:02.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fda400700 1 -- 172.21.15.42:0/869903800 reap_dead start 2024-05-27T10:16:02.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 shutdown_connections 2024-05-27T10:16:02.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 >> 172.21.15.42:0/869903800 conn(0x7f5fd4079d00 msgr2=0x7f5fd4101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:02.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 shutdown_connections 2024-05-27T10:16:02.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:02.489+0000 7f5fdae00700 1 -- 172.21.15.42:0/869903800 wait complete. 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 -- 172.21.15.42:0/735197960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d0075b50 msgr2=0x7f10d0075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 --2- 172.21.15.42:0/735197960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d0075b50 0x7f10d0075f60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009990 tx=0x7f10c0009ca0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 -- 172.21.15.42:0/735197960 shutdown_connections 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 --2- 172.21.15.42:0/735197960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d0115360 0x7f10d0117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 --2- 172.21.15.42:0/735197960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d0075b50 0x7f10d0075f60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 --2- 172.21.15.42:0/735197960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d00764a0 0x7f10d0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 -- 172.21.15.42:0/735197960 >> 172.21.15.42:0/735197960 conn(0x7f10d0100030 msgr2=0x7f10d0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 -- 172.21.15.42:0/735197960 shutdown_connections 2024-05-27T10:16:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.814+0000 7f10d6400700 1 -- 172.21.15.42:0/735197960 wait complete. 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.815+0000 7f10d6400700 1 Processor -- start 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.815+0000 7f10d6400700 1 -- start start 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10d6400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d0075b50 0x7f10d01a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10d6400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d00764a0 0x7f10d01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10d6400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0115360 0x7f10d01a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10d6400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f10d011a960 con 0x7f10d00764a0 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10d6400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f10d011a7e0 con 0x7f10d0115360 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10d6400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f10d011aae0 con 0x7f10d0075b50 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10cea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d00764a0 0x7f10d01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10cea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d00764a0 0x7f10d01a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36158/0 (socket says 172.21.15.42:36158) 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10cea00700 1 -- 172.21.15.42:0/2525688127 learned_addr learned my addr 172.21.15.42:0/2525688127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.816+0000 7f10cf400700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d0075b50 0x7f10d01a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cfe00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0115360 0x7f10d01a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cea00700 1 -- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d0075b50 msgr2=0x7f10d01a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cea00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d0075b50 0x7f10d01a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cea00700 1 -- 172.21.15.42:0/2525688127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0115360 msgr2=0x7f10d01a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cea00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0115360 0x7f10d01a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cea00700 1 -- 172.21.15.42:0/2525688127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d01a62c0 con 0x7f10d00764a0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.817+0000 7f10cea00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d00764a0 0x7f10d01a1990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d950 tx=0x7f10bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.818+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc016070 con 0x7f10d00764a0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.818+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10bc010460 con 0x7f10d00764a0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.818+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00f620 con 0x7f10d00764a0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.818+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d01a65b0 con 0x7f10d00764a0 2024-05-27T10:16:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.818+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01aded0 con 0x7f10d00764a0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.819+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10d0063ec0 con 0x7f10d00764a0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.823+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f10bc019070 con 0x7f10d00764a0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.825+0000 7f10c7e00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10b40b2910 0x7f10b40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.825+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f10bc0e2510 con 0x7f10d00764a0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.825+0000 7f10cf400700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10b40b2910 0x7f10b40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.825+0000 7f10c7e00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] conn(0x7f10b40bc420 0x7f10b40be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.826+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 --> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10b40befe0 con 0x7f10b40bc420 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.826+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f10bc0e28c0 con 0x7f10d00764a0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.826+0000 7f10cf400700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10b40b2910 0x7f10b40b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b5c0 tx=0x7f10c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.826+0000 7f10cfe00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] conn(0x7f10b40bc420 0x7f10b40be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.827+0000 7f10cfe00700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] conn(0x7f10b40bc420 0x7f10b40be910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.830+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== osd.11 v2:172.21.15.143:6824/1326686581 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b40befe0 con 0x7f10b40bc420 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.848+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 --> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10d0064340 con 0x7f10b40bc420 2024-05-27T10:16:03.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.849+0000 7f10c7e00700 1 -- 172.21.15.42:0/2525688127 <== osd.11 v2:172.21.15.143:6824/1326686581 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f10d0064340 con 0x7f10b40bc420 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.850+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] conn(0x7f10b40bc420 msgr2=0x7f10b40be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.850+0000 7f10d6400700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] conn(0x7f10b40bc420 0x7f10b40be910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10b40b2910 msgr2=0x7f10b40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f10b40b2910 0x7f10b40b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b5c0 tx=0x7f10c0005fb0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d00764a0 msgr2=0x7f10d01a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 --2- 172.21.15.42:0/2525688127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10d00764a0 0x7f10d01a1990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d950 tx=0x7f10bc00dd10 comp rx=0 tx=0).stop 2024-05-27T10:16:03.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10cfe00700 1 -- 172.21.15.42:0/2525688127 reap_dead start 2024-05-27T10:16:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 shutdown_connections 2024-05-27T10:16:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 >> 172.21.15.42:0/2525688127 conn(0x7f10d0100030 msgr2=0x7f10d0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 shutdown_connections 2024-05-27T10:16:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:00.851+0000 7f10d6400700 1 -- 172.21.15.42:0/2525688127 wait complete. 2024-05-27T10:16:03.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:03 smithi042 ceph-mon[31661]: pgmap v442: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:03.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:03 smithi104 ceph-mon[34812]: pgmap v442: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:03.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1887713611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94076440 msgr2=0x7fcf941106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1887713611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94076440 0x7fcf941106d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009990 tx=0x7fcf80009ca0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1887713611 shutdown_connections 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1887713611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf94110c10 0x7fcf94113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1887713611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf94075af0 0x7fcf94075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1887713611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94076440 0x7fcf941106d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1887713611 >> 172.21.15.42:0/1887713611 conn(0x7fcf9407af60 msgr2=0x7fcf9407b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1887713611 shutdown_connections 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.964+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1887713611 wait complete. 2024-05-27T10:16:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.966+0000 7fcf9ba00700 1 Processor -- start 2024-05-27T10:16:03.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.966+0000 7fcf9ba00700 1 -- start start 2024-05-27T10:16:03.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf9ba00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf94075af0 0x7fcf9419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf9ba00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf94076440 0x7fcf9419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf9ba00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94110c10 0x7fcf941a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf9ba00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94116570 con 0x7fcf94110c10 2024-05-27T10:16:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf9ba00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941163f0 con 0x7fcf94076440 2024-05-27T10:16:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf9ba00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941166f0 con 0x7fcf94075af0 2024-05-27T10:16:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf99a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94110c10 0x7fcf941a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf99a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94110c10 0x7fcf941a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45364/0 (socket says 172.21.15.42:45364) 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf99a00700 1 -- 172.21.15.42:0/1493685267 learned_addr learned my addr 172.21.15.42:0/1493685267 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.967+0000 7fcf99000700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf94075af0 0x7fcf9419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf93e00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf94076440 0x7fcf9419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf99a00700 1 -- 172.21.15.42:0/1493685267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf94075af0 msgr2=0x7fcf9419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf99a00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf94075af0 0x7fcf9419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf99a00700 1 -- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf94076440 msgr2=0x7fcf9419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf99a00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf94076440 0x7fcf9419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf99a00700 1 -- 172.21.15.42:0/1493685267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf941a1e90 con 0x7fcf94110c10 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf93e00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf94076440 0x7fcf9419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.968+0000 7fcf99a00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94110c10 0x7fcf941a17f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c007ee0 tx=0x7fcf8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.969+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00dbd0 con 0x7fcf94110c10 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.969+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf8c00dd70 con 0x7fcf94110c10 2024-05-27T10:16:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.969+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf941a2180 con 0x7fcf94110c10 2024-05-27T10:16:03.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.969+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0058d0 con 0x7fcf94110c10 2024-05-27T10:16:03.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.969+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a9aa0 con 0x7fcf94110c10 2024-05-27T10:16:03.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.970+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf74001030 con 0x7fcf94110c10 2024-05-27T10:16:03.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.973+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcf8c01a020 con 0x7fcf94110c10 2024-05-27T10:16:03.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.976+0000 7fcf91600700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fcf7c0b2910 0x7fcf7c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.976+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcf8c0e2430 con 0x7fcf94110c10 2024-05-27T10:16:03.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.976+0000 7fcf91600700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] conn(0x7fcf7c0bc420 0x7fcf7c0be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:03.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.976+0000 7fcf99000700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fcf7c0b2910 0x7fcf7c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.976+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 --> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf7c0befe0 con 0x7fcf7c0bc420 2024-05-27T10:16:03.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.976+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fcf8c0e2880 con 0x7fcf94110c10 2024-05-27T10:16:03.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.977+0000 7fcf93e00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] conn(0x7fcf7c0bc420 0x7fcf7c0be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:03.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.977+0000 7fcf99000700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fcf7c0b2910 0x7fcf7c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005950 tx=0x7fcf8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:03.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.978+0000 7fcf93e00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] conn(0x7fcf7c0bc420 0x7fcf7c0be910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:03.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.981+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== osd.1 v2:172.21.15.104:6800/2575126270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf7c0befe0 con 0x7fcf7c0bc420 2024-05-27T10:16:04.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.000+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 --> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf74002e60 con 0x7fcf7c0bc420 2024-05-27T10:16:04.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.001+0000 7fcf91600700 1 -- 172.21.15.42:0/1493685267 <== osd.1 v2:172.21.15.104:6800/2575126270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcf74002e60 con 0x7fcf7c0bc420 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.001+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] conn(0x7fcf7c0bc420 msgr2=0x7fcf7c0be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.001+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] conn(0x7fcf7c0bc420 0x7fcf7c0be910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fcf7c0b2910 msgr2=0x7fcf7c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fcf7c0b2910 0x7fcf7c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005950 tx=0x7fcf8800b410 comp rx=0 tx=0).stop 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94110c10 msgr2=0x7fcf941a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 --2- 172.21.15.42:0/1493685267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf94110c10 0x7fcf941a17f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c007ee0 tx=0x7fcf8c00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf99a00700 1 -- 172.21.15.42:0/1493685267 reap_dead start 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 shutdown_connections 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 >> 172.21.15.42:0/1493685267 conn(0x7fcf9407af60 msgr2=0x7fcf941011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 shutdown_connections 2024-05-27T10:16:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.002+0000 7fcf9ba00700 1 -- 172.21.15.42:0/1493685267 wait complete. 2024-05-27T10:16:04.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:03 smithi143 ceph-mon[35545]: pgmap v442: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:05 smithi104 ceph-mon[34812]: pgmap v443: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:16:05.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.970+0000 7fce82800700 1 -- 172.21.15.42:0/4114457984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 msgr2=0x7fce7c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.970+0000 7fce82800700 1 --2- 172.21.15.42:0/4114457984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c104820 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce700099c0 tx=0x7fce70009cd0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 -- 172.21.15.42:0/4114457984 shutdown_connections 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 --2- 172.21.15.42:0/4114457984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce7c109690 0x7fce7c110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 --2- 172.21.15.42:0/4114457984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c104820 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 --2- 172.21.15.42:0/4114457984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce7c108d40 0x7fce7c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 -- 172.21.15.42:0/4114457984 >> 172.21.15.42:0/4114457984 conn(0x7fce7c07aca0 msgr2=0x7fce7c07b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 -- 172.21.15.42:0/4114457984 shutdown_connections 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.971+0000 7fce82800700 1 -- 172.21.15.42:0/4114457984 wait complete. 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.972+0000 7fce82800700 1 Processor -- start 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 -- start start 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce7c108d40 0x7fce7c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce7c109690 0x7fce7c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c1141f0 con 0x7fce7c104410 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c114070 con 0x7fce7c109690 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce82800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c114370 con 0x7fce7c108d40 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45378/0 (socket says 172.21.15.42:45378) 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.973+0000 7fce81400700 1 -- 172.21.15.42:0/1269126343 learned_addr learned my addr 172.21.15.42:0/1269126343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce81e00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce7c109690 0x7fce7c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce80a00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce7c108d40 0x7fce7c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce81400700 1 -- 172.21.15.42:0/1269126343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce7c108d40 msgr2=0x7fce7c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce81400700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce7c108d40 0x7fce7c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce81400700 1 -- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce7c109690 msgr2=0x7fce7c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce81400700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce7c109690 0x7fce7c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce81400700 1 -- 172.21.15.42:0/1269126343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c1a1c60 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.974+0000 7fce80a00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce7c108d40 0x7fce7c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.975+0000 7fce81400700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c19ce40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce70000c00 tx=0x7fce7000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.975+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70026070 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.975+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce70005030 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.975+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004b70 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.975+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c1a1ef0 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.975+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c1a2420 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.977+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce7c101f90 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.979+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fce70029070 con 0x7fce7c104410 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.980+0000 7fce7a000700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fce640b2990 0x7fce640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.981+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fce700e3830 con 0x7fce7c104410 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.981+0000 7fce80a00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fce640b2990 0x7fce640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.981+0000 7fce7a000700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] conn(0x7fce640bc4a0 0x7fce640be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.981+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 --> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce7000fe50 con 0x7fce640bc4a0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.981+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fce700e3be0 con 0x7fce7c104410 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.982+0000 7fce80a00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fce640b2990 0x7fce640b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce68005fd0 tx=0x7fce68005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.982+0000 7fce81e00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] conn(0x7fce640bc4a0 0x7fce640be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.983+0000 7fce81e00700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] conn(0x7fce640bc4a0 0x7fce640be990 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:04.985+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== osd.7 v2:172.21.15.104:6816/2687473765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce7000fe50 con 0x7fce640bc4a0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.003+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 --> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce7c1058c0 con 0x7fce640bc4a0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.004+0000 7fce7a000700 1 -- 172.21.15.42:0/1269126343 <== osd.7 v2:172.21.15.104:6816/2687473765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fce7c1058c0 con 0x7fce640bc4a0 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] conn(0x7fce640bc4a0 msgr2=0x7fce640be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce82800700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] conn(0x7fce640bc4a0 0x7fce640be990 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fce640b2990 msgr2=0x7fce640b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce82800700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fce640b2990 0x7fce640b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce68005fd0 tx=0x7fce68005e30 comp rx=0 tx=0).stop 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 msgr2=0x7fce7c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce82800700 1 --2- 172.21.15.42:0/1269126343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce7c104410 0x7fce7c19ce40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce70000c00 tx=0x7fce7000f690 comp rx=0 tx=0).stop 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.005+0000 7fce81e00700 1 -- 172.21.15.42:0/1269126343 reap_dead start 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.006+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 shutdown_connections 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.006+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 >> 172.21.15.42:0/1269126343 conn(0x7fce7c07aca0 msgr2=0x7fce7c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.006+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 shutdown_connections 2024-05-27T10:16:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.006+0000 7fce82800700 1 -- 172.21.15.42:0/1269126343 wait complete. 2024-05-27T10:16:05.972 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:05 smithi042 ceph-mon[31661]: pgmap v443: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:16:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:05 smithi143 ceph-mon[35545]: pgmap v443: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 -- 172.21.15.42:0/2809664386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 msgr2=0x7f5e54106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/2809664386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e54106970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009990 tx=0x7f5e40009ca0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 -- 172.21.15.42:0/2809664386 shutdown_connections 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/2809664386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54106eb0 0x7f5e54113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/2809664386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410c5a0 0x7f5e5410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/2809664386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e54106970 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 -- 172.21.15.42:0/2809664386 >> 172.21.15.42:0/2809664386 conn(0x7f5e541020c0 msgr2=0x7f5e54104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 -- 172.21.15.42:0/2809664386 shutdown_connections 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.074+0000 7f5e5bc00700 1 -- 172.21.15.42:0/2809664386 wait complete. 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.076+0000 7f5e5bc00700 1 Processor -- start 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.076+0000 7f5e5bc00700 1 -- start start 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.076+0000 7f5e5bc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e540753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.076+0000 7f5e5bc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e54106eb0 0x7f5e5406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.076+0000 7f5e5bc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5410c5a0 0x7f5e5406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e5bc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54116580 con 0x7f5e54106560 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e5bc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54116400 con 0x7f5e54106eb0 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e5bc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54116700 con 0x7f5e5410c5a0 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e59200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e540753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:06.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e59200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e540753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45400/0 (socket says 172.21.15.42:45400) 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e59200700 1 -- 172.21.15.42:0/808794402 learned_addr learned my addr 172.21.15.42:0/808794402 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e53e00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e54106eb0 0x7f5e5406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e59c00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5410c5a0 0x7f5e5406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e59200700 1 -- 172.21.15.42:0/808794402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5410c5a0 msgr2=0x7f5e5406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.077+0000 7f5e59200700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5410c5a0 0x7f5e5406eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e59200700 1 -- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e54106eb0 msgr2=0x7f5e5406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e59200700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e54106eb0 0x7f5e5406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e59200700 1 -- 172.21.15.42:0/808794402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e54072ee0 con 0x7f5e54106560 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e59200700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e540753b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800d980 tx=0x7f5e4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48017070 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e48009dc0 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800f3c0 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.078+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e540731d0 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.079+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e54073690 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.080+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e54073c40 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.083+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5e48009930 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.085+0000 7f5e51600700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5e3c0b2910 0x7f5e3c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.085+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5e480e2210 con 0x7f5e54106560 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.085+0000 7f5e53e00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5e3c0b2910 0x7f5e3c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.085+0000 7f5e51600700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] conn(0x7f5e3c0bc420 0x7f5e3c0be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.085+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 --> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e4800fec0 con 0x7f5e3c0bc420 2024-05-27T10:16:06.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.085+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5e480e2660 con 0x7f5e54106560 2024-05-27T10:16:06.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.086+0000 7f5e59c00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] conn(0x7f5e3c0bc420 0x7f5e3c0be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:06.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.086+0000 7f5e53e00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5e3c0b2910 0x7f5e3c0b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e40006010 tx=0x7f5e4000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:06.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.087+0000 7f5e59c00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] conn(0x7f5e3c0bc420 0x7f5e3c0be910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:06.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.090+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== osd.10 v2:172.21.15.104:6824/1167441183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e4800fec0 con 0x7f5e3c0bc420 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.109+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 --> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e54063ec0 con 0x7f5e3c0bc420 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.110+0000 7f5e51600700 1 -- 172.21.15.42:0/808794402 <== osd.10 v2:172.21.15.104:6824/1167441183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5e54063ec0 con 0x7f5e3c0bc420 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.111+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] conn(0x7f5e3c0bc420 msgr2=0x7f5e3c0be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.111+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] conn(0x7f5e3c0bc420 0x7f5e3c0be910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.111+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5e3c0b2910 msgr2=0x7f5e3c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.111+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5e3c0b2910 0x7f5e3c0b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e40006010 tx=0x7f5e4000b540 comp rx=0 tx=0).stop 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.111+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 msgr2=0x7f5e540753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.111+0000 7f5e5bc00700 1 --2- 172.21.15.42:0/808794402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e54106560 0x7f5e540753b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800d980 tx=0x7f5e4800dc90 comp rx=0 tx=0).stop 2024-05-27T10:16:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.112+0000 7f5e59c00700 1 -- 172.21.15.42:0/808794402 reap_dead start 2024-05-27T10:16:06.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.112+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 shutdown_connections 2024-05-27T10:16:06.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.112+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 >> 172.21.15.42:0/808794402 conn(0x7f5e541020c0 msgr2=0x7f5e54104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:06.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.112+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 shutdown_connections 2024-05-27T10:16:06.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:05.112+0000 7f5e5bc00700 1 -- 172.21.15.42:0/808794402 wait complete. 2024-05-27T10:16:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.893+0000 7f6557e00700 1 -- 172.21.15.42:0/1902596217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65581043f0 msgr2=0x7f6558104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.893+0000 7f6557e00700 1 --2- 172.21.15.42:0/1902596217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65581043f0 0x7f6558104800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65480099c0 tx=0x7f6548009cd0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 -- 172.21.15.42:0/1902596217 shutdown_connections 2024-05-27T10:16:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 --2- 172.21.15.42:0/1902596217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558109670 0x7f6558110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 --2- 172.21.15.42:0/1902596217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65581043f0 0x7f6558104800 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 --2- 172.21.15.42:0/1902596217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6558108d20 0x7f6558109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 -- 172.21.15.42:0/1902596217 >> 172.21.15.42:0/1902596217 conn(0x7f6558079d00 msgr2=0x7f655807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 -- 172.21.15.42:0/1902596217 shutdown_connections 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.894+0000 7f6557e00700 1 -- 172.21.15.42:0/1902596217 wait complete. 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.895+0000 7f6557e00700 1 Processor -- start 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.895+0000 7f6557e00700 1 -- start start 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.895+0000 7f6557e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65581043f0 0x7f655819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.895+0000 7f6557e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558108d20 0x7f655819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6558109670 0x7f65581a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f65581141a0 con 0x7f6558109670 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6558114020 con 0x7f6558108d20 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6558114320 con 0x7f65581043f0 2024-05-27T10:16:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6558109670 0x7f65581a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6558109670 0x7f65581a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45442/0 (socket says 172.21.15.42:45442) 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 -- 172.21.15.42:0/470849803 learned_addr learned my addr 172.21.15.42:0/470849803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6556a00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65581043f0 0x7f655819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6556000700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558108d20 0x7f655819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 -- 172.21.15.42:0/470849803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65581043f0 msgr2=0x7f655819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65581043f0 0x7f655819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 -- 172.21.15.42:0/470849803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558108d20 msgr2=0x7f655819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558108d20 0x7f655819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6557400700 1 -- 172.21.15.42:0/470849803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65581a1c50 con 0x7f6558109670 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.896+0000 7f6556a00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65581043f0 0x7f655819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.897+0000 7f6557400700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6558109670 0x7f65581a1660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6550007d70 tx=0x7f655000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.897+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550016020 con 0x7f6558109670 2024-05-27T10:16:07.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.897+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655000ad40 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.897+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65581a1f40 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.897+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500115e0 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.897+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65581a2480 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.898+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65581025d0 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.901+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6550019020 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.902+0000 7f655ca00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f653c0b2990 0x7f653c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.903+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f65500e2770 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.903+0000 7f6556a00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f653c0b2990 0x7f653c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.903+0000 7f655ca00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] conn(0x7f653c0bc4a0 0x7f653c0be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.903+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 --> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f653c0bf060 con 0x7f653c0bc4a0 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.903+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f65500e2b20 con 0x7f6558109670 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.903+0000 7f6556000700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] conn(0x7f653c0bc4a0 0x7f653c0be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:07.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.904+0000 7f6556a00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f653c0b2990 0x7f653c0b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6548000c00 tx=0x7f6548005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.904+0000 7f6556000700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] conn(0x7f653c0bc4a0 0x7f653c0be990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.905+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== osd.9 v2:172.21.15.42:6826/2335274703 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f653c0bf060 con 0x7f653c0bc4a0 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.924+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 --> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65581058a0 con 0x7f653c0bc4a0 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.924+0000 7f655ca00700 1 -- 172.21.15.42:0/470849803 <== osd.9 v2:172.21.15.42:6826/2335274703 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f65581058a0 con 0x7f653c0bc4a0 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] conn(0x7f653c0bc4a0 msgr2=0x7f653c0be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] conn(0x7f653c0bc4a0 0x7f653c0be990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f653c0b2990 msgr2=0x7f653c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f653c0b2990 0x7f653c0b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6548000c00 tx=0x7f6548005fb0 comp rx=0 tx=0).stop 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6558109670 msgr2=0x7f65581a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 --2- 172.21.15.42:0/470849803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6558109670 0x7f65581a1660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6550007d70 tx=0x7f655000a340 comp rx=0 tx=0).stop 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557400700 1 -- 172.21.15.42:0/470849803 reap_dead start 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 shutdown_connections 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 >> 172.21.15.42:0/470849803 conn(0x7f6558079d00 msgr2=0x7f6558101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 shutdown_connections 2024-05-27T10:16:07.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.925+0000 7f6557e00700 1 -- 172.21.15.42:0/470849803 wait complete. 2024-05-27T10:16:07.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:07 smithi104 ceph-mon[34812]: pgmap v444: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T10:16:07.964 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:07 smithi042 ceph-mon[31661]: pgmap v444: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T10:16:08.025 INFO:teuthology.orchestra.run.smithi042.stdout:34359738491 2024-05-27T10:16:08.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-27T10:16:08.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:07 smithi143 ceph-mon[35545]: pgmap v444: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T10:16:09.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:09 smithi042 ceph-mon[31661]: pgmap v445: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:09.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:09 smithi104 ceph-mon[34812]: pgmap v445: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:09 smithi143 ceph-mon[35545]: pgmap v445: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout:55834574964 2024-05-27T10:16:10.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-27T10:16:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 -- 172.21.15.42:0/321692894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 msgr2=0x7fbb1c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 --2- 172.21.15.42:0/321692894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1111c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b0d0 tx=0x7fbb1400b3e0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 -- 172.21.15.42:0/321692894 shutdown_connections 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 --2- 172.21.15.42:0/321692894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1111c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 --2- 172.21.15.42:0/321692894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb1c1045e0 0x7fbb1c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 --2- 172.21.15.42:0/321692894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb1c108f10 0x7fbb1c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 -- 172.21.15.42:0/321692894 >> 172.21.15.42:0/321692894 conn(0x7fbb1c100030 msgr2=0x7fbb1c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 -- 172.21.15.42:0/321692894 shutdown_connections 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.009+0000 7fbb23400700 1 -- 172.21.15.42:0/321692894 wait complete. 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.010+0000 7fbb23400700 1 Processor -- start 2024-05-27T10:16:10.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 -- start start 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb1c1045e0 0x7fbb1c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb1c108f10 0x7fbb1c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c114390 con 0x7fbb1c109860 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c114210 con 0x7fbb1c1045e0 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.011+0000 7fbb23400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c114510 con 0x7fbb1c108f10 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45430/0 (socket says 172.21.15.42:45430) 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 -- 172.21.15.42:0/564078352 learned_addr learned my addr 172.21.15.42:0/564078352 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb20a00700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb1c1045e0 0x7fbb1c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb1be00700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb1c108f10 0x7fbb1c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 -- 172.21.15.42:0/564078352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb1c108f10 msgr2=0x7fbb1c19d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb1c108f10 0x7fbb1c19d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 -- 172.21.15.42:0/564078352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb1c1045e0 msgr2=0x7fbb1c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb1c1045e0 0x7fbb1c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.012+0000 7fbb21400700 1 -- 172.21.15.42:0/564078352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c1a1e50 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.013+0000 7fbb21400700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1a17b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007d10 tx=0x7fbb14007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.013+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1401f070 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.013+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb14003e10 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.013+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1401b720 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.013+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c1a20e0 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.013+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1a9a60 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.014+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb1c1a2610 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.017+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbb14022070 con 0x7fbb1c109860 2024-05-27T10:16:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.018+0000 7fbb19600700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbb040b2990 0x7fbb040b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.018+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbb1400f020 con 0x7fbb1c109860 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.019+0000 7fbb20a00700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbb040b2990 0x7fbb040b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.019+0000 7fbb19600700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] conn(0x7fbb040bc500 0x7fbb040be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.019+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 --> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb140075c0 con 0x7fbb040bc500 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.019+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fbb140e3d50 con 0x7fbb1c109860 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.019+0000 7fbb1be00700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] conn(0x7fbb040bc500 0x7fbb040be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.019+0000 7fbb20a00700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbb040b2990 0x7fbb040b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005950 tx=0x7fbb1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.020+0000 7fbb1be00700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] conn(0x7fbb040bc500 0x7fbb040be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.021+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== osd.3 v2:172.21.15.42:6810/2776846288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb140075c0 con 0x7fbb040bc500 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.039+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 --> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb1c108320 con 0x7fbb040bc500 2024-05-27T10:16:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.040+0000 7fbb19600700 1 -- 172.21.15.42:0/564078352 <== osd.3 v2:172.21.15.42:6810/2776846288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb1c108320 con 0x7fbb040bc500 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.040+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] conn(0x7fbb040bc500 msgr2=0x7fbb040be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] conn(0x7fbb040bc500 0x7fbb040be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbb040b2990 msgr2=0x7fbb040b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbb040b2990 0x7fbb040b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005950 tx=0x7fbb1000b410 comp rx=0 tx=0).stop 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 msgr2=0x7fbb1c1a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 --2- 172.21.15.42:0/564078352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb1c109860 0x7fbb1c1a17b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007d10 tx=0x7fbb14007d40 comp rx=0 tx=0).stop 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb21400700 1 -- 172.21.15.42:0/564078352 reap_dead start 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 shutdown_connections 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 >> 172.21.15.42:0/564078352 conn(0x7fbb1c100030 msgr2=0x7fbb1c110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 shutdown_connections 2024-05-27T10:16:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:06.041+0000 7fbb23400700 1 -- 172.21.15.42:0/564078352 wait complete. 2024-05-27T10:16:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 -- 172.21.15.42:0/677297322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a40764a0 msgr2=0x7f67a4114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 --2- 172.21.15.42:0/677297322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a40764a0 0x7f67a4114e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6790009990 tx=0x7f6790009ca0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 -- 172.21.15.42:0/677297322 shutdown_connections 2024-05-27T10:16:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 --2- 172.21.15.42:0/677297322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a4115360 0x7f67a4117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 --2- 172.21.15.42:0/677297322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4075b50 0x7f67a4075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 --2- 172.21.15.42:0/677297322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a40764a0 0x7f67a4114e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 -- 172.21.15.42:0/677297322 >> 172.21.15.42:0/677297322 conn(0x7f67a4100030 msgr2=0x7f67a4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.870+0000 7f67a8c00700 1 -- 172.21.15.42:0/677297322 shutdown_connections 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.871+0000 7f67a8c00700 1 -- 172.21.15.42:0/677297322 wait complete. 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.872+0000 7f67a8c00700 1 Processor -- start 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.872+0000 7f67a8c00700 1 -- start start 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a8c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a4075b50 0x7f67a41a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a8c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a40764a0 0x7f67a41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a4115360 0x7f67a41a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a8c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67a411a910 con 0x7f67a4115360 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a8c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f67a411a790 con 0x7f67a40764a0 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a8c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f67a411aa90 con 0x7f67a4075b50 2024-05-27T10:16:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a2800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a4115360 0x7f67a41a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a2800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a4115360 0x7f67a41a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45346/0 (socket says 172.21.15.42:45346) 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a2800700 1 -- 172.21.15.42:0/3555352343 learned_addr learned my addr 172.21.15.42:0/3555352343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a1e00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a4075b50 0x7f67a41a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.873+0000 7f67a1400700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a40764a0 0x7f67a41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a2800700 1 -- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a4075b50 msgr2=0x7f67a41a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a2800700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a4075b50 0x7f67a41a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a2800700 1 -- 172.21.15.42:0/3555352343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a40764a0 msgr2=0x7f67a41a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a2800700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a40764a0 0x7f67a41a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a2800700 1 -- 172.21.15.42:0/3555352343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a41a61d0 con 0x7f67a4115360 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a2800700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a4115360 0x7f67a41a5b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f679c007ee0 tx=0x7f679c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c00dbd0 con 0x7f67a4115360 2024-05-27T10:16:10.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f679c00dd70 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41a64c0 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.874+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0058d0 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.875+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41adde0 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.876+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67a4063f40 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.878+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f679c01a020 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.880+0000 7f679aa00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f67880b2910 0x7f67880b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.880+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f679c0e25f0 con 0x7f67a4115360 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.880+0000 7f67a1e00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f67880b2910 0x7f67880b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.880+0000 7f679aa00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] conn(0x7f67880bc420 0x7f67880be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.881+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 --> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f679c00be50 con 0x7f67880bc420 2024-05-27T10:16:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.881+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f679c0e29a0 con 0x7f67a4115360 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.881+0000 7f67a1400700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] conn(0x7f67880bc420 0x7f67880be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.881+0000 7f67a1e00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f67880b2910 0x7f67880b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6794005950 tx=0x7f67940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.882+0000 7f67a1400700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] conn(0x7f67880bc420 0x7f67880be910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.885+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== osd.8 v2:172.21.15.143:6816/3963187375 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f679c00be50 con 0x7f67880bc420 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.904+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 --> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67a4064340 con 0x7f67880bc420 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.905+0000 7f679aa00700 1 -- 172.21.15.42:0/3555352343 <== osd.8 v2:172.21.15.143:6816/3963187375 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f67a4064340 con 0x7f67880bc420 2024-05-27T10:16:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] conn(0x7f67880bc420 msgr2=0x7f67880be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] conn(0x7f67880bc420 0x7f67880be910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f67880b2910 msgr2=0x7f67880b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f67880b2910 0x7f67880b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6794005950 tx=0x7f67940058e0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a4115360 msgr2=0x7f67a41a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 --2- 172.21.15.42:0/3555352343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67a4115360 0x7f67a41a5b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f679c007ee0 tx=0x7f679c00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a2800700 1 -- 172.21.15.42:0/3555352343 reap_dead start 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 shutdown_connections 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 >> 172.21.15.42:0/3555352343 conn(0x7f67a4100030 msgr2=0x7f67a41006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 shutdown_connections 2024-05-27T10:16:10.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:03.906+0000 7f67a8c00700 1 -- 172.21.15.42:0/3555352343 wait complete. 2024-05-27T10:16:11.812 INFO:teuthology.orchestra.run.smithi042.stdout:206158430280 2024-05-27T10:16:11.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-27T10:16:11.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:11 smithi042 ceph-mon[31661]: pgmap v446: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:11.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:11 smithi042 ceph-mon[31661]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-27T10:16:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:11 smithi143 ceph-mon[35545]: pgmap v446: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:11 smithi143 ceph-mon[35545]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-27T10:16:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:11 smithi104 ceph-mon[34812]: pgmap v446: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:11 smithi104 ceph-mon[34812]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-27T10:16:12.599 INFO:teuthology.orchestra.run.smithi042.stdout:279172874290 2024-05-27T10:16:12.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-27T10:16:12.993 INFO:teuthology.orchestra.run.smithi042.stdout:257698037817 2024-05-27T10:16:12.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-27T10:16:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:111669149799 2024-05-27T10:16:13.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-27T10:16:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 -- 172.21.15.42:0/4251465178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 msgr2=0x7f44f4113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 --2- 172.21.15.42:0/4251465178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f4113340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00b0a0 tx=0x7f44ec00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 -- 172.21.15.42:0/4251465178 shutdown_connections 2024-05-27T10:16:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 --2- 172.21.15.42:0/4251465178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f4113340 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 --2- 172.21.15.42:0/4251465178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f410c5a0 0x7f44f410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 --2- 172.21.15.42:0/4251465178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44f4106560 0x7f44f4106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 -- 172.21.15.42:0/4251465178 >> 172.21.15.42:0/4251465178 conn(0x7f44f4102080 msgr2=0x7f44f41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.495+0000 7f44f9e00700 1 -- 172.21.15.42:0/4251465178 shutdown_connections 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.496+0000 7f44f9e00700 1 -- 172.21.15.42:0/4251465178 wait complete. 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 Processor -- start 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 -- start start 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f4106560 0x7f44f419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44f410c5a0 0x7f44f41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41165d0 con 0x7f44f4106eb0 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4116450 con 0x7f44f4106560 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.497+0000 7f44f9e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4116750 con 0x7f44f410c5a0 2024-05-27T10:16:13.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53596/0 (socket says 172.21.15.42:53596) 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 -- 172.21.15.42:0/1242988701 learned_addr learned my addr 172.21.15.42:0/1242988701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2a00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f4106560 0x7f44f419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f3400700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44f410c5a0 0x7f44f41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 -- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44f410c5a0 msgr2=0x7f44f41a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44f410c5a0 0x7f44f41a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 -- 172.21.15.42:0/1242988701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f4106560 msgr2=0x7f44f419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f4106560 0x7f44f419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.498+0000 7f44f2000700 1 -- 172.21.15.42:0/1242988701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f41a1d90 con 0x7f44f4106eb0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.499+0000 7f44f2000700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f419d460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c8c0 tx=0x7f44e000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.499+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0004220 con 0x7f44f4106eb0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.499+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f41a2020 con 0x7f44f4106eb0 2024-05-27T10:16:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.499+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44e00043c0 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.499+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0003680 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.499+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f41a99a0 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.500+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44f4063f00 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.504+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f44e0004750 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.505+0000 7f44f8a00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f44d80b2900 0x7f44d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.505+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f44e00e2070 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.505+0000 7f44f8a00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] conn(0x7f44d80bc410 0x7f44d80be900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.506+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 --> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44e0010e80 con 0x7f44d80bc410 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.506+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f44e00e24c0 con 0x7f44f4106eb0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.506+0000 7f44f2a00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f44d80b2900 0x7f44d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.506+0000 7f44f3400700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] conn(0x7f44d80bc410 0x7f44d80be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.507+0000 7f44f2a00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f44d80b2900 0x7f44d80b4e10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005950 tx=0x7f44e400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.507+0000 7f44f3400700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] conn(0x7f44d80bc410 0x7f44d80be900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.510+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== osd.2 v2:172.21.15.143:6800/793524838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44e0010e80 con 0x7f44d80bc410 2024-05-27T10:16:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.529+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 --> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44f4064300 con 0x7f44d80bc410 2024-05-27T10:16:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f8a00700 1 -- 172.21.15.42:0/1242988701 <== osd.2 v2:172.21.15.143:6800/793524838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f44f4064300 con 0x7f44d80bc410 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] conn(0x7f44d80bc410 msgr2=0x7f44d80be900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] conn(0x7f44d80bc410 0x7f44d80be900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f44d80b2900 msgr2=0x7f44d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f44d80b2900 0x7f44d80b4e10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005950 tx=0x7f44e400a300 comp rx=0 tx=0).stop 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 msgr2=0x7f44f419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 --2- 172.21.15.42:0/1242988701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44f4106eb0 0x7f44f419d460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c8c0 tx=0x7f44e000cbd0 comp rx=0 tx=0).stop 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f3400700 1 -- 172.21.15.42:0/1242988701 reap_dead start 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 shutdown_connections 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.530+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 >> 172.21.15.42:0/1242988701 conn(0x7f44f4102080 msgr2=0x7f44f41047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.531+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 shutdown_connections 2024-05-27T10:16:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:13.531+0000 7f44f9e00700 1 -- 172.21.15.42:0/1242988701 wait complete. 2024-05-27T10:16:13.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:13 smithi042 ceph-mon[31661]: pgmap v447: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-27T10:16:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:13 smithi143 ceph-mon[35545]: pgmap v447: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-27T10:16:14.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:13 smithi104 ceph-mon[34812]: pgmap v447: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-27T10:16:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:77309411441 2024-05-27T10:16:14.595 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-27T10:16:14.783 INFO:teuthology.orchestra.run.smithi042.stdout:231928234049 2024-05-27T10:16:14.783 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-27T10:16:15.517 INFO:teuthology.orchestra.run.smithi042.stdout:304942678057 2024-05-27T10:16:15.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-27T10:16:16.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:15 smithi143 ceph-mon[35545]: pgmap v448: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-27T10:16:16.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:15 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:16:16.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:15 smithi042 ceph-mon[31661]: pgmap v448: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-27T10:16:16.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:15 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:16:16.190 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:15 smithi104 ceph-mon[34812]: pgmap v448: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-27T10:16:16.191 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:15 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:16:16.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 -- 172.21.15.42:0/1347965516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 msgr2=0x7f2440110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 --2- 172.21.15.42:0/1347965516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f2440110d70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2430009990 tx=0x7f2430009ca0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 -- 172.21.15.42:0/1347965516 shutdown_connections 2024-05-27T10:16:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 --2- 172.21.15.42:0/1347965516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244010b2d0 0x7f2440117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 --2- 172.21.15.42:0/1347965516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f2440110d70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 --2- 172.21.15.42:0/1347965516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f244010a980 0x7f244010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 -- 172.21.15.42:0/1347965516 >> 172.21.15.42:0/1347965516 conn(0x7f2440077c70 msgr2=0x7f24400780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 -- 172.21.15.42:0/1347965516 shutdown_connections 2024-05-27T10:16:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.503+0000 7f2446600700 1 -- 172.21.15.42:0/1347965516 wait complete. 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.504+0000 7f2446600700 1 Processor -- start 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 -- start start 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244010a980 0x7f24401a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f244010b2d0 0x7f24401a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f24401a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f244011a920 con 0x7f2440110960 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f244011a7a0 con 0x7f244010b2d0 2024-05-27T10:16:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.505+0000 7f2446600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f244011aaa0 con 0x7f244010a980 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f24401a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f24401a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53620/0 (socket says 172.21.15.42:53620) 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 -- 172.21.15.42:0/3679317404 learned_addr learned my addr 172.21.15.42:0/3679317404 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243f400700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244010a980 0x7f24401a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f2437e00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f244010b2d0 0x7f24401a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 -- 172.21.15.42:0/3679317404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244010a980 msgr2=0x7f24401a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244010a980 0x7f24401a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 -- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f244010b2d0 msgr2=0x7f24401a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f244010b2d0 0x7f24401a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243fe00700 1 -- 172.21.15.42:0/3679317404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24401a62a0 con 0x7f2440110960 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f243f400700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244010a980 0x7f24401a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.506+0000 7f2437e00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f244010b2d0 0x7f24401a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.507+0000 7f243fe00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f24401a5c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f243800ef30 tx=0x7f243800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.507+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380092e0 con 0x7f2440110960 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.507+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2438004bb0 con 0x7f2440110960 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.507+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24401a6530 con 0x7f2440110960 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.507+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438004020 con 0x7f2440110960 2024-05-27T10:16:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.507+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24401adeb0 con 0x7f2440110960 2024-05-27T10:16:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.508+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2440063ec0 con 0x7f2440110960 2024-05-27T10:16:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.511+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f243801b030 con 0x7f2440110960 2024-05-27T10:16:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.512+0000 7f243cc00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f24240b2900 0x7f24240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.512+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f24380e2630 con 0x7f2440110960 2024-05-27T10:16:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.512+0000 7f243f400700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f24240b2900 0x7f24240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.513+0000 7f243cc00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] conn(0x7f24240bc410 0x7f24240be900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.513+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 --> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2438008ec0 con 0x7f24240bc410 2024-05-27T10:16:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.513+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f24380e29e0 con 0x7f2440110960 2024-05-27T10:16:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.513+0000 7f243f400700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f24240b2900 0x7f24240b4e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f243000b2e0 tx=0x7f2430005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.513+0000 7f2437e00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] conn(0x7f24240bc410 0x7f24240be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.514+0000 7f2437e00700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] conn(0x7f24240bc410 0x7f24240be900 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:16.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.517+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== osd.4 v2:172.21.15.104:6808/3588429935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2438008ec0 con 0x7f24240bc410 2024-05-27T10:16:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.536+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 --> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2440064340 con 0x7f24240bc410 2024-05-27T10:16:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.537+0000 7f243cc00700 1 -- 172.21.15.42:0/3679317404 <== osd.4 v2:172.21.15.104:6808/3588429935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2440064340 con 0x7f24240bc410 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] conn(0x7f24240bc410 msgr2=0x7f24240be900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] conn(0x7f24240bc410 0x7f24240be900 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f24240b2900 msgr2=0x7f24240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f24240b2900 0x7f24240b4e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f243000b2e0 tx=0x7f2430005fb0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 msgr2=0x7f24401a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 --2- 172.21.15.42:0/3679317404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2440110960 0x7f24401a5c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f243800ef30 tx=0x7f243800c3a0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f243fe00700 1 -- 172.21.15.42:0/3679317404 reap_dead start 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 shutdown_connections 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 >> 172.21.15.42:0/3679317404 conn(0x7f2440077c70 msgr2=0x7f244007bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 shutdown_connections 2024-05-27T10:16:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.538+0000 7f2446600700 1 -- 172.21.15.42:0/3679317404 wait complete. 2024-05-27T10:16:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 -- 172.21.15.42:0/2696042927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494108d20 msgr2=0x7fc494109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 --2- 172.21.15.42:0/2696042927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494108d20 0x7fc494109130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc4800099c0 tx=0x7fc480009cd0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 -- 172.21.15.42:0/2696042927 shutdown_connections 2024-05-27T10:16:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 --2- 172.21.15.42:0/2696042927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc494109670 0x7fc494110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 --2- 172.21.15.42:0/2696042927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4941043f0 0x7fc494104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 --2- 172.21.15.42:0/2696042927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494108d20 0x7fc494109130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 -- 172.21.15.42:0/2696042927 >> 172.21.15.42:0/2696042927 conn(0x7fc494079d00 msgr2=0x7fc49407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 -- 172.21.15.42:0/2696042927 shutdown_connections 2024-05-27T10:16:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.810+0000 7fc49a000700 1 -- 172.21.15.42:0/2696042927 wait complete. 2024-05-27T10:16:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 Processor -- start 2024-05-27T10:16:16.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 -- start start 2024-05-27T10:16:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4941043f0 0x7fc49419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc494108d20 0x7fc49419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494109670 0x7fc4941a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc494114130 con 0x7fc494109670 2024-05-27T10:16:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc494113fb0 con 0x7fc4941043f0 2024-05-27T10:16:16.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.812+0000 7fc49a000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4941142b0 con 0x7fc494108d20 2024-05-27T10:16:16.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494109670 0x7fc4941a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc498c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4941043f0 0x7fc49419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494109670 0x7fc4941a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53648/0 (socket says 172.21.15.42:53648) 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc498c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4941043f0 0x7fc49419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:60410/0 (socket says 172.21.15.42:60410) 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 -- 172.21.15.42:0/516812299 learned_addr learned my addr 172.21.15.42:0/516812299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc493e00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc494108d20 0x7fc49419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 -- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc494108d20 msgr2=0x7fc49419d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc494108d20 0x7fc49419d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 -- 172.21.15.42:0/516812299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4941043f0 msgr2=0x7fc49419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4941043f0 0x7fc49419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc499600700 1 -- 172.21.15.42:0/516812299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4941a1c50 con 0x7fc494109670 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc493e00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc494108d20 0x7fc49419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.813+0000 7fc498c00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4941043f0 0x7fc49419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.814+0000 7fc499600700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494109670 0x7fc4941a15b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00ef30 tx=0x7fc48c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.814+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0092e0 con 0x7fc494109670 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.814+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4941a1ee0 con 0x7fc494109670 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.814+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc48c004bb0 con 0x7fc494109670 2024-05-27T10:16:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.814+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c004020 con 0x7fc494109670 2024-05-27T10:16:16.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.814+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4941a23a0 con 0x7fc494109670 2024-05-27T10:16:16.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.815+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc49410fdb0 con 0x7fc494109670 2024-05-27T10:16:16.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.818+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc48c01b030 con 0x7fc494109670 2024-05-27T10:16:16.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.819+0000 7fc491600700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc47c0b2880 0x7fc47c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc48c0e2740 con 0x7fc494109670 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc498c00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc47c0b2880 0x7fc47c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc491600700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] conn(0x7fc47c0bc390 0x7fc47c0be880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 --> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc48c008ec0 con 0x7fc47c0bc390 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc48c0e2af0 con 0x7fc494109670 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc493e00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] conn(0x7fc47c0bc390 0x7fc47c0be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:16.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.820+0000 7fc498c00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc47c0b2880 0x7fc47c0b4d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc488005950 tx=0x7fc48800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:16.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.821+0000 7fc493e00700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] conn(0x7fc47c0bc390 0x7fc47c0be880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:16.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.822+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== osd.5 v2:172.21.15.143:6808/658500194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc48c008ec0 con 0x7fc47c0bc390 2024-05-27T10:16:16.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.842+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 --> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4941058a0 con 0x7fc47c0bc390 2024-05-27T10:16:16.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.843+0000 7fc491600700 1 -- 172.21.15.42:0/516812299 <== osd.5 v2:172.21.15.143:6808/658500194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc4941058a0 con 0x7fc47c0bc390 2024-05-27T10:16:16.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] conn(0x7fc47c0bc390 msgr2=0x7fc47c0be880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] conn(0x7fc47c0bc390 0x7fc47c0be880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc47c0b2880 msgr2=0x7fc47c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc47c0b2880 0x7fc47c0b4d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc488005950 tx=0x7fc48800b410 comp rx=0 tx=0).stop 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494109670 msgr2=0x7fc4941a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 --2- 172.21.15.42:0/516812299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc494109670 0x7fc4941a15b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00ef30 tx=0x7fc48c00c3a0 comp rx=0 tx=0).stop 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc499600700 1 -- 172.21.15.42:0/516812299 reap_dead start 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 shutdown_connections 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 >> 172.21.15.42:0/516812299 conn(0x7fc494079d00 msgr2=0x7fc49410f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 shutdown_connections 2024-05-27T10:16:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:16.844+0000 7fc49a000700 1 -- 172.21.15.42:0/516812299 wait complete. 2024-05-27T10:16:17.955 INFO:teuthology.orchestra.run.smithi042.stdout:158913790042 2024-05-27T10:16:17.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-27T10:16:18.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:17 smithi042 ceph-mon[31661]: pgmap v449: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-27T10:16:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:17 smithi143 ceph-mon[35545]: pgmap v449: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-27T10:16:18.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:17 smithi104 ceph-mon[34812]: pgmap v449: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-27T10:16:18.475 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:18.476 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:18.549 INFO:teuthology.orchestra.run.smithi042.stdout:137438953570 2024-05-27T10:16:18.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-27T10:16:18.587 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:18.648 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.058 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.061 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.061 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.061 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:18 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:18 smithi042 ceph-mon[31661]: pgmap v450: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:18 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:18 smithi104 ceph-mon[34812]: pgmap v450: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:19.237 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.242 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.247 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.247 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:19.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:18 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:19.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:18 smithi143 ceph-mon[35545]: pgmap v450: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:21.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:21 smithi143 ceph-mon[35545]: pgmap v451: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:21.826 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:21 smithi042 ceph-mon[31661]: pgmap v451: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:21.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:21 smithi104 ceph-mon[34812]: pgmap v451: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:23 smithi143 ceph-mon[35545]: pgmap v452: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:23.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:23 smithi042 ceph-mon[31661]: pgmap v452: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:23 smithi104 ceph-mon[34812]: pgmap v452: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-27T10:16:25.797 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:25 smithi042 ceph-mon[31661]: pgmap v453: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-27T10:16:25.810 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:25 smithi143 ceph-mon[35545]: pgmap v453: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-27T10:16:25.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:25 smithi104 ceph-mon[34812]: pgmap v453: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-27T10:16:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:27 smithi143 ceph-mon[35545]: pgmap v454: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:16:27.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:27 smithi042 ceph-mon[31661]: pgmap v454: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:16:27.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:27 smithi104 ceph-mon[34812]: pgmap v454: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:16:29.592 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:29 smithi143 ceph-mon[35545]: pgmap v455: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:29.826 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:29 smithi042 ceph-mon[31661]: pgmap v455: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:29 smithi104 ceph-mon[34812]: pgmap v455: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:31.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:31 smithi143 ceph-mon[35545]: pgmap v456: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:31.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:31 smithi042 ceph-mon[31661]: pgmap v456: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:31 smithi104 ceph-mon[34812]: pgmap v456: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.312+0000 7fede4000700 1 -- 172.21.15.42:0/2557938358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc108f20 msgr2=0x7feddc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.312+0000 7fede4000700 1 --2- 172.21.15.42:0/2557938358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc108f20 0x7feddc109330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fedc80099c0 tx=0x7fedc8009cd0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 -- 172.21.15.42:0/2557938358 shutdown_connections 2024-05-27T10:16:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 --2- 172.21.15.42:0/2557938358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feddc109870 0x7feddc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 --2- 172.21.15.42:0/2557938358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc104640 0x7feddc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 --2- 172.21.15.42:0/2557938358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc108f20 0x7feddc109330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 -- 172.21.15.42:0/2557938358 >> 172.21.15.42:0/2557938358 conn(0x7feddc079ef0 msgr2=0x7feddc07a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 -- 172.21.15.42:0/2557938358 shutdown_connections 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.313+0000 7fede4000700 1 -- 172.21.15.42:0/2557938358 wait complete. 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.314+0000 7fede4000700 1 Processor -- start 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 -- start start 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 0x7feddc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc108f20 0x7feddc19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feddc109870 0x7feddc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feddc114380 con 0x7feddc104640 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feddc114200 con 0x7feddc108f20 2024-05-27T10:16:32.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede4000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feddc114500 con 0x7feddc109870 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 0x7feddc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 0x7feddc19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33418/0 (socket says 172.21.15.42:33418) 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.315+0000 7fede1600700 1 -- 172.21.15.42:0/3903146914 learned_addr learned my addr 172.21.15.42:0/3903146914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede0c00700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc108f20 0x7feddc19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede2000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feddc109870 0x7feddc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede1600700 1 -- 172.21.15.42:0/3903146914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feddc109870 msgr2=0x7feddc1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede1600700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feddc109870 0x7feddc1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede1600700 1 -- 172.21.15.42:0/3903146914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc108f20 msgr2=0x7feddc19d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede1600700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc108f20 0x7feddc19d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede1600700 1 -- 172.21.15.42:0/3903146914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc1a1dd0 con 0x7feddc104640 2024-05-27T10:16:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.316+0000 7fede1600700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 0x7feddc19cfb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fedd00060b0 tx=0x7fedd0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.317+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd001a070 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.317+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedd000de20 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.317+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0014690 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.317+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc1a20c0 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.317+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a25d0 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.318+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc0512b0 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.320+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fedd001d070 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.322+0000 7fedda000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fedc40b2880 0x7fedc40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.322+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fedd00e1af0 con 0x7feddc104640 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.322+0000 7fede0c00700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fedc40b2880 0x7fedc40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.323+0000 7fede0c00700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fedc40b2880 0x7fedc40b4d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009990 tx=0x7fedc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.327+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fedd0010070 con 0x7feddc104640 2024-05-27T10:16:32.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.572+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7feddc1a9b50 con 0x7feddc104640 2024-05-27T10:16:32.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.573+0000 7fedda000700 1 -- 172.21.15.42:0/3903146914 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fedd00a97e0 con 0x7feddc104640 2024-05-27T10:16:32.574 INFO:teuthology.orchestra.run.smithi042.stdout:206158430285 2024-05-27T10:16:32.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fedc40b2880 msgr2=0x7fedc40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:32.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fedc40b2880 0x7fedc40b4d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009990 tx=0x7fedc8005fb0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 msgr2=0x7feddc19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:32.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 0x7feddc19cfb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fedd00060b0 tx=0x7fedd0007f90 comp rx=0 tx=0).stop 2024-05-27T10:16:32.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 shutdown_connections 2024-05-27T10:16:32.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feddc109870 0x7feddc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feddc104640 0x7feddc19cfb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fedc40b2880 0x7fedc40b4d90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 --2- 172.21.15.42:0/3903146914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc108f20 0x7feddc19d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:32.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 >> 172.21.15.42:0/3903146914 conn(0x7feddc079ef0 msgr2=0x7feddc10f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:32.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 shutdown_connections 2024-05-27T10:16:32.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:32.576+0000 7fede4000700 1 -- 172.21.15.42:0/3903146914 wait complete. 2024-05-27T10:16:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:33 smithi143 ceph-mon[35545]: pgmap v457: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:33.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:33 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3903146914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T10:16:33.814 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:33 smithi042 ceph-mon[31661]: pgmap v457: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:33.814 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:33 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3903146914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T10:16:33.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:33 smithi104 ceph-mon[34812]: pgmap v457: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:33.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:33 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3903146914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T10:16:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.388+0000 7f4746e00700 1 -- 172.21.15.42:0/469172426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740110b90 msgr2=0x7f4740112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.388+0000 7f4746e00700 1 --2- 172.21.15.42:0/469172426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740110b90 0x7f4740112ff0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f473800b0a0 tx=0x7f473800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 -- 172.21.15.42:0/469172426 shutdown_connections 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 --2- 172.21.15.42:0/469172426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740110b90 0x7f4740112ff0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 --2- 172.21.15.42:0/469172426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4740075d20 0x7f4740076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 --2- 172.21.15.42:0/469172426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 0x7f4740110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 -- 172.21.15.42:0/469172426 >> 172.21.15.42:0/469172426 conn(0x7f4740100030 msgr2=0x7f4740102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 -- 172.21.15.42:0/469172426 shutdown_connections 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.389+0000 7f4746e00700 1 -- 172.21.15.42:0/469172426 wait complete. 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.390+0000 7f4746e00700 1 Processor -- start 2024-05-27T10:16:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 -- start start 2024-05-27T10:16:34.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 0x7f474019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:34.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 0x7f474019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:34.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4740110b90 0x7f47401a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:34.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4740116570 con 0x7f4740075d20 2024-05-27T10:16:34.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47401163f0 con 0x7f4740076670 2024-05-27T10:16:34.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.391+0000 7f4746e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f47401166f0 con 0x7f4740110b90 2024-05-27T10:16:34.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 0x7f474019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:34.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 0x7f474019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47734/0 (socket says 172.21.15.42:47734) 2024-05-27T10:16:34.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 -- 172.21.15.42:0/1243658156 learned_addr learned my addr 172.21.15.42:0/1243658156 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f4737400700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 0x7f474019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f4744e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4740110b90 0x7f47401a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 -- 172.21.15.42:0/1243658156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4740110b90 msgr2=0x7f47401a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4740110b90 0x7f47401a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 -- 172.21.15.42:0/1243658156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 msgr2=0x7f474019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 0x7f474019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f473fe00700 1 -- 172.21.15.42:0/1243658156 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47401a1cc0 con 0x7f4740075d20 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.392+0000 7f4737400700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 0x7f474019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.393+0000 7f473fe00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 0x7f474019cea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f473000c9b0 tx=0x7f473000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.393+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730007b00 con 0x7f4740075d20 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.393+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47401a1fb0 con 0x7f4740075d20 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.393+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4730007ca0 con 0x7f4740075d20 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.393+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730018660 con 0x7f4740075d20 2024-05-27T10:16:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.393+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47401a9980 con 0x7f4740075d20 2024-05-27T10:16:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.395+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47400512b0 con 0x7f4740075d20 2024-05-27T10:16:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.397+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f473000f450 con 0x7f4740075d20 2024-05-27T10:16:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.398+0000 7f473d600700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47240b2900 0x7f47240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.399+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f47300e2240 con 0x7f4740075d20 2024-05-27T10:16:34.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.399+0000 7f4737400700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47240b2900 0x7f47240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:34.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.400+0000 7f4737400700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47240b2900 0x7f47240b4e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4728005fd0 tx=0x7f472800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.403+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4730014030 con 0x7f4740075d20 2024-05-27T10:16:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.672+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f474019e200 con 0x7f4740075d20 2024-05-27T10:16:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.672+0000 7f473d600700 1 -- 172.21.15.42:0/1243658156 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f47300a9f30 con 0x7f4740075d20 2024-05-27T10:16:34.673 INFO:teuthology.orchestra.run.smithi042.stdout:304942678064 2024-05-27T10:16:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47240b2900 msgr2=0x7f47240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47240b2900 0x7f47240b4e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4728005fd0 tx=0x7f472800a5e0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 msgr2=0x7f474019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 0x7f474019cea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f473000c9b0 tx=0x7f473000cd70 comp rx=0 tx=0).stop 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 shutdown_connections 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4740110b90 0x7f47401a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4740075d20 0x7f474019cea0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f47240b2900 0x7f47240b4e10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 --2- 172.21.15.42:0/1243658156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4740076670 0x7f474019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 >> 172.21.15.42:0/1243658156 conn(0x7f4740100030 msgr2=0x7f4740101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.677+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 shutdown_connections 2024-05-27T10:16:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7f4746e00700 1 -- 172.21.15.42:0/1243658156 wait complete. 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 -- 172.21.15.42:0/3632919859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64110b90 msgr2=0x7fdd64112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 --2- 172.21.15.42:0/3632919859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64110b90 0x7fdd64112ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b0a0 tx=0x7fdd5c00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 -- 172.21.15.42:0/3632919859 shutdown_connections 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 --2- 172.21.15.42:0/3632919859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64110b90 0x7fdd64112ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 --2- 172.21.15.42:0/3632919859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd64076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 --2- 172.21.15.42:0/3632919859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd64076670 0x7fdd64110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 -- 172.21.15.42:0/3632919859 >> 172.21.15.42:0/3632919859 conn(0x7fdd64100030 msgr2=0x7fdd64102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.678+0000 7fdd6a000700 1 -- 172.21.15.42:0/3632919859 shutdown_connections 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.679+0000 7fdd6a000700 1 -- 172.21.15.42:0/3632919859 wait complete. 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 Processor -- start 2024-05-27T10:16:35.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 -- start start 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd6419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64076670 0x7fdd6419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd64110b90 0x7fdd641a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64116520 con 0x7fdd64076670 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641163a0 con 0x7fdd64110b90 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.680+0000 7fdd6a000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641166a0 con 0x7fdd64075d20 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.681+0000 7fdd63400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd6419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.681+0000 7fdd63400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd6419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:60094/0 (socket says 172.21.15.42:60094) 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.681+0000 7fdd63400700 1 -- 172.21.15.42:0/2747044820 learned_addr learned my addr 172.21.15.42:0/2747044820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.681+0000 7fdd63e00700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd64110b90 0x7fdd641a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.681+0000 7fdd62a00700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64076670 0x7fdd6419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd63400700 1 -- 172.21.15.42:0/2747044820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd64110b90 msgr2=0x7fdd641a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd63400700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd64110b90 0x7fdd641a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd63400700 1 -- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64076670 msgr2=0x7fdd6419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd63400700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64076670 0x7fdd6419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd63400700 1 -- 172.21.15.42:0/2747044820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd641a1e00 con 0x7fdd64075d20 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd62a00700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64076670 0x7fdd6419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.682+0000 7fdd63400700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd6419cf30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400e9a0 tx=0x7fdd5400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.683+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400ca10 con 0x7fdd64075d20 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.683+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641a2090 con 0x7fdd64075d20 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.683+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd5400cbb0 con 0x7fdd64075d20 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.683+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54018680 con 0x7fdd64075d20 2024-05-27T10:16:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.683+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a9a10 con 0x7fdd64075d20 2024-05-27T10:16:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.684+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd640512b0 con 0x7fdd64075d20 2024-05-27T10:16:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.688+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdd54010430 con 0x7fdd64075d20 2024-05-27T10:16:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.690+0000 7fdd5be00700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdd480b2880 0x7fdd480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.690+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdd540e2790 con 0x7fdd64075d20 2024-05-27T10:16:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.690+0000 7fdd62a00700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdd480b2880 0x7fdd480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.134 INFO:teuthology.orchestra.run.smithi042.stdout:184683593814 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.691+0000 7fdd62a00700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdd480b2880 0x7fdd480b4d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd64103550 tx=0x7fdd500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.704+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd540aa480 con 0x7fdd64075d20 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.957+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdd6419e290 con 0x7fdd64075d20 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.958+0000 7fdd5be00700 1 -- 172.21.15.42:0/2747044820 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdd540a9d80 con 0x7fdd64075d20 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdd480b2880 msgr2=0x7fdd480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdd480b2880 0x7fdd480b4d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd64103550 tx=0x7fdd500073d0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 msgr2=0x7fdd6419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd6419cf30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400e9a0 tx=0x7fdd5400ecb0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 shutdown_connections 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fdd480b2880 0x7fdd480b4d90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd64110b90 0x7fdd641a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd64075d20 0x7fdd6419cf30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 --2- 172.21.15.42:0/2747044820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd64076670 0x7fdd6419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 >> 172.21.15.42:0/2747044820 conn(0x7fdd64100030 msgr2=0x7fdd64100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 shutdown_connections 2024-05-27T10:16:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.961+0000 7fdd6a000700 1 -- 172.21.15.42:0/2747044820 wait complete. 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 -- 172.21.15.42:0/1625101614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 msgr2=0x7fd32c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 --2- 172.21.15.42:0/1625101614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c104820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0099c0 tx=0x7fd31c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 -- 172.21.15.42:0/1625101614 shutdown_connections 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 --2- 172.21.15.42:0/1625101614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd32c109690 0x7fd32c110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 --2- 172.21.15.42:0/1625101614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c104820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 --2- 172.21.15.42:0/1625101614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 0x7fd32c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.906+0000 7fd331000700 1 -- 172.21.15.42:0/1625101614 >> 172.21.15.42:0/1625101614 conn(0x7fd32c07aca0 msgr2=0x7fd32c07b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.907+0000 7fd331000700 1 -- 172.21.15.42:0/1625101614 shutdown_connections 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.907+0000 7fd331000700 1 -- 172.21.15.42:0/1625101614 wait complete. 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.908+0000 7fd331000700 1 Processor -- start 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.908+0000 7fd331000700 1 -- start start 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.908+0000 7fd331000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.908+0000 7fd331000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 0x7fd32c075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.908+0000 7fd331000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd32c109690 0x7fd32c076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.908+0000 7fd331000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c114170 con 0x7fd32c104410 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd331000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c113ff0 con 0x7fd32c108d40 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd331000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1142f0 con 0x7fd32c109690 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c077600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47750/0 (socket says 172.21.15.42:47750) 2024-05-27T10:16:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 -- 172.21.15.42:0/3316568974 learned_addr learned my addr 172.21.15.42:0/3316568974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd322a00700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 0x7fd32c075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32be00700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd32c109690 0x7fd32c076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 -- 172.21.15.42:0/3316568974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd32c109690 msgr2=0x7fd32c076050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd32c109690 0x7fd32c076050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 -- 172.21.15.42:0/3316568974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 msgr2=0x7fd32c075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 0x7fd32c075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd32b400700 1 -- 172.21.15.42:0/3316568974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c076880 con 0x7fd32c104410 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.909+0000 7fd322a00700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 0x7fd32c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.910+0000 7fd32b400700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c077600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000c00 tx=0x7fd31c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.910+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00f940 con 0x7fd32c104410 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.910+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd31c004830 con 0x7fd32c104410 2024-05-27T10:16:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.910+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1ad590 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.910+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0054b0 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.910+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c1ada40 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.911+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c1058c0 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.914+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd31c029060 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.916+0000 7fd328c00700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd3100b2900 0x7fd3100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.916+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd31c0e2ff0 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.916+0000 7fd322a00700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd3100b2900 0x7fd3100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.917+0000 7fd322a00700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd3100b2900 0x7fd3100b4e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd32c100f10 tx=0x7fd31400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:34.919+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd31c013070 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.164+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd32c02e5e0 con 0x7fd32c104410 2024-05-27T10:16:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.165+0000 7fd328c00700 1 -- 172.21.15.42:0/3316568974 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd31c0aadd0 con 0x7fd32c104410 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stdout:77309411445 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd3100b2900 msgr2=0x7fd3100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd3100b2900 0x7fd3100b4e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd32c100f10 tx=0x7fd31400b5f0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 msgr2=0x7fd32c077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c077600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000c00 tx=0x7fd31c00f510 comp rx=0 tx=0).stop 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 shutdown_connections 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd32c109690 0x7fd32c076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd32c104410 0x7fd32c077600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd3100b2900 0x7fd3100b4e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 --2- 172.21.15.42:0/3316568974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32c108d40 0x7fd32c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:35.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 >> 172.21.15.42:0/3316568974 conn(0x7fd32c07aca0 msgr2=0x7fd32c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:35.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 shutdown_connections 2024-05-27T10:16:35.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:35.169+0000 7fd331000700 1 -- 172.21.15.42:0/3316568974 wait complete. 2024-05-27T10:16:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:35 smithi143 ceph-mon[35545]: pgmap v458: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:16:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:35 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1243658156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T10:16:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:35 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2747044820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T10:16:35.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:35 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3316568974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T10:16:35.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:35 smithi042 ceph-mon[31661]: pgmap v458: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:16:35.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:35 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1243658156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T10:16:35.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:35 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2747044820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T10:16:35.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:35 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3316568974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T10:16:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:35 smithi104 ceph-mon[34812]: pgmap v458: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:16:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:35 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1243658156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T10:16:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:35 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2747044820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T10:16:35.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:35 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3316568974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T10:16:37.812 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:37 smithi042 ceph-mon[31661]: pgmap v459: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:16:37.813 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:37 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/235211565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T10:16:37.813 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:37 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1626891586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T10:16:37.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 -- 172.21.15.42:0/1493978465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 msgr2=0x7f939c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:37.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 --2- 172.21.15.42:0/1493978465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c1111c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f939400b0d0 tx=0x7f939400b3e0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 -- 172.21.15.42:0/1493978465 shutdown_connections 2024-05-27T10:16:37.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 --2- 172.21.15.42:0/1493978465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c1111c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 --2- 172.21.15.42:0/1493978465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f939c1045e0 0x7f939c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 --2- 172.21.15.42:0/1493978465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f939c108f10 0x7f939c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 -- 172.21.15.42:0/1493978465 >> 172.21.15.42:0/1493978465 conn(0x7f939c100030 msgr2=0x7f939c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 -- 172.21.15.42:0/1493978465 shutdown_connections 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.583+0000 7f93a3600700 1 -- 172.21.15.42:0/1493978465 wait complete. 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.584+0000 7f93a3600700 1 Processor -- start 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 -- start start 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f939c1045e0 0x7f939c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f939c108f10 0x7f939c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c06eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f939c114390 con 0x7f939c109860 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f939c114210 con 0x7f939c108f10 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a3600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f939c114510 con 0x7f939c1045e0 2024-05-27T10:16:37.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.585+0000 7f93a1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c06eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c06eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47774/0 (socket says 172.21.15.42:47774) 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 -- 172.21.15.42:0/235211565 learned_addr learned my addr 172.21.15.42:0/235211565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a0c00700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f939c1045e0 0x7f939c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f939be00700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f939c108f10 0x7f939c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 -- 172.21.15.42:0/235211565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f939c1045e0 msgr2=0x7f939c0753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f939c1045e0 0x7f939c0753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 -- 172.21.15.42:0/235211565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f939c108f10 msgr2=0x7f939c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f939c108f10 0x7f939c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 -- 172.21.15.42:0/235211565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c06f320 con 0x7f939c109860 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.586+0000 7f93a1600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c06eb50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9394007d10 tx=0x7f9394007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.587+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939401f070 con 0x7f939c109860 2024-05-27T10:16:37.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.587+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c072fc0 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.587+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9394003e10 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.587+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939401b820 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.587+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c0734a0 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.588+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c105a90 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.591+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9394022070 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.592+0000 7f9399600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f93840b2880 0x7f93840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.593+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f939400f020 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.593+0000 7f93a0c00700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f93840b2880 0x7f93840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.594+0000 7f93a0c00700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f93840b2880 0x7f93840b4d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9390005ea0 tx=0x7f9390005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.595+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9394029020 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.850+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f939c0512b0 con 0x7f939c109860 2024-05-27T10:16:37.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.851+0000 7f9399600700 1 -- 172.21.15.42:0/235211565 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f939c0512b0 con 0x7f939c109860 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stdout:55834574970 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f93840b2880 msgr2=0x7f93840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f93840b2880 0x7f93840b4d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9390005ea0 tx=0x7f9390005e30 comp rx=0 tx=0).stop 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 msgr2=0x7f939c06eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c06eb50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9394007d10 tx=0x7f9394007df0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 shutdown_connections 2024-05-27T10:16:37.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f939c109860 0x7f939c06eb50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f939c1045e0 0x7f939c0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f93840b2880 0x7f93840b4d90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 --2- 172.21.15.42:0/235211565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f939c108f10 0x7f939c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:37.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 >> 172.21.15.42:0/235211565 conn(0x7f939c100030 msgr2=0x7f939c108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:37.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 shutdown_connections 2024-05-27T10:16:37.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:36.855+0000 7f93a3600700 1 -- 172.21.15.42:0/235211565 wait complete. 2024-05-27T10:16:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:37 smithi104 ceph-mon[34812]: pgmap v459: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:16:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:37 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/235211565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T10:16:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:37 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1626891586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T10:16:38.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:37 smithi143 ceph-mon[35545]: pgmap v459: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:16:38.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:37 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/235211565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T10:16:38.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:37 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1626891586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 -- 172.21.15.42:0/3806777743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 msgr2=0x7f69ac110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 --2- 172.21.15.42:0/3806777743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac110090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69a400b0d0 tx=0x7f69a400b3e0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 -- 172.21.15.42:0/3806777743 shutdown_connections 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 --2- 172.21.15.42:0/3806777743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac110090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 --2- 172.21.15.42:0/3806777743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ac1034b0 0x7f69ac1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 --2- 172.21.15.42:0/3806777743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac107de0 0x7f69ac1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.260+0000 7f69b2e00700 1 -- 172.21.15.42:0/3806777743 >> 172.21.15.42:0/3806777743 conn(0x7f69ac0fedb0 msgr2=0x7f69ac1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.261+0000 7f69b2e00700 1 -- 172.21.15.42:0/3806777743 shutdown_connections 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.261+0000 7f69b2e00700 1 -- 172.21.15.42:0/3806777743 wait complete. 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.262+0000 7f69b2e00700 1 Processor -- start 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.262+0000 7f69b2e00700 1 -- start start 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b2e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ac1034b0 0x7f69ac199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b2e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac107de0 0x7f69ac199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b2e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b2e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac113210 con 0x7f69ac108730 2024-05-27T10:16:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b2e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac113090 con 0x7f69ac107de0 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b2e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac113390 con 0x7f69ac1034b0 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47794/0 (socket says 172.21.15.42:47794) 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69b0e00700 1 -- 172.21.15.42:0/1626891586 learned_addr learned my addr 172.21.15.42:0/1626891586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.263+0000 7f69abe00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ac1034b0 0x7f69ac199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69ab400700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac107de0 0x7f69ac199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69b0e00700 1 -- 172.21.15.42:0/1626891586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ac1034b0 msgr2=0x7f69ac199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69b0e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ac1034b0 0x7f69ac199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69b0e00700 1 -- 172.21.15.42:0/1626891586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac107de0 msgr2=0x7f69ac199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69b0e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac107de0 0x7f69ac199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69b0e00700 1 -- 172.21.15.42:0/1626891586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac1a8f80 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.264+0000 7f69b0e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac19a380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007dc0 tx=0x7f69a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.265+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a401f030 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.265+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69a401bb90 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.265+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac1a9210 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.265+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4012bb0 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.265+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac1a96c0 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.266+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac104960 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.269+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f69a4022030 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.270+0000 7f69a8c00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f69900b2880 0x7f69900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.270+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f69a400f020 con 0x7f69ac108730 2024-05-27T10:16:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.271+0000 7f69abe00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f69900b2880 0x7f69900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:38.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.271+0000 7f69abe00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f69900b2880 0x7f69900b4d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f699c00ac50 tx=0x7f699c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:38.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.273+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69a4029020 con 0x7f69ac108730 2024-05-27T10:16:38.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.518+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f69ac02e5e0 con 0x7f69ac108730 2024-05-27T10:16:38.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.519+0000 7f69a8c00700 1 -- 172.21.15.42:0/1626891586 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69a40ab910 con 0x7f69ac108730 2024-05-27T10:16:38.475 INFO:teuthology.orchestra.run.smithi042.stdout:34359738498 2024-05-27T10:16:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f69900b2880 msgr2=0x7f69900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f69900b2880 0x7f69900b4d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f699c00ac50 tx=0x7f699c00a380 comp rx=0 tx=0).stop 2024-05-27T10:16:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 msgr2=0x7f69ac19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac19a380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007dc0 tx=0x7f69a4007df0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 shutdown_connections 2024-05-27T10:16:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69ac108730 0x7f69ac19a380 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ac1034b0 0x7f69ac199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f69900b2880 0x7f69900b4d90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 --2- 172.21.15.42:0/1626891586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac107de0 0x7f69ac199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 >> 172.21.15.42:0/1626891586 conn(0x7f69ac0fedb0 msgr2=0x7f69ac107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 shutdown_connections 2024-05-27T10:16:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:37.523+0000 7f69b2e00700 1 -- 172.21.15.42:0/1626891586 wait complete. 2024-05-27T10:16:39.304 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593806 got 184683593814 for osd.6 2024-05-27T10:16:39.305 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:39.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:39 smithi042 ceph-mon[31661]: pgmap v460: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:39.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:39 smithi104 ceph-mon[34812]: pgmap v460: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:39.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430280 got 206158430285 for osd.7 2024-05-27T10:16:39.959 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:40.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:39 smithi143 ceph-mon[35545]: pgmap v460: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:16:40.677 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:40 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2268616961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T10:16:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:40 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2268616961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T10:16:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:40 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2268616961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T10:16:41.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:41 smithi042 ceph-mon[31661]: pgmap v461: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:41.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:41 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1639551939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T10:16:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:41 smithi104 ceph-mon[34812]: pgmap v461: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:41 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1639551939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T10:16:42.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:41 smithi143 ceph-mon[35545]: pgmap v461: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:16:42.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:41 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1639551939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T10:16:42.686 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411441 got 77309411445 for osd.2 2024-05-27T10:16:42.686 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 -- 172.21.15.42:0/2539236205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c110c60 msgr2=0x7f5b7c113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 --2- 172.21.15.42:0/2539236205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c110c60 0x7f5b7c113080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400b0a0 tx=0x7f5b7400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 -- 172.21.15.42:0/2539236205 shutdown_connections 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 --2- 172.21.15.42:0/2539236205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c110c60 0x7f5b7c113080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 --2- 172.21.15.42:0/2539236205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 --2- 172.21.15.42:0/2539236205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c103a00 0x7f5b7c110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 -- 172.21.15.42:0/2539236205 >> 172.21.15.42:0/2539236205 conn(0x7f5b7c0fedb0 msgr2=0x7f5b7c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 -- 172.21.15.42:0/2539236205 shutdown_connections 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.063+0000 7f5b81a00700 1 -- 172.21.15.42:0/2539236205 wait complete. 2024-05-27T10:16:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.065+0000 7f5b81a00700 1 Processor -- start 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.065+0000 7f5b81a00700 1 -- start start 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b81a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b81a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c103a00 0x7f5b7c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b81a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c110c60 0x7f5b7c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b81a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c116570 con 0x7f5b7c1030b0 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b81a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c1163f0 con 0x7f5b7c110c60 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b81a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c1166f0 con 0x7f5b7c103a00 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47834/0 (socket says 172.21.15.42:47834) 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1639551939 learned_addr learned my addr 172.21.15.42:0/1639551939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7a000700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c103a00 0x7f5b7c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7b400700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c110c60 0x7f5b7c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1639551939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c103a00 msgr2=0x7f5b7c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c103a00 0x7f5b7c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1639551939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c110c60 msgr2=0x7f5b7c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c110c60 0x7f5b7c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1639551939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c1a1d90 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7a000700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c103a00 0x7f5b7c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.066+0000 7f5b7b400700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c110c60 0x7f5b7c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.067+0000 7f5b7aa00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c19cec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00d950 tx=0x7f5b6c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.067+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c016070 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.067+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b6c010460 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.067+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c1a2020 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.067+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00f620 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.067+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c1a99a0 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.068+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c0512b0 con 0x7f5b7c1030b0 2024-05-27T10:16:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.071+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5b6c019070 con 0x7f5b7c1030b0 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.072+0000 7f5b73400700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5b600b2880 0x7f5b600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.073+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5b6c0e1e50 con 0x7f5b7c1030b0 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.073+0000 7f5b7a000700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5b600b2880 0x7f5b600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.074+0000 7f5b7a000700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5b600b2880 0x7f5b600b4d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c112370 tx=0x7f5b68009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.076+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b6c0a9b40 con 0x7f5b7c1030b0 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.339+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5b7c02e5e0 con 0x7f5b7c1030b0 2024-05-27T10:16:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.340+0000 7f5b73400700 1 -- 172.21.15.42:0/1639551939 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5b6c0a9440 con 0x7f5b7c1030b0 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stdout:257698037824 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5b600b2880 msgr2=0x7f5b600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5b600b2880 0x7f5b600b4d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c112370 tx=0x7f5b68009550 comp rx=0 tx=0).stop 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 msgr2=0x7f5b7c19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c19cec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00d950 tx=0x7f5b6c00dd10 comp rx=0 tx=0).stop 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 shutdown_connections 2024-05-27T10:16:43.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f5b600b2880 0x7f5b600b4d90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b7c110c60 0x7f5b7c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b7c1030b0 0x7f5b7c19cec0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 --2- 172.21.15.42:0/1639551939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c103a00 0x7f5b7c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.343+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 >> 172.21.15.42:0/1639551939 conn(0x7f5b7c0fedb0 msgr2=0x7f5b7c112730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.344+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 shutdown_connections 2024-05-27T10:16:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:41.344+0000 7f5b81a00700 1 -- 172.21.15.42:0/1639551939 wait complete. 2024-05-27T10:16:43.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:43 smithi042 ceph-mon[31661]: pgmap v462: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:43.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:43 smithi104 ceph-mon[34812]: pgmap v462: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:43 smithi143 ceph-mon[35545]: pgmap v462: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:16:44.207 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037817 got 257698037824 for osd.9 2024-05-27T10:16:44.207 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:44.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 -- 172.21.15.42:0/1276408620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee010e920 msgr2=0x7fbee0110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:44.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 --2- 172.21.15.42:0/1276408620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee010e920 0x7fbee0110dc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbed80098f0 tx=0x7fbed8009c00 comp rx=0 tx=0).stop 2024-05-27T10:16:44.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 -- 172.21.15.42:0/1276408620 shutdown_connections 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 --2- 172.21.15.42:0/1276408620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee010e920 0x7fbee0110dc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 --2- 172.21.15.42:0/1276408620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee01054f0 0x7fbee0105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 --2- 172.21.15.42:0/1276408620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbee0105e40 0x7fbee010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 -- 172.21.15.42:0/1276408620 >> 172.21.15.42:0/1276408620 conn(0x7fbee00fede0 msgr2=0x7fbee0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 -- 172.21.15.42:0/1276408620 shutdown_connections 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.268+0000 7fbee8400700 1 -- 172.21.15.42:0/1276408620 wait complete. 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.269+0000 7fbee8400700 1 Processor -- start 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.269+0000 7fbee8400700 1 -- start start 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee8400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee01054f0 0x7fbee019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee8400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 0x7fbee019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee8400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbee010e920 0x7fbee019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee8400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0114340 con 0x7fbee0105e40 2024-05-27T10:16:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee8400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01141c0 con 0x7fbee01054f0 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee8400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01144c0 con 0x7fbee010e920 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 0x7fbee019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 0x7fbee019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47812/0 (socket says 172.21.15.42:47812) 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.270+0000 7fbee5000700 1 -- 172.21.15.42:0/2268616961 learned_addr learned my addr 172.21.15.42:0/2268616961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5a00700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee01054f0 0x7fbee019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5000700 1 -- 172.21.15.42:0/2268616961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbee010e920 msgr2=0x7fbee019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5000700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbee010e920 0x7fbee019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5000700 1 -- 172.21.15.42:0/2268616961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee01054f0 msgr2=0x7fbee019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5000700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee01054f0 0x7fbee019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5000700 1 -- 172.21.15.42:0/2268616961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee019fc00 con 0x7fbee0105e40 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee5000700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 0x7fbee019b2d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00e9c0 tx=0x7fbecc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00cbd0 con 0x7fbee0105e40 2024-05-27T10:16:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee019fef0 con 0x7fbee0105e40 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a7810 con 0x7fbee0105e40 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.271+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbecc00cd70 con 0x7fbee0105e40 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.272+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc005590 con 0x7fbee0105e40 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.273+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec0005330 con 0x7fbee0105e40 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.276+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbecc004750 con 0x7fbee0105e40 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.277+0000 7fbede000700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbec80b2900 0x7fbec80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:44.277 INFO:teuthology.orchestra.run.smithi042.stdout:111669149805 2024-05-27T10:16:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.278+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbecc0e2470 con 0x7fbee0105e40 2024-05-27T10:16:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.278+0000 7fbee5a00700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbec80b2900 0x7fbec80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.278+0000 7fbee5a00700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbec80b2900 0x7fbec80b4e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbee006d140 tx=0x7fbed400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.279+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbecc054df0 con 0x7fbee0105e40 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.524+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbec00058f0 con 0x7fbee0105e40 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.526+0000 7fbede000700 1 -- 172.21.15.42:0/2268616961 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbecc0aa160 con 0x7fbee0105e40 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbec80b2900 msgr2=0x7fbec80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbec80b2900 0x7fbec80b4e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbee006d140 tx=0x7fbed400b040 comp rx=0 tx=0).stop 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 msgr2=0x7fbee019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 0x7fbee019b2d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00e9c0 tx=0x7fbecc00ed80 comp rx=0 tx=0).stop 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 shutdown_connections 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbee010e920 0x7fbee019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbec80b2900 0x7fbec80b4e10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee01054f0 0x7fbee019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 --2- 172.21.15.42:0/2268616961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee0105e40 0x7fbee019b2d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.529+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 >> 172.21.15.42:0/2268616961 conn(0x7fbee00fede0 msgr2=0x7fbee00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.530+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 shutdown_connections 2024-05-27T10:16:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:40.530+0000 7fbee8400700 1 -- 172.21.15.42:0/2268616961 wait complete. 2024-05-27T10:16:44.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:44 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4186311254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T10:16:44.927 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:44.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:44 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4186311254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T10:16:44.957 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738491 got 34359738498 for osd.0 2024-05-27T10:16:44.957 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:45.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 -- 172.21.15.42:0/3217150661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78107de0 msgr2=0x7f6a781081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/3217150661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78107de0 0x7f6a781081f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400fb20 tx=0x7f6a6400fe30 comp rx=0 tx=0).stop 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 -- 172.21.15.42:0/3217150661 shutdown_connections 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/3217150661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78108730 0x7f6a78110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/3217150661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 0x7f6a781038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/3217150661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78107de0 0x7f6a781081f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 -- 172.21.15.42:0/3217150661 >> 172.21.15.42:0/3217150661 conn(0x7f6a780fedb0 msgr2=0x7f6a781011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.935+0000 7f6a7fe00700 1 -- 172.21.15.42:0/3217150661 shutdown_connections 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.936+0000 7f6a7fe00700 1 -- 172.21.15.42:0/3217150661 wait complete. 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.937+0000 7f6a7fe00700 1 Processor -- start 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 -- start start 2024-05-27T10:16:45.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 0x7f6a7819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78107de0 0x7f6a7819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 0x7f6a78198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78112170 con 0x7f6a78108730 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78111ff0 con 0x7f6a781034b0 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.938+0000 7f6a7fe00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6a781122f0 con 0x7f6a78107de0 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.939+0000 7f6a7de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 0x7f6a78198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.939+0000 7f6a7de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 0x7f6a78198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52824/0 (socket says 172.21.15.42:52824) 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.939+0000 7f6a7de00700 1 -- 172.21.15.42:0/475103245 learned_addr learned my addr 172.21.15.42:0/475103245 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.939+0000 7f6a7d400700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 0x7f6a7819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.939+0000 7f6a7ca00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78107de0 0x7f6a7819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7de00700 1 -- 172.21.15.42:0/475103245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78107de0 msgr2=0x7f6a7819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7de00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78107de0 0x7f6a7819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7de00700 1 -- 172.21.15.42:0/475103245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 msgr2=0x7f6a7819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7de00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 0x7f6a7819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7de00700 1 -- 172.21.15.42:0/475103245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a78198fe0 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7ca00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78107de0 0x7f6a7819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7d400700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 0x7f6a7819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.940+0000 7f6a7de00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 0x7f6a78198930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000f3e0 tx=0x7f6a7000f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.941+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004bb0 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.941+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a78199270 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.941+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a70004d50 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.941+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a781a55a0 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.942+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70003c00 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.943+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a780512b0 con 0x7f6a78108730 2024-05-27T10:16:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.948+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a7001c030 con 0x7f6a78108730 2024-05-27T10:16:45.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.950+0000 7f6a76000700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6a600b2a80 0x7f6a600b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:45.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.950+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6a70003a00 con 0x7f6a78108730 2024-05-27T10:16:45.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.950+0000 7f6a7d400700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6a600b2a80 0x7f6a600b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:45.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.952+0000 7f6a7d400700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6a600b2a80 0x7f6a600b4f90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6a78199a70 tx=0x7f6a6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:45.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.954+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a70023020 con 0x7f6a78108730 2024-05-27T10:16:45.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:44 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4186311254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T10:16:45.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.213+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f6a7802e5e0 con 0x7f6a78108730 2024-05-27T10:16:45.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.214+0000 7f6a76000700 1 -- 172.21.15.42:0/475103245 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a700aa650 con 0x7f6a78108730 2024-05-27T10:16:45.215 INFO:teuthology.orchestra.run.smithi042.stdout:231928234057 2024-05-27T10:16:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6a600b2a80 msgr2=0x7f6a600b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6a600b2a80 0x7f6a600b4f90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6a78199a70 tx=0x7f6a6c009260 comp rx=0 tx=0).stop 2024-05-27T10:16:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 msgr2=0x7f6a78198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:45.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 0x7f6a78198930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000f3e0 tx=0x7f6a7000f6f0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 shutdown_connections 2024-05-27T10:16:45.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a78108730 0x7f6a78198930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6a600b2a80 0x7f6a600b4f90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a781034b0 0x7f6a7819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 --2- 172.21.15.42:0/475103245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a78107de0 0x7f6a7819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:45.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 >> 172.21.15.42:0/475103245 conn(0x7f6a780fedb0 msgr2=0x7f6a78109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:45.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 shutdown_connections 2024-05-27T10:16:45.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:45.217+0000 7f6a7fe00700 1 -- 172.21.15.42:0/475103245 wait complete. 2024-05-27T10:16:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:45 smithi042 ceph-mon[31661]: pgmap v463: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T10:16:45.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:45 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/475103245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T10:16:45.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:45 smithi104 ceph-mon[34812]: pgmap v463: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T10:16:45.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:45 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/475103245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T10:16:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:45 smithi143 ceph-mon[35545]: pgmap v463: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T10:16:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/475103245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T10:16:46.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234049 got 231928234057 for osd.8 2024-05-27T10:16:46.659 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:46.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 -- 172.21.15.42:0/3128341854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 msgr2=0x7f15b81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 --2- 172.21.15.42:0/3128341854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b81111c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15b000b0d0 tx=0x7f15b000b3e0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 -- 172.21.15.42:0/3128341854 shutdown_connections 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 --2- 172.21.15.42:0/3128341854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b81111c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 --2- 172.21.15.42:0/3128341854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15b81045e0 0x7f15b81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 --2- 172.21.15.42:0/3128341854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15b8108f10 0x7f15b8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.008+0000 7f15c0400700 1 -- 172.21.15.42:0/3128341854 >> 172.21.15.42:0/3128341854 conn(0x7f15b8100030 msgr2=0x7f15b8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.009+0000 7f15c0400700 1 -- 172.21.15.42:0/3128341854 shutdown_connections 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.009+0000 7f15c0400700 1 -- 172.21.15.42:0/3128341854 wait complete. 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 Processor -- start 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 -- start start 2024-05-27T10:16:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15b81045e0 0x7f15b8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15b8108f10 0x7f15b8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8114340 con 0x7f15b8109860 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81141c0 con 0x7f15b81045e0 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.010+0000 7f15c0400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81144c0 con 0x7f15b8108f10 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b8076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52802/0 (socket says 172.21.15.42:52802) 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 -- 172.21.15.42:0/4186311254 learned_addr learned my addr 172.21.15.42:0/4186311254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15bd000700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15b8108f10 0x7f15b8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15bda00700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15b81045e0 0x7f15b8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:46.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 -- 172.21.15.42:0/4186311254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15b8108f10 msgr2=0x7f15b8075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15b8108f10 0x7f15b8075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 -- 172.21.15.42:0/4186311254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15b81045e0 msgr2=0x7f15b8075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15b81045e0 0x7f15b8075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.011+0000 7f15be400700 1 -- 172.21.15.42:0/4186311254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b81a90b0 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.012+0000 7f15be400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b8076530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15b0007a00 tx=0x7f15b0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.012+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b001f070 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.012+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b0004c50 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.012+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b001b690 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.012+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b81a9340 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.012+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b81a9740 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.014+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b80512b0 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.016+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15b0022070 con 0x7f15b8109860 2024-05-27T10:16:46.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.017+0000 7f15b6000700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f15a00b2880 0x7f15a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:46.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.018+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f15b000f020 con 0x7f15b8109860 2024-05-27T10:16:46.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.018+0000 7f15bda00700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f15a00b2880 0x7f15a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:46.728 INFO:teuthology.orchestra.run.smithi042.stdout:158913790047 2024-05-27T10:16:46.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.019+0000 7f15bda00700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f15a00b2880 0x7f15a00b4d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009c60 tx=0x7f15ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:46.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.022+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15b0029020 con 0x7f15b8109860 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.267+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f15b802e5e0 con 0x7f15b8109860 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.268+0000 7f15b6000700 1 -- 172.21.15.42:0/4186311254 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f15b00ab040 con 0x7f15b8109860 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f15a00b2880 msgr2=0x7f15a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f15a00b2880 0x7f15a00b4d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009c60 tx=0x7f15ac009260 comp rx=0 tx=0).stop 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 msgr2=0x7f15b8076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b8076530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15b0007a00 tx=0x7f15b0007a30 comp rx=0 tx=0).stop 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 shutdown_connections 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15b8109860 0x7f15b8076530 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f15a00b2880 0x7f15a00b4d90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15b81045e0 0x7f15b8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 --2- 172.21.15.42:0/4186311254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15b8108f10 0x7f15b8075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:46.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.271+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 >> 172.21.15.42:0/4186311254 conn(0x7f15b8100030 msgr2=0x7f15b8108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:46.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.272+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 shutdown_connections 2024-05-27T10:16:46.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:44.273+0000 7f15c0400700 1 -- 172.21.15.42:0/4186311254 wait complete. 2024-05-27T10:16:46.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:46 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/689710768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T10:16:46.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:46 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/689710768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T10:16:47.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:46 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/689710768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T10:16:47.800 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678057 got 304942678064 for osd.11 2024-05-27T10:16:47.800 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:47 smithi042 ceph-mon[31661]: pgmap v464: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T10:16:47.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:47 smithi104 ceph-mon[34812]: pgmap v464: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T10:16:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:47 smithi143 ceph-mon[35545]: pgmap v464: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 -- 172.21.15.42:0/2470313686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 msgr2=0x7fa178117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 --2- 172.21.15.42:0/2470313686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa178117800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b0a0 tx=0x7fa17000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 -- 172.21.15.42:0/2470313686 shutdown_connections 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 --2- 172.21.15.42:0/2470313686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa178117800 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 --2- 172.21.15.42:0/2470313686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 0x7fa178110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 --2- 172.21.15.42:0/2470313686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17810a980 0x7fa17810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 -- 172.21.15.42:0/2470313686 >> 172.21.15.42:0/2470313686 conn(0x7fa178077c70 msgr2=0x7fa1780780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 -- 172.21.15.42:0/2470313686 shutdown_connections 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.020+0000 7fa17d200700 1 -- 172.21.15.42:0/2470313686 wait complete. 2024-05-27T10:16:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.021+0000 7fa17d200700 1 Processor -- start 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.021+0000 7fa17d200700 1 -- start start 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa17d200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17810a980 0x7fa1781a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa17d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa1781a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa17d200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 0x7fa1781a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa17d200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa17811a980 con 0x7fa17810b2d0 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa17d200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa17811a800 con 0x7fa178110960 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa17d200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa17811ab00 con 0x7fa17810a980 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa16fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa1781a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa16fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa1781a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52842/0 (socket says 172.21.15.42:52842) 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa16fe00700 1 -- 172.21.15.42:0/689710768 learned_addr learned my addr 172.21.15.42:0/689710768 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.022+0000 7fa175e00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17810a980 0x7fa1781a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa176800700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 0x7fa1781a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:48.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16fe00700 1 -- 172.21.15.42:0/689710768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17810a980 msgr2=0x7fa1781a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16fe00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17810a980 0x7fa1781a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16fe00700 1 -- 172.21.15.42:0/689710768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 msgr2=0x7fa1781a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16fe00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 0x7fa1781a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16fe00700 1 -- 172.21.15.42:0/689710768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1781a6200 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa176800700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 0x7fa1781a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16fe00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa1781a18d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa16000d980 tx=0x7fa16000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160017070 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa160010460 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16000de60 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.023+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1781a6490 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.024+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1781ade10 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.025+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1780512b0 con 0x7fa17810b2d0 2024-05-27T10:16:48.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.028+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa160010610 con 0x7fa17810b2d0 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.031+0000 7fa16ea00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa15c0b2880 0x7fa15c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.031+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa1600e1d60 con 0x7fa17810b2d0 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.031+0000 7fa175e00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa15c0b2880 0x7fa15c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.032+0000 7fa175e00700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa15c0b2880 0x7fa15c0b4d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa168005950 tx=0x7fa1680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stdout:279172874298 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.033+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1600a9a50 con 0x7fa17810b2d0 2024-05-27T10:16:48.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.295+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa17802e5e0 con 0x7fa17810b2d0 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.296+0000 7fa16ea00700 1 -- 172.21.15.42:0/689710768 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa1600a9350 con 0x7fa17810b2d0 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa15c0b2880 msgr2=0x7fa15c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa15c0b2880 0x7fa15c0b4d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa168005950 tx=0x7fa1680058e0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 msgr2=0x7fa1781a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa1781a18d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa16000d980 tx=0x7fa16000dd40 comp rx=0 tx=0).stop 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 shutdown_connections 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fa15c0b2880 0x7fa15c0b4d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa178110960 0x7fa1781a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17810a980 0x7fa1781a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 --2- 172.21.15.42:0/689710768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa17810b2d0 0x7fa1781a18d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 >> 172.21.15.42:0/689710768 conn(0x7fa178077c70 msgr2=0x7fa17807b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 shutdown_connections 2024-05-27T10:16:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:46.299+0000 7fa17d200700 1 -- 172.21.15.42:0/689710768 wait complete. 2024-05-27T10:16:48.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574964 got 55834574970 for osd.1 2024-05-27T10:16:48.884 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:49.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149799 got 111669149805 for osd.3 2024-05-27T10:16:49.832 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:49.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:49 smithi042 ceph-mon[31661]: pgmap v465: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T10:16:49.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:49 smithi104 ceph-mon[34812]: pgmap v465: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T10:16:49.978 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790042 got 158913790047 for osd.5 2024-05-27T10:16:49.979 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:50.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:49 smithi143 ceph-mon[35545]: pgmap v465: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T10:16:50.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874290 got 279172874298 for osd.10 2024-05-27T10:16:50.505 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:51.869 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:51 smithi042 ceph-mon[31661]: pgmap v466: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-27T10:16:51.869 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:51.869 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:51 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:51 smithi104 ceph-mon[34812]: pgmap v466: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-27T10:16:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:51 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.988+0000 7fe211e00700 1 -- 172.21.15.42:0/1766200512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c1034b0 msgr2=0x7fe20c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.988+0000 7fe211e00700 1 --2- 172.21.15.42:0/1766200512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c1034b0 0x7fe20c1038c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0099c0 tx=0x7fe1fc009cd0 comp rx=0 tx=0).stop 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 -- 172.21.15.42:0/1766200512 shutdown_connections 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 --2- 172.21.15.42:0/1766200512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe20c108730 0x7fe20c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 --2- 172.21.15.42:0/1766200512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c1034b0 0x7fe20c1038c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 --2- 172.21.15.42:0/1766200512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe20c107de0 0x7fe20c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 -- 172.21.15.42:0/1766200512 >> 172.21.15.42:0/1766200512 conn(0x7fe20c0fedb0 msgr2=0x7fe20c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 -- 172.21.15.42:0/1766200512 shutdown_connections 2024-05-27T10:16:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.989+0000 7fe211e00700 1 -- 172.21.15.42:0/1766200512 wait complete. 2024-05-27T10:16:51.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.990+0000 7fe211e00700 1 Processor -- start 2024-05-27T10:16:51.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.990+0000 7fe211e00700 1 -- start start 2024-05-27T10:16:51.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe211e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe20c1034b0 0x7fe20c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:51.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe211e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe20c107de0 0x7fe20c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:51.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe211e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 0x7fe20c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:51.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe211e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c06bd60 con 0x7fe20c108730 2024-05-27T10:16:51.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe211e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c06bbe0 con 0x7fe20c107de0 2024-05-27T10:16:51.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe211e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c06bee0 con 0x7fe20c1034b0 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe20b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 0x7fe20c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe20b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 0x7fe20c19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52854/0 (socket says 172.21.15.42:52854) 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.991+0000 7fe20b400700 1 -- 172.21.15.42:0/3477064025 learned_addr learned my addr 172.21.15.42:0/3477064025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20aa00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe20c1034b0 0x7fe20c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20a000700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe20c107de0 0x7fe20c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20b400700 1 -- 172.21.15.42:0/3477064025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe20c1034b0 msgr2=0x7fe20c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20b400700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe20c1034b0 0x7fe20c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20b400700 1 -- 172.21.15.42:0/3477064025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe20c107de0 msgr2=0x7fe20c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20b400700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe20c107de0 0x7fe20c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20b400700 1 -- 172.21.15.42:0/3477064025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c19fc90 con 0x7fe20c108730 2024-05-27T10:16:51.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe20b400700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 0x7fe20c19f5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe20400ef30 tx=0x7fe20400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:51.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.992+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2040092e0 con 0x7fe20c108730 2024-05-27T10:16:51.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.993+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe204004bb0 con 0x7fe20c108730 2024-05-27T10:16:51.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.993+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c19ff20 con 0x7fe20c108730 2024-05-27T10:16:51.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.993+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204004080 con 0x7fe20c108730 2024-05-27T10:16:51.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.993+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c1a78a0 con 0x7fe20c108730 2024-05-27T10:16:51.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.994+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c0512b0 con 0x7fe20c108730 2024-05-27T10:16:51.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.997+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe20401b030 con 0x7fe20c108730 2024-05-27T10:16:51.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.998+0000 7fe210a00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1f00b29d0 0x7fe1f00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:51.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.998+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe2040e26a0 con 0x7fe20c108730 2024-05-27T10:16:51.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.999+0000 7fe20aa00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1f00b29d0 0x7fe1f00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:52.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:51.999+0000 7fe20aa00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1f00b29d0 0x7fe1f00b4ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006010 tx=0x7fe1fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:52.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.001+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe204022020 con 0x7fe20c108730 2024-05-27T10:16:52.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:51 smithi143 ceph-mon[35545]: pgmap v466: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-27T10:16:52.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:52.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:51 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:52.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.262+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe20c02e5e0 con 0x7fe20c108730 2024-05-27T10:16:52.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.262+0000 7fe210a00700 1 -- 172.21.15.42:0/3477064025 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe204010040 con 0x7fe20c108730 2024-05-27T10:16:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:137438953577 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1f00b29d0 msgr2=0x7fe1f00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1f00b29d0 0x7fe1f00b4ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006010 tx=0x7fe1fc01a040 comp rx=0 tx=0).stop 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 msgr2=0x7fe20c19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 0x7fe20c19f5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe20400ef30 tx=0x7fe20400c450 comp rx=0 tx=0).stop 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 shutdown_connections 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe20c108730 0x7fe20c19f5f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe20c1034b0 0x7fe20c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe1f00b29d0 0x7fe1f00b4ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:52.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 --2- 172.21.15.42:0/3477064025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe20c107de0 0x7fe20c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:52.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.267+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 >> 172.21.15.42:0/3477064025 conn(0x7fe20c0fedb0 msgr2=0x7fe20c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:52.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.268+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 shutdown_connections 2024-05-27T10:16:52.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:52.268+0000 7fe211e00700 1 -- 172.21.15.42:0/3477064025 wait complete. 2024-05-27T10:16:52.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:52 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3477064025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T10:16:53.040 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:52 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3477064025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T10:16:53.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:52 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3477064025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T10:16:53.256 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953570 got 137438953577 for osd.4 2024-05-27T10:16:53.256 DEBUG:teuthology.parallel:result is None 2024-05-27T10:16:53.256 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T10:16:53.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:53.917 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:53 smithi042 ceph-mon[31661]: pgmap v467: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-27T10:16:53.917 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:16:53.917 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:16:53.918 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:53 smithi104 ceph-mon[34812]: pgmap v467: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-27T10:16:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:16:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:16:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:53 smithi143 ceph-mon[35545]: pgmap v467: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-27T10:16:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:16:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:16:54.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:16:55.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:55 smithi042 ceph-mon[31661]: pgmap v468: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-27T10:16:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:55 smithi104 ceph-mon[34812]: pgmap v468: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-27T10:16:56.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:55 smithi143 ceph-mon[35545]: pgmap v468: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-27T10:16:56.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 -- 172.21.15.42:0/450882529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 msgr2=0x7fb2c8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:56.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 --2- 172.21.15.42:0/450882529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c8110090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000b0d0 tx=0x7fb2c000b3e0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 -- 172.21.15.42:0/450882529 shutdown_connections 2024-05-27T10:16:56.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 --2- 172.21.15.42:0/450882529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c8110090 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 --2- 172.21.15.42:0/450882529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 0x7fb2c81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 --2- 172.21.15.42:0/450882529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 0x7fb2c81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 -- 172.21.15.42:0/450882529 >> 172.21.15.42:0/450882529 conn(0x7fb2c80fedb0 msgr2=0x7fb2c81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 -- 172.21.15.42:0/450882529 shutdown_connections 2024-05-27T10:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.233+0000 7fb2cca00700 1 -- 172.21.15.42:0/450882529 wait complete. 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.234+0000 7fb2cca00700 1 Processor -- start 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.234+0000 7fb2cca00700 1 -- start start 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2cca00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 0x7fb2c81a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2cca00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 0x7fb2c81a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2cca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2cca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8113210 con 0x7fb2c8108730 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2cca00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8113090 con 0x7fb2c8107de0 2024-05-27T10:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2cca00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8113390 con 0x7fb2c81034b0 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2c6800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2c6800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35854/0 (socket says 172.21.15.42:35854) 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.235+0000 7fb2c6800700 1 -- 172.21.15.42:0/4129701040 learned_addr learned my addr 172.21.15.42:0/4129701040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c5e00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 0x7fb2c81a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c5400700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 0x7fb2c81a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c6800700 1 -- 172.21.15.42:0/4129701040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 msgr2=0x7fb2c81a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c6800700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 0x7fb2c81a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c6800700 1 -- 172.21.15.42:0/4129701040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 msgr2=0x7fb2c81a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c6800700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 0x7fb2c81a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c6800700 1 -- 172.21.15.42:0/4129701040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c819b1c0 con 0x7fb2c8108730 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c5e00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 0x7fb2c81a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c5400700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 0x7fb2c81a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.236+0000 7fb2c6800700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c819aae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009850 tx=0x7fb2c0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.237+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c001f030 con 0x7fb2c8108730 2024-05-27T10:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.237+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2c001bad0 con 0x7fb2c8108730 2024-05-27T10:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.237+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c819b450 con 0x7fb2c8108730 2024-05-27T10:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.237+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012bb0 con 0x7fb2c8108730 2024-05-27T10:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.237+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a7750 con 0x7fb2c8108730 2024-05-27T10:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.238+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c80512b0 con 0x7fb2c8108730 2024-05-27T10:16:56.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.241+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb2c0022030 con 0x7fb2c8108730 2024-05-27T10:16:56.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.242+0000 7fb2bea00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2ac0b2900 0x7fb2ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:16:56.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.243+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb2c000f020 con 0x7fb2c8108730 2024-05-27T10:16:56.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.243+0000 7fb2c5e00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2ac0b2900 0x7fb2ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:16:56.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.243+0000 7fb2c5e00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2ac0b2900 0x7fb2ac0b4e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005950 tx=0x7fb2b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:16:56.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.245+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2c0029020 con 0x7fb2c8108730 2024-05-27T10:16:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.480+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2c819bfd0 con 0x7fb2ac0b2900 2024-05-27T10:16:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.503+0000 7fb2bea00700 1 -- 172.21.15.42:0/4129701040 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258982 (secure 0 0 0) 0x7fb2c819bfd0 con 0x7fb2ac0b2900 2024-05-27T10:16:56.509 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:16:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.509+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2ac0b2900 msgr2=0x7fb2ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.509+0000 7fb2cca00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2ac0b2900 0x7fb2ac0b4e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005950 tx=0x7fb2b80058e0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.509+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 msgr2=0x7fb2c819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.509+0000 7fb2cca00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c819aae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009850 tx=0x7fb2c0007e40 comp rx=0 tx=0).stop 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 shutdown_connections 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c8108730 0x7fb2c819aae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c81034b0 0x7fb2c81a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2ac0b2900 0x7fb2ac0b4e10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 --2- 172.21.15.42:0/4129701040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c8107de0 0x7fb2c81a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 >> 172.21.15.42:0/4129701040 conn(0x7fb2c80fedb0 msgr2=0x7fb2c806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 shutdown_connections 2024-05-27T10:16:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:16:56.510+0000 7fb2cca00700 1 -- 172.21.15.42:0/4129701040 wait complete. 2024-05-27T10:16:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:16:57.611 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-05-27T10:16:56.466207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654181,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":901,"num_read_kb":548,"num_write":300,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359904,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581920,"statfs":{"total":1151940427776,"available":1151571886080,"internally_reserved":0,"allocated":11010048,"data_stored":7092591,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005981"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393767+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:16:10.393767+0000","last_peered":"2024-05-27T10:16:10.393767+0000","last_clean":"2024-05-27T10:16:10.393767+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:16:10.393767+0000","last_undegraded":"2024-05-27T10:16:10.393767+0000","last_fullsized":"2024-05-27T10:16:10.393767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392172+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:16:10.392172+0000","last_peered":"2024-05-27T10:16:10.392172+0000","last_clean":"2024-05-27T10:16:10.392172+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:16:10.392172+0000","last_undegraded":"2024-05-27T10:16:10.392172+0000","last_fullsized":"2024-05-27T10:16:10.392172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393525+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:16:10.393525+0000","last_peered":"2024-05-27T10:16:10.393525+0000","last_clean":"2024-05-27T10:16:10.393525+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:16:10.393525+0000","last_undegraded":"2024-05-27T10:16:10.393525+0000","last_fullsized":"2024-05-27T10:16:10.393525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378541+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:16:10.378541+0000","last_peered":"2024-05-27T10:16:10.378541+0000","last_clean":"2024-05-27T10:16:10.378541+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:16:10.378541+0000","last_undegraded":"2024-05-27T10:16:10.378541+0000","last_fullsized":"2024-05-27T10:16:10.378541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390872+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:16:10.390872+0000","last_peered":"2024-05-27T10:16:10.390872+0000","last_clean":"2024-05-27T10:16:10.390872+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:16:10.390872+0000","last_undegraded":"2024-05-27T10:16:10.390872+0000","last_fullsized":"2024-05-27T10:16:10.390872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.375579+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:16:10.375579+0000","last_peered":"2024-05-27T10:16:10.375579+0000","last_clean":"2024-05-27T10:16:10.375579+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:16:10.375579+0000","last_undegraded":"2024-05-27T10:16:10.375579+0000","last_fullsized":"2024-05-27T10:16:10.375579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377111+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:16:10.377111+0000","last_peered":"2024-05-27T10:16:10.377111+0000","last_clean":"2024-05-27T10:16:10.377111+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:16:10.377111+0000","last_undegraded":"2024-05-27T10:16:10.377111+0000","last_fullsized":"2024-05-27T10:16:10.377111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390434+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:16:10.390434+0000","last_peered":"2024-05-27T10:16:10.390434+0000","last_clean":"2024-05-27T10:16:10.390434+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:16:10.390434+0000","last_undegraded":"2024-05-27T10:16:10.390434+0000","last_fullsized":"2024-05-27T10:16:10.390434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392015+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:16:10.392015+0000","last_peered":"2024-05-27T10:16:10.392015+0000","last_clean":"2024-05-27T10:16:10.392015+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:16:10.392015+0000","last_undegraded":"2024-05-27T10:16:10.392015+0000","last_fullsized":"2024-05-27T10:16:10.392015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.375724+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:16:10.375724+0000","last_peered":"2024-05-27T10:16:10.375724+0000","last_clean":"2024-05-27T10:16:10.375724+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:16:10.375724+0000","last_undegraded":"2024-05-27T10:16:10.375724+0000","last_fullsized":"2024-05-27T10:16:10.375724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.387337+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:16:10.387337+0000","last_peered":"2024-05-27T10:16:10.387337+0000","last_clean":"2024-05-27T10:16:10.387337+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:16:10.387337+0000","last_undegraded":"2024-05-27T10:16:10.387337+0000","last_fullsized":"2024-05-27T10:16:10.387337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.374971+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:16:10.374971+0000","last_peered":"2024-05-27T10:16:10.374971+0000","last_clean":"2024-05-27T10:16:10.374971+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:16:10.374971+0000","last_undegraded":"2024-05-27T10:16:10.374971+0000","last_fullsized":"2024-05-27T10:16:10.374971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.388420+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:16:10.388420+0000","last_peered":"2024-05-27T10:16:10.388420+0000","last_clean":"2024-05-27T10:16:10.388420+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:16:10.388420+0000","last_undegraded":"2024-05-27T10:16:10.388420+0000","last_fullsized":"2024-05-27T10:16:10.388420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391553+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:16:10.391553+0000","last_peered":"2024-05-27T10:16:10.391553+0000","last_clean":"2024-05-27T10:16:10.391553+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:16:10.391553+0000","last_undegraded":"2024-05-27T10:16:10.391553+0000","last_fullsized":"2024-05-27T10:16:10.391553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.382898+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:16:10.382898+0000","last_peered":"2024-05-27T10:16:10.382898+0000","last_clean":"2024-05-27T10:16:10.382898+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:16:10.382898+0000","last_undegraded":"2024-05-27T10:16:10.382898+0000","last_fullsized":"2024-05-27T10:16:10.382898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.396771+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:16:10.396771+0000","last_peered":"2024-05-27T10:16:10.396771+0000","last_clean":"2024-05-27T10:16:10.396771+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:16:10.396771+0000","last_undegraded":"2024-05-27T10:16:10.396771+0000","last_fullsized":"2024-05-27T10:16:10.396771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394995+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:16:10.394995+0000","last_peered":"2024-05-27T10:16:10.394995+0000","last_clean":"2024-05-27T10:16:10.394995+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:16:10.394995+0000","last_undegraded":"2024-05-27T10:16:10.394995+0000","last_fullsized":"2024-05-27T10:16:10.394995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.387315+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:16:10.387315+0000","last_peered":"2024-05-27T10:16:10.387315+0000","last_clean":"2024-05-27T10:16:10.387315+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:16:10.387315+0000","last_undegraded":"2024-05-27T10:16:10.387315+0000","last_fullsized":"2024-05-27T10:16:10.387315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392661+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:16:10.392661+0000","last_peered":"2024-05-27T10:16:10.392661+0000","last_clean":"2024-05-27T10:16:10.392661+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:16:10.392661+0000","last_undegraded":"2024-05-27T10:16:10.392661+0000","last_fullsized":"2024-05-27T10:16:10.392661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.661226+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:14:41.661226+0000","last_peered":"2024-05-27T10:14:41.661226+0000","last_clean":"2024-05-27T10:14:41.661226+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:14:41.661226+0000","last_undegraded":"2024-05-27T10:14:41.661226+0000","last_fullsized":"2024-05-27T10:14:41.661226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391503+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:16:10.391503+0000","last_peered":"2024-05-27T10:16:10.391503+0000","last_clean":"2024-05-27T10:16:10.391503+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:16:10.391503+0000","last_undegraded":"2024-05-27T10:16:10.391503+0000","last_fullsized":"2024-05-27T10:16:10.391503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648255+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:14:41.648255+0000","last_peered":"2024-05-27T10:14:41.648255+0000","last_clean":"2024-05-27T10:14:41.648255+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:14:41.648255+0000","last_undegraded":"2024-05-27T10:14:41.648255+0000","last_fullsized":"2024-05-27T10:14:41.648255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394588+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:16:10.394588+0000","last_peered":"2024-05-27T10:16:10.394588+0000","last_clean":"2024-05-27T10:16:10.394588+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:16:10.394588+0000","last_undegraded":"2024-05-27T10:16:10.394588+0000","last_fullsized":"2024-05-27T10:16:10.394588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644580+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:14:41.644580+0000","last_peered":"2024-05-27T10:14:41.644580+0000","last_clean":"2024-05-27T10:14:41.644580+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:14:41.644580+0000","last_undegraded":"2024-05-27T10:14:41.644580+0000","last_fullsized":"2024-05-27T10:14:41.644580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.399544+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:16:10.399544+0000","last_peered":"2024-05-27T10:16:10.399544+0000","last_clean":"2024-05-27T10:16:10.399544+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:16:10.399544+0000","last_undegraded":"2024-05-27T10:16:10.399544+0000","last_fullsized":"2024-05-27T10:16:10.399544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644093+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:14:41.644093+0000","last_peered":"2024-05-27T10:14:41.644093+0000","last_clean":"2024-05-27T10:14:41.644093+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:14:41.644093+0000","last_undegraded":"2024-05-27T10:14:41.644093+0000","last_fullsized":"2024-05-27T10:14:41.644093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.386315+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:16:10.386315+0000","last_peered":"2024-05-27T10:16:10.386315+0000","last_clean":"2024-05-27T10:16:10.386315+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:16:10.386315+0000","last_undegraded":"2024-05-27T10:16:10.386315+0000","last_fullsized":"2024-05-27T10:16:10.386315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":169,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:51.070957+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:16:51.070957+0000","last_peered":"2024-05-27T10:16:51.070957+0000","last_clean":"2024-05-27T10:16:51.070957+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:16:51.070957+0000","last_undegraded":"2024-05-27T10:16:51.070957+0000","last_fullsized":"2024-05-27T10:16:51.070957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.375765+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:16:10.375765+0000","last_peered":"2024-05-27T10:16:10.375765+0000","last_clean":"2024-05-27T10:16:10.375765+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:16:10.375765+0000","last_undegraded":"2024-05-27T10:16:10.375765+0000","last_fullsized":"2024-05-27T10:16:10.375765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640251+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:14:41.640251+0000","last_peered":"2024-05-27T10:14:41.640251+0000","last_clean":"2024-05-27T10:14:41.640251+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:14:41.640251+0000","last_undegraded":"2024-05-27T10:14:41.640251+0000","last_fullsized":"2024-05-27T10:14:41.640251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391287+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:16:10.391287+0000","last_peered":"2024-05-27T10:16:10.391287+0000","last_clean":"2024-05-27T10:16:10.391287+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:16:10.391287+0000","last_undegraded":"2024-05-27T10:16:10.391287+0000","last_fullsized":"2024-05-27T10:16:10.391287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636541+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:14:41.636541+0000","last_peered":"2024-05-27T10:14:41.636541+0000","last_clean":"2024-05-27T10:14:41.636541+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:14:41.636541+0000","last_undegraded":"2024-05-27T10:14:41.636541+0000","last_fullsized":"2024-05-27T10:14:41.636541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390238+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:16:10.390238+0000","last_peered":"2024-05-27T10:16:10.390238+0000","last_clean":"2024-05-27T10:16:10.390238+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:16:10.390238+0000","last_undegraded":"2024-05-27T10:16:10.390238+0000","last_fullsized":"2024-05-27T10:16:10.390238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652572+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:14:41.652572+0000","last_peered":"2024-05-27T10:14:41.652572+0000","last_clean":"2024-05-27T10:14:41.652572+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:14:41.652572+0000","last_undegraded":"2024-05-27T10:14:41.652572+0000","last_fullsized":"2024-05-27T10:14:41.652572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.395799+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:16:10.395799+0000","last_peered":"2024-05-27T10:16:10.395799+0000","last_clean":"2024-05-27T10:16:10.395799+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:16:10.395799+0000","last_undegraded":"2024-05-27T10:16:10.395799+0000","last_fullsized":"2024-05-27T10:16:10.395799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":163,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:52.072715+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:16:52.072715+0000","last_peered":"2024-05-27T10:16:52.072715+0000","last_clean":"2024-05-27T10:16:52.072715+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:16:52.072715+0000","last_undegraded":"2024-05-27T10:16:52.072715+0000","last_fullsized":"2024-05-27T10:16:52.072715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391413+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:16:10.391413+0000","last_peered":"2024-05-27T10:16:10.391413+0000","last_clean":"2024-05-27T10:16:10.391413+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:16:10.391413+0000","last_undegraded":"2024-05-27T10:16:10.391413+0000","last_fullsized":"2024-05-27T10:16:10.391413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642954+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:14:41.642954+0000","last_peered":"2024-05-27T10:14:41.642954+0000","last_clean":"2024-05-27T10:14:41.642954+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:14:41.642954+0000","last_undegraded":"2024-05-27T10:14:41.642954+0000","last_fullsized":"2024-05-27T10:14:41.642954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394457+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:16:10.394457+0000","last_peered":"2024-05-27T10:16:10.394457+0000","last_clean":"2024-05-27T10:16:10.394457+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:16:10.394457+0000","last_undegraded":"2024-05-27T10:16:10.394457+0000","last_fullsized":"2024-05-27T10:16:10.394457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.658256+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:14:41.658256+0000","last_peered":"2024-05-27T10:14:41.658256+0000","last_clean":"2024-05-27T10:14:41.658256+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:14:41.658256+0000","last_undegraded":"2024-05-27T10:14:41.658256+0000","last_fullsized":"2024-05-27T10:14:41.658256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.397694+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:16:10.397694+0000","last_peered":"2024-05-27T10:16:10.397694+0000","last_clean":"2024-05-27T10:16:10.397694+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:16:10.397694+0000","last_undegraded":"2024-05-27T10:16:10.397694+0000","last_fullsized":"2024-05-27T10:16:10.397694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'31","reported_seq":332,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:53.061752+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:16:53.061752+0000","last_peered":"2024-05-27T10:16:53.061752+0000","last_clean":"2024-05-27T10:16:53.061752+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:16:53.061752+0000","last_undegraded":"2024-05-27T10:16:53.061752+0000","last_fullsized":"2024-05-27T10:16:53.061752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58905,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":274,"num_read_kb":194,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.369550+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:16:10.369550+0000","last_peered":"2024-05-27T10:16:10.369550+0000","last_clean":"2024-05-27T10:16:10.369550+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:16:10.369550+0000","last_undegraded":"2024-05-27T10:16:10.369550+0000","last_fullsized":"2024-05-27T10:16:10.369550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648501+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:14:41.648501+0000","last_peered":"2024-05-27T10:14:41.648501+0000","last_clean":"2024-05-27T10:14:41.648501+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:14:41.648501+0000","last_undegraded":"2024-05-27T10:14:41.648501+0000","last_fullsized":"2024-05-27T10:14:41.648501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.399060+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:16:10.399060+0000","last_peered":"2024-05-27T10:16:10.399060+0000","last_clean":"2024-05-27T10:16:10.399060+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:16:10.399060+0000","last_undegraded":"2024-05-27T10:16:10.399060+0000","last_fullsized":"2024-05-27T10:16:10.399060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652886+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:14:41.652886+0000","last_peered":"2024-05-27T10:14:41.652886+0000","last_clean":"2024-05-27T10:14:41.652886+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:14:41.652886+0000","last_undegraded":"2024-05-27T10:14:41.652886+0000","last_fullsized":"2024-05-27T10:14:41.652886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390193+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:16:10.390193+0000","last_peered":"2024-05-27T10:16:10.390193+0000","last_clean":"2024-05-27T10:16:10.390193+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:16:10.390193+0000","last_undegraded":"2024-05-27T10:16:10.390193+0000","last_fullsized":"2024-05-27T10:16:10.390193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":166,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:50.064960+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:16:50.064960+0000","last_peered":"2024-05-27T10:16:50.064960+0000","last_clean":"2024-05-27T10:16:50.064960+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:16:50.064960+0000","last_undegraded":"2024-05-27T10:16:50.064960+0000","last_fullsized":"2024-05-27T10:16:50.064960+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377079+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:16:10.377079+0000","last_peered":"2024-05-27T10:16:10.377079+0000","last_clean":"2024-05-27T10:16:10.377079+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:16:10.377079+0000","last_undegraded":"2024-05-27T10:16:10.377079+0000","last_fullsized":"2024-05-27T10:16:10.377079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.650658+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:14:41.650658+0000","last_peered":"2024-05-27T10:14:41.650658+0000","last_clean":"2024-05-27T10:14:41.650658+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:14:41.650658+0000","last_undegraded":"2024-05-27T10:14:41.650658+0000","last_fullsized":"2024-05-27T10:14:41.650658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.379141+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:16:10.379141+0000","last_peered":"2024-05-27T10:16:10.379141+0000","last_clean":"2024-05-27T10:16:10.379141+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:16:10.379141+0000","last_undegraded":"2024-05-27T10:16:10.379141+0000","last_fullsized":"2024-05-27T10:16:10.379141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648826+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:14:41.648826+0000","last_peered":"2024-05-27T10:14:41.648826+0000","last_clean":"2024-05-27T10:14:41.648826+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:14:41.648826+0000","last_undegraded":"2024-05-27T10:14:41.648826+0000","last_fullsized":"2024-05-27T10:14:41.648826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.374617+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:16:10.374617+0000","last_peered":"2024-05-27T10:16:10.374617+0000","last_clean":"2024-05-27T10:16:10.374617+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:16:10.374617+0000","last_undegraded":"2024-05-27T10:16:10.374617+0000","last_fullsized":"2024-05-27T10:16:10.374617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646918+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:14:41.646918+0000","last_peered":"2024-05-27T10:14:41.646918+0000","last_clean":"2024-05-27T10:14:41.646918+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:14:41.646918+0000","last_undegraded":"2024-05-27T10:14:41.646918+0000","last_fullsized":"2024-05-27T10:14:41.646918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391439+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:16:10.391439+0000","last_peered":"2024-05-27T10:16:10.391439+0000","last_clean":"2024-05-27T10:16:10.391439+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:16:10.391439+0000","last_undegraded":"2024-05-27T10:16:10.391439+0000","last_fullsized":"2024-05-27T10:16:10.391439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642554+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:14:41.642554+0000","last_peered":"2024-05-27T10:14:41.642554+0000","last_clean":"2024-05-27T10:14:41.642554+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:14:41.642554+0000","last_undegraded":"2024-05-27T10:14:41.642554+0000","last_fullsized":"2024-05-27T10:14:41.642554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392985+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:16:10.392985+0000","last_peered":"2024-05-27T10:16:10.392985+0000","last_clean":"2024-05-27T10:16:10.392985+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:16:10.392985+0000","last_undegraded":"2024-05-27T10:16:10.392985+0000","last_fullsized":"2024-05-27T10:16:10.392985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.374917+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:16:10.374917+0000","last_peered":"2024-05-27T10:16:10.374917+0000","last_clean":"2024-05-27T10:16:10.374917+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:16:10.374917+0000","last_undegraded":"2024-05-27T10:16:10.374917+0000","last_fullsized":"2024-05-27T10:16:10.374917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":163,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:51.066817+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:16:51.066817+0000","last_peered":"2024-05-27T10:16:51.066817+0000","last_clean":"2024-05-27T10:16:51.066817+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:16:51.066817+0000","last_undegraded":"2024-05-27T10:16:51.066817+0000","last_fullsized":"2024-05-27T10:16:51.066817+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.396650+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:16:10.396650+0000","last_peered":"2024-05-27T10:16:10.396650+0000","last_clean":"2024-05-27T10:16:10.396650+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:16:10.396650+0000","last_undegraded":"2024-05-27T10:16:10.396650+0000","last_fullsized":"2024-05-27T10:16:10.396650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.655991+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:14:41.655991+0000","last_peered":"2024-05-27T10:14:41.655991+0000","last_clean":"2024-05-27T10:14:41.655991+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:14:41.655991+0000","last_undegraded":"2024-05-27T10:14:41.655991+0000","last_fullsized":"2024-05-27T10:14:41.655991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643102+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:14:41.643102+0000","last_peered":"2024-05-27T10:14:41.643102+0000","last_clean":"2024-05-27T10:14:41.643102+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:14:41.643102+0000","last_undegraded":"2024-05-27T10:14:41.643102+0000","last_fullsized":"2024-05-27T10:14:41.643102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.385694+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:16:10.385694+0000","last_peered":"2024-05-27T10:16:10.385694+0000","last_clean":"2024-05-27T10:16:10.385694+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:16:10.385694+0000","last_undegraded":"2024-05-27T10:16:10.385694+0000","last_fullsized":"2024-05-27T10:16:10.385694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643327+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:14:41.643327+0000","last_peered":"2024-05-27T10:14:41.643327+0000","last_clean":"2024-05-27T10:14:41.643327+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:14:41.643327+0000","last_undegraded":"2024-05-27T10:14:41.643327+0000","last_fullsized":"2024-05-27T10:14:41.643327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.650613+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:14:41.650613+0000","last_peered":"2024-05-27T10:14:41.650613+0000","last_clean":"2024-05-27T10:14:41.650613+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:14:41.650613+0000","last_undegraded":"2024-05-27T10:14:41.650613+0000","last_fullsized":"2024-05-27T10:14:41.650613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.383907+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:16:10.383907+0000","last_peered":"2024-05-27T10:16:10.383907+0000","last_clean":"2024-05-27T10:16:10.383907+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:16:10.383907+0000","last_undegraded":"2024-05-27T10:16:10.383907+0000","last_fullsized":"2024-05-27T10:16:10.383907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:11.220553+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:16:11.220553+0000","last_peered":"2024-05-27T10:16:11.220553+0000","last_clean":"2024-05-27T10:16:11.220553+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:16:11.220553+0000","last_undegraded":"2024-05-27T10:16:11.220553+0000","last_fullsized":"2024-05-27T10:16:11.220553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392646+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:16:10.392646+0000","last_peered":"2024-05-27T10:16:10.392646+0000","last_clean":"2024-05-27T10:16:10.392646+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:16:10.392646+0000","last_undegraded":"2024-05-27T10:16:10.392646+0000","last_fullsized":"2024-05-27T10:16:10.392646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652618+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:14:41.652618+0000","last_peered":"2024-05-27T10:14:41.652618+0000","last_clean":"2024-05-27T10:14:41.652618+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:14:41.652618+0000","last_undegraded":"2024-05-27T10:14:41.652618+0000","last_fullsized":"2024-05-27T10:14:41.652618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.396797+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:16:10.396797+0000","last_peered":"2024-05-27T10:16:10.396797+0000","last_clean":"2024-05-27T10:16:10.396797+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:16:10.396797+0000","last_undegraded":"2024-05-27T10:16:10.396797+0000","last_fullsized":"2024-05-27T10:16:10.396797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053356+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:14:42.053356+0000","last_peered":"2024-05-27T10:14:42.053356+0000","last_clean":"2024-05-27T10:14:42.053356+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:14:42.053356+0000","last_undegraded":"2024-05-27T10:14:42.053356+0000","last_fullsized":"2024-05-27T10:14:42.053356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.368416+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:16:10.368416+0000","last_peered":"2024-05-27T10:16:10.368416+0000","last_clean":"2024-05-27T10:16:10.368416+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:16:10.368416+0000","last_undegraded":"2024-05-27T10:16:10.368416+0000","last_fullsized":"2024-05-27T10:16:10.368416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645532+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:14:41.645532+0000","last_peered":"2024-05-27T10:14:41.645532+0000","last_clean":"2024-05-27T10:14:41.645532+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:14:41.645532+0000","last_undegraded":"2024-05-27T10:14:41.645532+0000","last_fullsized":"2024-05-27T10:14:41.645532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392887+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:16:10.392887+0000","last_peered":"2024-05-27T10:16:10.392887+0000","last_clean":"2024-05-27T10:16:10.392887+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:16:10.392887+0000","last_undegraded":"2024-05-27T10:16:10.392887+0000","last_fullsized":"2024-05-27T10:16:10.392887+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.659780+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:14:41.659780+0000","last_peered":"2024-05-27T10:14:41.659780+0000","last_clean":"2024-05-27T10:14:41.659780+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:14:41.659780+0000","last_undegraded":"2024-05-27T10:14:41.659780+0000","last_fullsized":"2024-05-27T10:14:41.659780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.395969+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:16:10.395969+0000","last_peered":"2024-05-27T10:16:10.395969+0000","last_clean":"2024-05-27T10:16:10.395969+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:16:10.395969+0000","last_undegraded":"2024-05-27T10:16:10.395969+0000","last_fullsized":"2024-05-27T10:16:10.395969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.663558+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:14:41.663558+0000","last_peered":"2024-05-27T10:14:41.663558+0000","last_clean":"2024-05-27T10:14:41.663558+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:14:41.663558+0000","last_undegraded":"2024-05-27T10:14:41.663558+0000","last_fullsized":"2024-05-27T10:14:41.663558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392881+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:16:10.392881+0000","last_peered":"2024-05-27T10:16:10.392881+0000","last_clean":"2024-05-27T10:16:10.392881+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:16:10.392881+0000","last_undegraded":"2024-05-27T10:16:10.392881+0000","last_fullsized":"2024-05-27T10:16:10.392881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.661244+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:14:41.661244+0000","last_peered":"2024-05-27T10:14:41.661244+0000","last_clean":"2024-05-27T10:14:41.661244+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:14:41.661244+0000","last_undegraded":"2024-05-27T10:14:41.661244+0000","last_fullsized":"2024-05-27T10:14:41.661244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391786+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:16:10.391786+0000","last_peered":"2024-05-27T10:16:10.391786+0000","last_clean":"2024-05-27T10:16:10.391786+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:16:10.391786+0000","last_undegraded":"2024-05-27T10:16:10.391786+0000","last_fullsized":"2024-05-27T10:16:10.391786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.654081+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:14:41.654081+0000","last_peered":"2024-05-27T10:14:41.654081+0000","last_clean":"2024-05-27T10:14:41.654081+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:14:41.654081+0000","last_undegraded":"2024-05-27T10:14:41.654081+0000","last_fullsized":"2024-05-27T10:14:41.654081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394633+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:16:10.394633+0000","last_peered":"2024-05-27T10:16:10.394633+0000","last_clean":"2024-05-27T10:16:10.394633+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:16:10.394633+0000","last_undegraded":"2024-05-27T10:16:10.394633+0000","last_fullsized":"2024-05-27T10:16:10.394633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637296+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:14:41.637296+0000","last_peered":"2024-05-27T10:14:41.637296+0000","last_clean":"2024-05-27T10:14:41.637296+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:14:41.637296+0000","last_undegraded":"2024-05-27T10:14:41.637296+0000","last_fullsized":"2024-05-27T10:14:41.637296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.379728+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:16:10.379728+0000","last_peered":"2024-05-27T10:16:10.379728+0000","last_clean":"2024-05-27T10:16:10.379728+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:16:10.379728+0000","last_undegraded":"2024-05-27T10:16:10.379728+0000","last_fullsized":"2024-05-27T10:16:10.379728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378900+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:16:10.378900+0000","last_peered":"2024-05-27T10:16:10.378900+0000","last_clean":"2024-05-27T10:16:10.378900+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:16:10.378900+0000","last_undegraded":"2024-05-27T10:16:10.378900+0000","last_fullsized":"2024-05-27T10:16:10.378900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378832+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:16:10.378832+0000","last_peered":"2024-05-27T10:16:10.378832+0000","last_clean":"2024-05-27T10:16:10.378832+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:16:10.378832+0000","last_undegraded":"2024-05-27T10:16:10.378832+0000","last_fullsized":"2024-05-27T10:16:10.378832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.397129+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:16:10.397129+0000","last_peered":"2024-05-27T10:16:10.397129+0000","last_clean":"2024-05-27T10:16:10.397129+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:16:10.397129+0000","last_undegraded":"2024-05-27T10:16:10.397129+0000","last_fullsized":"2024-05-27T10:16:10.397129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393730+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:16:10.393730+0000","last_peered":"2024-05-27T10:16:10.393730+0000","last_clean":"2024-05-27T10:16:10.393730+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:16:10.393730+0000","last_undegraded":"2024-05-27T10:16:10.393730+0000","last_fullsized":"2024-05-27T10:16:10.393730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.398037+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:16:10.398037+0000","last_peered":"2024-05-27T10:16:10.398037+0000","last_clean":"2024-05-27T10:16:10.398037+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:16:10.398037+0000","last_undegraded":"2024-05-27T10:16:10.398037+0000","last_fullsized":"2024-05-27T10:16:10.398037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377203+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:16:10.377203+0000","last_peered":"2024-05-27T10:16:10.377203+0000","last_clean":"2024-05-27T10:16:10.377203+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:16:10.377203+0000","last_undegraded":"2024-05-27T10:16:10.377203+0000","last_fullsized":"2024-05-27T10:16:10.377203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392482+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:16:10.392482+0000","last_peered":"2024-05-27T10:16:10.392482+0000","last_clean":"2024-05-27T10:16:10.392482+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:16:10.392482+0000","last_undegraded":"2024-05-27T10:16:10.392482+0000","last_fullsized":"2024-05-27T10:16:10.392482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377051+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:16:10.377051+0000","last_peered":"2024-05-27T10:16:10.377051+0000","last_clean":"2024-05-27T10:16:10.377051+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:16:10.377051+0000","last_undegraded":"2024-05-27T10:16:10.377051+0000","last_fullsized":"2024-05-27T10:16:10.377051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390861+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:16:10.390861+0000","last_peered":"2024-05-27T10:16:10.390861+0000","last_clean":"2024-05-27T10:16:10.390861+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:16:10.390861+0000","last_undegraded":"2024-05-27T10:16:10.390861+0000","last_fullsized":"2024-05-27T10:16:10.390861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393140+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:16:10.393140+0000","last_peered":"2024-05-27T10:16:10.393140+0000","last_clean":"2024-05-27T10:16:10.393140+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:16:10.393140+0000","last_undegraded":"2024-05-27T10:16:10.393140+0000","last_fullsized":"2024-05-27T10:16:10.393140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.398261+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:16:10.398261+0000","last_peered":"2024-05-27T10:16:10.398261+0000","last_clean":"2024-05-27T10:16:10.398261+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:16:10.398261+0000","last_undegraded":"2024-05-27T10:16:10.398261+0000","last_fullsized":"2024-05-27T10:16:10.398261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.397533+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:16:10.397533+0000","last_peered":"2024-05-27T10:16:10.397533+0000","last_clean":"2024-05-27T10:16:10.397533+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:16:10.397533+0000","last_undegraded":"2024-05-27T10:16:10.397533+0000","last_fullsized":"2024-05-27T10:16:10.397533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378459+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:16:10.378459+0000","last_peered":"2024-05-27T10:16:10.378459+0000","last_clean":"2024-05-27T10:16:10.378459+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:16:10.378459+0000","last_undegraded":"2024-05-27T10:16:10.378459+0000","last_fullsized":"2024-05-27T10:16:10.378459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63579,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":820,"num_read_kb":479,"num_write":175,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":428086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1079004,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037827,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":428322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234059,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":428242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430290,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":1020845,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593818,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":427711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":486409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":428982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":488291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149809,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":428022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953578,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":428680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":1019997,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:16:57.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:16:57.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:57 smithi042 ceph-mon[31661]: pgmap v469: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T10:16:57.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:57 smithi042 ceph-mon[31661]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:16:57.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:57 smithi104 ceph-mon[34812]: pgmap v469: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T10:16:57.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:57 smithi104 ceph-mon[34812]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:16:58.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:57 smithi143 ceph-mon[35545]: pgmap v469: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T10:16:58.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:57 smithi143 ceph-mon[35545]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:16:58.132 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:16:59.937 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:16:59 smithi042 ceph-mon[31661]: pgmap v470: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T10:16:59.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:16:59 smithi104 ceph-mon[34812]: pgmap v470: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T10:17:00.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:16:59 smithi143 ceph-mon[35545]: pgmap v470: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T10:17:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 -- 172.21.15.42:0/2993112088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c10b890 msgr2=0x7f8a5c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 --2- 172.21.15.42:0/2993112088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c10b890 0x7f8a5c1131f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a540099c0 tx=0x7f8a54009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 -- 172.21.15.42:0/2993112088 shutdown_connections 2024-05-27T10:17:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 --2- 172.21.15.42:0/2993112088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c10b890 0x7f8a5c1131f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 --2- 172.21.15.42:0/2993112088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c106610 0x7f8a5c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 --2- 172.21.15.42:0/2993112088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a5c10af40 0x7f8a5c10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 -- 172.21.15.42:0/2993112088 >> 172.21.15.42:0/2993112088 conn(0x7f8a5c102020 msgr2=0x7f8a5c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.561+0000 7f8a61000700 1 -- 172.21.15.42:0/2993112088 shutdown_connections 2024-05-27T10:17:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.562+0000 7f8a61000700 1 -- 172.21.15.42:0/2993112088 wait complete. 2024-05-27T10:17:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 Processor -- start 2024-05-27T10:17:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 -- start start 2024-05-27T10:17:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 0x7f8a5c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c10af40 0x7f8a5c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a5c10b890 0x7f8a5c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1163c0 con 0x7f8a5c106610 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c116240 con 0x7f8a5c10af40 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.563+0000 7f8a61000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c116540 con 0x7f8a5c10b890 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 0x7f8a5c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 0x7f8a5c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35864/0 (socket says 172.21.15.42:35864) 2024-05-27T10:17:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 -- 172.21.15.42:0/2807684769 learned_addr learned my addr 172.21.15.42:0/2807684769 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5be00700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a5c10b890 0x7f8a5c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a52a00700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c10af40 0x7f8a5c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 -- 172.21.15.42:0/2807684769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a5c10b890 msgr2=0x7f8a5c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a5c10b890 0x7f8a5c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 -- 172.21.15.42:0/2807684769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c10af40 msgr2=0x7f8a5c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c10af40 0x7f8a5c19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 -- 172.21.15.42:0/2807684769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c1a1c00 con 0x7f8a5c106610 2024-05-27T10:17:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.564+0000 7f8a5b400700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 0x7f8a5c19cd30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c007f10 tx=0x7f8a4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.565+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c003a00 con 0x7f8a5c106610 2024-05-27T10:17:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.565+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a4c003ba0 con 0x7f8a5c106610 2024-05-27T10:17:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.565+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c1a1ef0 con 0x7f8a5c106610 2024-05-27T10:17:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.565+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c015660 con 0x7f8a5c106610 2024-05-27T10:17:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.565+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1a9810 con 0x7f8a5c106610 2024-05-27T10:17:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.566+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c0512b0 con 0x7f8a5c106610 2024-05-27T10:17:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.569+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8a4c004000 con 0x7f8a5c106610 2024-05-27T10:17:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.572+0000 7f8a58c00700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8a400b2900 0x7f8a400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.572+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8a4c0e2690 con 0x7f8a5c106610 2024-05-27T10:17:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.572+0000 7f8a52a00700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8a400b2900 0x7f8a400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.573+0000 7f8a52a00700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8a400b2900 0x7f8a400b4e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a44005950 tx=0x7f8a440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.574+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a4c023070 con 0x7f8a5c106610 2024-05-27T10:17:00.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.810+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a5c063bf0 con 0x7f8a400b2900 2024-05-27T10:17:00.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.820+0000 7f8a58c00700 1 -- 172.21.15.42:0/2807684769 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258982 (secure 0 0 0) 0x7f8a5c063bf0 con 0x7f8a400b2900 2024-05-27T10:17:00.825 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.825+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8a400b2900 msgr2=0x7f8a400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.825+0000 7f8a61000700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8a400b2900 0x7f8a400b4e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a44005950 tx=0x7f8a440058e0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.825+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 msgr2=0x7f8a5c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 0x7f8a5c19cd30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c007f10 tx=0x7f8a4c00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 shutdown_connections 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a5c10b890 0x7f8a5c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a5c106610 0x7f8a5c19cd30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8a400b2900 0x7f8a400b4e10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 --2- 172.21.15.42:0/2807684769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a5c10af40 0x7f8a5c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:00.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.826+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 >> 172.21.15.42:0/2807684769 conn(0x7f8a5c102020 msgr2=0x7f8a5c10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:00.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.828+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 shutdown_connections 2024-05-27T10:17:00.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:00.828+0000 7f8a61000700 1 -- 172.21.15.42:0/2807684769 wait complete. 2024-05-27T10:17:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:17:01.812 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-05-27T10:17:00.468249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654181,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":931,"num_read_kb":563,"num_write":300,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359904,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581920,"statfs":{"total":1151940427776,"available":1151571886080,"internally_reserved":0,"allocated":11010048,"data_stored":7092591,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":39,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005940"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393767+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:16:10.393767+0000","last_peered":"2024-05-27T10:16:10.393767+0000","last_clean":"2024-05-27T10:16:10.393767+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:16:10.393767+0000","last_undegraded":"2024-05-27T10:16:10.393767+0000","last_fullsized":"2024-05-27T10:16:10.393767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392172+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:16:10.392172+0000","last_peered":"2024-05-27T10:16:10.392172+0000","last_clean":"2024-05-27T10:16:10.392172+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:16:10.392172+0000","last_undegraded":"2024-05-27T10:16:10.392172+0000","last_fullsized":"2024-05-27T10:16:10.392172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393525+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:16:10.393525+0000","last_peered":"2024-05-27T10:16:10.393525+0000","last_clean":"2024-05-27T10:16:10.393525+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:16:10.393525+0000","last_undegraded":"2024-05-27T10:16:10.393525+0000","last_fullsized":"2024-05-27T10:16:10.393525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378541+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:16:10.378541+0000","last_peered":"2024-05-27T10:16:10.378541+0000","last_clean":"2024-05-27T10:16:10.378541+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:16:10.378541+0000","last_undegraded":"2024-05-27T10:16:10.378541+0000","last_fullsized":"2024-05-27T10:16:10.378541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390872+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:16:10.390872+0000","last_peered":"2024-05-27T10:16:10.390872+0000","last_clean":"2024-05-27T10:16:10.390872+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:16:10.390872+0000","last_undegraded":"2024-05-27T10:16:10.390872+0000","last_fullsized":"2024-05-27T10:16:10.390872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.375579+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:16:10.375579+0000","last_peered":"2024-05-27T10:16:10.375579+0000","last_clean":"2024-05-27T10:16:10.375579+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:16:10.375579+0000","last_undegraded":"2024-05-27T10:16:10.375579+0000","last_fullsized":"2024-05-27T10:16:10.375579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377111+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:16:10.377111+0000","last_peered":"2024-05-27T10:16:10.377111+0000","last_clean":"2024-05-27T10:16:10.377111+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:16:10.377111+0000","last_undegraded":"2024-05-27T10:16:10.377111+0000","last_fullsized":"2024-05-27T10:16:10.377111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390434+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:16:10.390434+0000","last_peered":"2024-05-27T10:16:10.390434+0000","last_clean":"2024-05-27T10:16:10.390434+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:16:10.390434+0000","last_undegraded":"2024-05-27T10:16:10.390434+0000","last_fullsized":"2024-05-27T10:16:10.390434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392015+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:16:10.392015+0000","last_peered":"2024-05-27T10:16:10.392015+0000","last_clean":"2024-05-27T10:16:10.392015+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:16:10.392015+0000","last_undegraded":"2024-05-27T10:16:10.392015+0000","last_fullsized":"2024-05-27T10:16:10.392015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.375724+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:16:10.375724+0000","last_peered":"2024-05-27T10:16:10.375724+0000","last_clean":"2024-05-27T10:16:10.375724+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:16:10.375724+0000","last_undegraded":"2024-05-27T10:16:10.375724+0000","last_fullsized":"2024-05-27T10:16:10.375724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.387337+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:16:10.387337+0000","last_peered":"2024-05-27T10:16:10.387337+0000","last_clean":"2024-05-27T10:16:10.387337+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:16:10.387337+0000","last_undegraded":"2024-05-27T10:16:10.387337+0000","last_fullsized":"2024-05-27T10:16:10.387337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.374971+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:16:10.374971+0000","last_peered":"2024-05-27T10:16:10.374971+0000","last_clean":"2024-05-27T10:16:10.374971+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:16:10.374971+0000","last_undegraded":"2024-05-27T10:16:10.374971+0000","last_fullsized":"2024-05-27T10:16:10.374971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.388420+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:16:10.388420+0000","last_peered":"2024-05-27T10:16:10.388420+0000","last_clean":"2024-05-27T10:16:10.388420+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:16:10.388420+0000","last_undegraded":"2024-05-27T10:16:10.388420+0000","last_fullsized":"2024-05-27T10:16:10.388420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391553+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:16:10.391553+0000","last_peered":"2024-05-27T10:16:10.391553+0000","last_clean":"2024-05-27T10:16:10.391553+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:16:10.391553+0000","last_undegraded":"2024-05-27T10:16:10.391553+0000","last_fullsized":"2024-05-27T10:16:10.391553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.382898+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:16:10.382898+0000","last_peered":"2024-05-27T10:16:10.382898+0000","last_clean":"2024-05-27T10:16:10.382898+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:16:10.382898+0000","last_undegraded":"2024-05-27T10:16:10.382898+0000","last_fullsized":"2024-05-27T10:16:10.382898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.396771+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:16:10.396771+0000","last_peered":"2024-05-27T10:16:10.396771+0000","last_clean":"2024-05-27T10:16:10.396771+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:16:10.396771+0000","last_undegraded":"2024-05-27T10:16:10.396771+0000","last_fullsized":"2024-05-27T10:16:10.396771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394995+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:16:10.394995+0000","last_peered":"2024-05-27T10:16:10.394995+0000","last_clean":"2024-05-27T10:16:10.394995+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:16:10.394995+0000","last_undegraded":"2024-05-27T10:16:10.394995+0000","last_fullsized":"2024-05-27T10:16:10.394995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.387315+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:16:10.387315+0000","last_peered":"2024-05-27T10:16:10.387315+0000","last_clean":"2024-05-27T10:16:10.387315+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:16:10.387315+0000","last_undegraded":"2024-05-27T10:16:10.387315+0000","last_fullsized":"2024-05-27T10:16:10.387315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392661+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:16:10.392661+0000","last_peered":"2024-05-27T10:16:10.392661+0000","last_clean":"2024-05-27T10:16:10.392661+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:16:10.392661+0000","last_undegraded":"2024-05-27T10:16:10.392661+0000","last_fullsized":"2024-05-27T10:16:10.392661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.661226+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:14:41.661226+0000","last_peered":"2024-05-27T10:14:41.661226+0000","last_clean":"2024-05-27T10:14:41.661226+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:14:41.661226+0000","last_undegraded":"2024-05-27T10:14:41.661226+0000","last_fullsized":"2024-05-27T10:14:41.661226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391503+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:16:10.391503+0000","last_peered":"2024-05-27T10:16:10.391503+0000","last_clean":"2024-05-27T10:16:10.391503+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:16:10.391503+0000","last_undegraded":"2024-05-27T10:16:10.391503+0000","last_fullsized":"2024-05-27T10:16:10.391503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648255+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:14:41.648255+0000","last_peered":"2024-05-27T10:14:41.648255+0000","last_clean":"2024-05-27T10:14:41.648255+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:14:41.648255+0000","last_undegraded":"2024-05-27T10:14:41.648255+0000","last_fullsized":"2024-05-27T10:14:41.648255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394588+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:16:10.394588+0000","last_peered":"2024-05-27T10:16:10.394588+0000","last_clean":"2024-05-27T10:16:10.394588+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:16:10.394588+0000","last_undegraded":"2024-05-27T10:16:10.394588+0000","last_fullsized":"2024-05-27T10:16:10.394588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644580+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:14:41.644580+0000","last_peered":"2024-05-27T10:14:41.644580+0000","last_clean":"2024-05-27T10:14:41.644580+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:14:41.644580+0000","last_undegraded":"2024-05-27T10:14:41.644580+0000","last_fullsized":"2024-05-27T10:14:41.644580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.399544+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:16:10.399544+0000","last_peered":"2024-05-27T10:16:10.399544+0000","last_clean":"2024-05-27T10:16:10.399544+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:16:10.399544+0000","last_undegraded":"2024-05-27T10:16:10.399544+0000","last_fullsized":"2024-05-27T10:16:10.399544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.644093+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:14:41.644093+0000","last_peered":"2024-05-27T10:14:41.644093+0000","last_clean":"2024-05-27T10:14:41.644093+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:14:41.644093+0000","last_undegraded":"2024-05-27T10:14:41.644093+0000","last_fullsized":"2024-05-27T10:14:41.644093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.386315+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:16:10.386315+0000","last_peered":"2024-05-27T10:16:10.386315+0000","last_clean":"2024-05-27T10:16:10.386315+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:16:10.386315+0000","last_undegraded":"2024-05-27T10:16:10.386315+0000","last_fullsized":"2024-05-27T10:16:10.386315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":174,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:56.087257+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:16:56.087257+0000","last_peered":"2024-05-27T10:16:56.087257+0000","last_clean":"2024-05-27T10:16:56.087257+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:16:56.087257+0000","last_undegraded":"2024-05-27T10:16:56.087257+0000","last_fullsized":"2024-05-27T10:16:56.087257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.375765+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:16:10.375765+0000","last_peered":"2024-05-27T10:16:10.375765+0000","last_clean":"2024-05-27T10:16:10.375765+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:16:10.375765+0000","last_undegraded":"2024-05-27T10:16:10.375765+0000","last_fullsized":"2024-05-27T10:16:10.375765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.640251+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:14:41.640251+0000","last_peered":"2024-05-27T10:14:41.640251+0000","last_clean":"2024-05-27T10:14:41.640251+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:14:41.640251+0000","last_undegraded":"2024-05-27T10:14:41.640251+0000","last_fullsized":"2024-05-27T10:14:41.640251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391287+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:16:10.391287+0000","last_peered":"2024-05-27T10:16:10.391287+0000","last_clean":"2024-05-27T10:16:10.391287+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:16:10.391287+0000","last_undegraded":"2024-05-27T10:16:10.391287+0000","last_fullsized":"2024-05-27T10:16:10.391287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.636541+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:14:41.636541+0000","last_peered":"2024-05-27T10:14:41.636541+0000","last_clean":"2024-05-27T10:14:41.636541+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:14:41.636541+0000","last_undegraded":"2024-05-27T10:14:41.636541+0000","last_fullsized":"2024-05-27T10:14:41.636541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390238+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:16:10.390238+0000","last_peered":"2024-05-27T10:16:10.390238+0000","last_clean":"2024-05-27T10:16:10.390238+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:16:10.390238+0000","last_undegraded":"2024-05-27T10:16:10.390238+0000","last_fullsized":"2024-05-27T10:16:10.390238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652572+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:14:41.652572+0000","last_peered":"2024-05-27T10:14:41.652572+0000","last_clean":"2024-05-27T10:14:41.652572+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:14:41.652572+0000","last_undegraded":"2024-05-27T10:14:41.652572+0000","last_fullsized":"2024-05-27T10:14:41.652572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.395799+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:16:10.395799+0000","last_peered":"2024-05-27T10:16:10.395799+0000","last_clean":"2024-05-27T10:16:10.395799+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:16:10.395799+0000","last_undegraded":"2024-05-27T10:16:10.395799+0000","last_fullsized":"2024-05-27T10:16:10.395799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":168,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:57.089339+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:16:57.089339+0000","last_peered":"2024-05-27T10:16:57.089339+0000","last_clean":"2024-05-27T10:16:57.089339+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:16:57.089339+0000","last_undegraded":"2024-05-27T10:16:57.089339+0000","last_fullsized":"2024-05-27T10:16:57.089339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391413+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:16:10.391413+0000","last_peered":"2024-05-27T10:16:10.391413+0000","last_clean":"2024-05-27T10:16:10.391413+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:16:10.391413+0000","last_undegraded":"2024-05-27T10:16:10.391413+0000","last_fullsized":"2024-05-27T10:16:10.391413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642954+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:14:41.642954+0000","last_peered":"2024-05-27T10:14:41.642954+0000","last_clean":"2024-05-27T10:14:41.642954+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:14:41.642954+0000","last_undegraded":"2024-05-27T10:14:41.642954+0000","last_fullsized":"2024-05-27T10:14:41.642954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394457+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:16:10.394457+0000","last_peered":"2024-05-27T10:16:10.394457+0000","last_clean":"2024-05-27T10:16:10.394457+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:16:10.394457+0000","last_undegraded":"2024-05-27T10:16:10.394457+0000","last_fullsized":"2024-05-27T10:16:10.394457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.658256+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:14:41.658256+0000","last_peered":"2024-05-27T10:14:41.658256+0000","last_clean":"2024-05-27T10:14:41.658256+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:14:41.658256+0000","last_undegraded":"2024-05-27T10:14:41.658256+0000","last_fullsized":"2024-05-27T10:14:41.658256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.397694+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:16:10.397694+0000","last_peered":"2024-05-27T10:16:10.397694+0000","last_clean":"2024-05-27T10:16:10.397694+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:16:10.397694+0000","last_undegraded":"2024-05-27T10:16:10.397694+0000","last_fullsized":"2024-05-27T10:16:10.397694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'31","reported_seq":342,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:58.076353+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:16:58.076353+0000","last_peered":"2024-05-27T10:16:58.076353+0000","last_clean":"2024-05-27T10:16:58.076353+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:16:58.076353+0000","last_undegraded":"2024-05-27T10:16:58.076353+0000","last_fullsized":"2024-05-27T10:16:58.076353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58905,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":284,"num_read_kb":199,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.369550+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:16:10.369550+0000","last_peered":"2024-05-27T10:16:10.369550+0000","last_clean":"2024-05-27T10:16:10.369550+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:16:10.369550+0000","last_undegraded":"2024-05-27T10:16:10.369550+0000","last_fullsized":"2024-05-27T10:16:10.369550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648501+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:14:41.648501+0000","last_peered":"2024-05-27T10:14:41.648501+0000","last_clean":"2024-05-27T10:14:41.648501+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:14:41.648501+0000","last_undegraded":"2024-05-27T10:14:41.648501+0000","last_fullsized":"2024-05-27T10:14:41.648501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.399060+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:16:10.399060+0000","last_peered":"2024-05-27T10:16:10.399060+0000","last_clean":"2024-05-27T10:16:10.399060+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:16:10.399060+0000","last_undegraded":"2024-05-27T10:16:10.399060+0000","last_fullsized":"2024-05-27T10:16:10.399060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652886+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:14:41.652886+0000","last_peered":"2024-05-27T10:14:41.652886+0000","last_clean":"2024-05-27T10:14:41.652886+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:14:41.652886+0000","last_undegraded":"2024-05-27T10:14:41.652886+0000","last_fullsized":"2024-05-27T10:14:41.652886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390193+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:16:10.390193+0000","last_peered":"2024-05-27T10:16:10.390193+0000","last_clean":"2024-05-27T10:16:10.390193+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:16:10.390193+0000","last_undegraded":"2024-05-27T10:16:10.390193+0000","last_fullsized":"2024-05-27T10:16:10.390193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":171,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:55.080966+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:16:55.080966+0000","last_peered":"2024-05-27T10:16:55.080966+0000","last_clean":"2024-05-27T10:16:55.080966+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:16:55.080966+0000","last_undegraded":"2024-05-27T10:16:55.080966+0000","last_fullsized":"2024-05-27T10:16:55.080966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377079+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:16:10.377079+0000","last_peered":"2024-05-27T10:16:10.377079+0000","last_clean":"2024-05-27T10:16:10.377079+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:16:10.377079+0000","last_undegraded":"2024-05-27T10:16:10.377079+0000","last_fullsized":"2024-05-27T10:16:10.377079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.650658+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:14:41.650658+0000","last_peered":"2024-05-27T10:14:41.650658+0000","last_clean":"2024-05-27T10:14:41.650658+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:14:41.650658+0000","last_undegraded":"2024-05-27T10:14:41.650658+0000","last_fullsized":"2024-05-27T10:14:41.650658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.379141+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:16:10.379141+0000","last_peered":"2024-05-27T10:16:10.379141+0000","last_clean":"2024-05-27T10:16:10.379141+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:16:10.379141+0000","last_undegraded":"2024-05-27T10:16:10.379141+0000","last_fullsized":"2024-05-27T10:16:10.379141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.648826+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:14:41.648826+0000","last_peered":"2024-05-27T10:14:41.648826+0000","last_clean":"2024-05-27T10:14:41.648826+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:14:41.648826+0000","last_undegraded":"2024-05-27T10:14:41.648826+0000","last_fullsized":"2024-05-27T10:14:41.648826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.374617+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:16:10.374617+0000","last_peered":"2024-05-27T10:16:10.374617+0000","last_clean":"2024-05-27T10:16:10.374617+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:16:10.374617+0000","last_undegraded":"2024-05-27T10:16:10.374617+0000","last_fullsized":"2024-05-27T10:16:10.374617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.646918+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:14:41.646918+0000","last_peered":"2024-05-27T10:14:41.646918+0000","last_clean":"2024-05-27T10:14:41.646918+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:14:41.646918+0000","last_undegraded":"2024-05-27T10:14:41.646918+0000","last_fullsized":"2024-05-27T10:14:41.646918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391439+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:16:10.391439+0000","last_peered":"2024-05-27T10:16:10.391439+0000","last_clean":"2024-05-27T10:16:10.391439+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:16:10.391439+0000","last_undegraded":"2024-05-27T10:16:10.391439+0000","last_fullsized":"2024-05-27T10:16:10.391439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.642554+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:14:41.642554+0000","last_peered":"2024-05-27T10:14:41.642554+0000","last_clean":"2024-05-27T10:14:41.642554+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:14:41.642554+0000","last_undegraded":"2024-05-27T10:14:41.642554+0000","last_fullsized":"2024-05-27T10:14:41.642554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392985+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:16:10.392985+0000","last_peered":"2024-05-27T10:16:10.392985+0000","last_clean":"2024-05-27T10:16:10.392985+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:16:10.392985+0000","last_undegraded":"2024-05-27T10:16:10.392985+0000","last_fullsized":"2024-05-27T10:16:10.392985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.374917+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:16:10.374917+0000","last_peered":"2024-05-27T10:16:10.374917+0000","last_clean":"2024-05-27T10:16:10.374917+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:16:10.374917+0000","last_undegraded":"2024-05-27T10:16:10.374917+0000","last_fullsized":"2024-05-27T10:16:10.374917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":168,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:56.082827+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:16:56.082827+0000","last_peered":"2024-05-27T10:16:56.082827+0000","last_clean":"2024-05-27T10:16:56.082827+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:16:56.082827+0000","last_undegraded":"2024-05-27T10:16:56.082827+0000","last_fullsized":"2024-05-27T10:16:56.082827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.396650+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:16:10.396650+0000","last_peered":"2024-05-27T10:16:10.396650+0000","last_clean":"2024-05-27T10:16:10.396650+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:16:10.396650+0000","last_undegraded":"2024-05-27T10:16:10.396650+0000","last_fullsized":"2024-05-27T10:16:10.396650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.655991+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:14:41.655991+0000","last_peered":"2024-05-27T10:14:41.655991+0000","last_clean":"2024-05-27T10:14:41.655991+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:14:41.655991+0000","last_undegraded":"2024-05-27T10:14:41.655991+0000","last_fullsized":"2024-05-27T10:14:41.655991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643102+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:14:41.643102+0000","last_peered":"2024-05-27T10:14:41.643102+0000","last_clean":"2024-05-27T10:14:41.643102+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:14:41.643102+0000","last_undegraded":"2024-05-27T10:14:41.643102+0000","last_fullsized":"2024-05-27T10:14:41.643102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.385694+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:16:10.385694+0000","last_peered":"2024-05-27T10:16:10.385694+0000","last_clean":"2024-05-27T10:16:10.385694+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:16:10.385694+0000","last_undegraded":"2024-05-27T10:16:10.385694+0000","last_fullsized":"2024-05-27T10:16:10.385694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.643327+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:14:41.643327+0000","last_peered":"2024-05-27T10:14:41.643327+0000","last_clean":"2024-05-27T10:14:41.643327+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:14:41.643327+0000","last_undegraded":"2024-05-27T10:14:41.643327+0000","last_fullsized":"2024-05-27T10:14:41.643327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.650613+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:14:41.650613+0000","last_peered":"2024-05-27T10:14:41.650613+0000","last_clean":"2024-05-27T10:14:41.650613+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:14:41.650613+0000","last_undegraded":"2024-05-27T10:14:41.650613+0000","last_fullsized":"2024-05-27T10:14:41.650613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.383907+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:16:10.383907+0000","last_peered":"2024-05-27T10:16:10.383907+0000","last_clean":"2024-05-27T10:16:10.383907+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:16:10.383907+0000","last_undegraded":"2024-05-27T10:16:10.383907+0000","last_fullsized":"2024-05-27T10:16:10.383907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:11.220553+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:16:11.220553+0000","last_peered":"2024-05-27T10:16:11.220553+0000","last_clean":"2024-05-27T10:16:11.220553+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:16:11.220553+0000","last_undegraded":"2024-05-27T10:16:11.220553+0000","last_fullsized":"2024-05-27T10:16:11.220553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392646+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:16:10.392646+0000","last_peered":"2024-05-27T10:16:10.392646+0000","last_clean":"2024-05-27T10:16:10.392646+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:16:10.392646+0000","last_undegraded":"2024-05-27T10:16:10.392646+0000","last_fullsized":"2024-05-27T10:16:10.392646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.652618+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:14:41.652618+0000","last_peered":"2024-05-27T10:14:41.652618+0000","last_clean":"2024-05-27T10:14:41.652618+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:14:41.652618+0000","last_undegraded":"2024-05-27T10:14:41.652618+0000","last_fullsized":"2024-05-27T10:14:41.652618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.396797+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:16:10.396797+0000","last_peered":"2024-05-27T10:16:10.396797+0000","last_clean":"2024-05-27T10:16:10.396797+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:16:10.396797+0000","last_undegraded":"2024-05-27T10:16:10.396797+0000","last_fullsized":"2024-05-27T10:16:10.396797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:42.053356+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:14:42.053356+0000","last_peered":"2024-05-27T10:14:42.053356+0000","last_clean":"2024-05-27T10:14:42.053356+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:14:42.053356+0000","last_undegraded":"2024-05-27T10:14:42.053356+0000","last_fullsized":"2024-05-27T10:14:42.053356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.368416+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:16:10.368416+0000","last_peered":"2024-05-27T10:16:10.368416+0000","last_clean":"2024-05-27T10:16:10.368416+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:16:10.368416+0000","last_undegraded":"2024-05-27T10:16:10.368416+0000","last_fullsized":"2024-05-27T10:16:10.368416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.645532+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:14:41.645532+0000","last_peered":"2024-05-27T10:14:41.645532+0000","last_clean":"2024-05-27T10:14:41.645532+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:14:41.645532+0000","last_undegraded":"2024-05-27T10:14:41.645532+0000","last_fullsized":"2024-05-27T10:14:41.645532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392887+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:16:10.392887+0000","last_peered":"2024-05-27T10:16:10.392887+0000","last_clean":"2024-05-27T10:16:10.392887+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:16:10.392887+0000","last_undegraded":"2024-05-27T10:16:10.392887+0000","last_fullsized":"2024-05-27T10:16:10.392887+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.659780+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:14:41.659780+0000","last_peered":"2024-05-27T10:14:41.659780+0000","last_clean":"2024-05-27T10:14:41.659780+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:14:41.659780+0000","last_undegraded":"2024-05-27T10:14:41.659780+0000","last_fullsized":"2024-05-27T10:14:41.659780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.395969+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:16:10.395969+0000","last_peered":"2024-05-27T10:16:10.395969+0000","last_clean":"2024-05-27T10:16:10.395969+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:16:10.395969+0000","last_undegraded":"2024-05-27T10:16:10.395969+0000","last_fullsized":"2024-05-27T10:16:10.395969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.663558+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:14:41.663558+0000","last_peered":"2024-05-27T10:14:41.663558+0000","last_clean":"2024-05-27T10:14:41.663558+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:14:41.663558+0000","last_undegraded":"2024-05-27T10:14:41.663558+0000","last_fullsized":"2024-05-27T10:14:41.663558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392881+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:16:10.392881+0000","last_peered":"2024-05-27T10:16:10.392881+0000","last_clean":"2024-05-27T10:16:10.392881+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:16:10.392881+0000","last_undegraded":"2024-05-27T10:16:10.392881+0000","last_fullsized":"2024-05-27T10:16:10.392881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.661244+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:14:41.661244+0000","last_peered":"2024-05-27T10:14:41.661244+0000","last_clean":"2024-05-27T10:14:41.661244+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:14:41.661244+0000","last_undegraded":"2024-05-27T10:14:41.661244+0000","last_fullsized":"2024-05-27T10:14:41.661244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.391786+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:16:10.391786+0000","last_peered":"2024-05-27T10:16:10.391786+0000","last_clean":"2024-05-27T10:16:10.391786+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:16:10.391786+0000","last_undegraded":"2024-05-27T10:16:10.391786+0000","last_fullsized":"2024-05-27T10:16:10.391786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.654081+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:14:41.654081+0000","last_peered":"2024-05-27T10:14:41.654081+0000","last_clean":"2024-05-27T10:14:41.654081+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:14:41.654081+0000","last_undegraded":"2024-05-27T10:14:41.654081+0000","last_fullsized":"2024-05-27T10:14:41.654081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.394633+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:16:10.394633+0000","last_peered":"2024-05-27T10:16:10.394633+0000","last_clean":"2024-05-27T10:16:10.394633+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:16:10.394633+0000","last_undegraded":"2024-05-27T10:16:10.394633+0000","last_fullsized":"2024-05-27T10:16:10.394633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:14:41.637296+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:14:41.637296+0000","last_peered":"2024-05-27T10:14:41.637296+0000","last_clean":"2024-05-27T10:14:41.637296+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:14:41.637296+0000","last_undegraded":"2024-05-27T10:14:41.637296+0000","last_fullsized":"2024-05-27T10:14:41.637296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.379728+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:16:10.379728+0000","last_peered":"2024-05-27T10:16:10.379728+0000","last_clean":"2024-05-27T10:16:10.379728+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:16:10.379728+0000","last_undegraded":"2024-05-27T10:16:10.379728+0000","last_fullsized":"2024-05-27T10:16:10.379728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378900+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:16:10.378900+0000","last_peered":"2024-05-27T10:16:10.378900+0000","last_clean":"2024-05-27T10:16:10.378900+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:16:10.378900+0000","last_undegraded":"2024-05-27T10:16:10.378900+0000","last_fullsized":"2024-05-27T10:16:10.378900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378832+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:16:10.378832+0000","last_peered":"2024-05-27T10:16:10.378832+0000","last_clean":"2024-05-27T10:16:10.378832+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:16:10.378832+0000","last_undegraded":"2024-05-27T10:16:10.378832+0000","last_fullsized":"2024-05-27T10:16:10.378832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.397129+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:16:10.397129+0000","last_peered":"2024-05-27T10:16:10.397129+0000","last_clean":"2024-05-27T10:16:10.397129+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:16:10.397129+0000","last_undegraded":"2024-05-27T10:16:10.397129+0000","last_fullsized":"2024-05-27T10:16:10.397129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393730+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:16:10.393730+0000","last_peered":"2024-05-27T10:16:10.393730+0000","last_clean":"2024-05-27T10:16:10.393730+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:16:10.393730+0000","last_undegraded":"2024-05-27T10:16:10.393730+0000","last_fullsized":"2024-05-27T10:16:10.393730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.398037+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:16:10.398037+0000","last_peered":"2024-05-27T10:16:10.398037+0000","last_clean":"2024-05-27T10:16:10.398037+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:16:10.398037+0000","last_undegraded":"2024-05-27T10:16:10.398037+0000","last_fullsized":"2024-05-27T10:16:10.398037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377203+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:16:10.377203+0000","last_peered":"2024-05-27T10:16:10.377203+0000","last_clean":"2024-05-27T10:16:10.377203+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:16:10.377203+0000","last_undegraded":"2024-05-27T10:16:10.377203+0000","last_fullsized":"2024-05-27T10:16:10.377203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.392482+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:16:10.392482+0000","last_peered":"2024-05-27T10:16:10.392482+0000","last_clean":"2024-05-27T10:16:10.392482+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:16:10.392482+0000","last_undegraded":"2024-05-27T10:16:10.392482+0000","last_fullsized":"2024-05-27T10:16:10.392482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.377051+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:16:10.377051+0000","last_peered":"2024-05-27T10:16:10.377051+0000","last_clean":"2024-05-27T10:16:10.377051+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:16:10.377051+0000","last_undegraded":"2024-05-27T10:16:10.377051+0000","last_fullsized":"2024-05-27T10:16:10.377051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.390861+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:16:10.390861+0000","last_peered":"2024-05-27T10:16:10.390861+0000","last_clean":"2024-05-27T10:16:10.390861+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:16:10.390861+0000","last_undegraded":"2024-05-27T10:16:10.390861+0000","last_fullsized":"2024-05-27T10:16:10.390861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.393140+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:16:10.393140+0000","last_peered":"2024-05-27T10:16:10.393140+0000","last_clean":"2024-05-27T10:16:10.393140+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:16:10.393140+0000","last_undegraded":"2024-05-27T10:16:10.393140+0000","last_fullsized":"2024-05-27T10:16:10.393140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.398261+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:16:10.398261+0000","last_peered":"2024-05-27T10:16:10.398261+0000","last_clean":"2024-05-27T10:16:10.398261+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:16:10.398261+0000","last_undegraded":"2024-05-27T10:16:10.398261+0000","last_fullsized":"2024-05-27T10:16:10.398261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.397533+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:16:10.397533+0000","last_peered":"2024-05-27T10:16:10.397533+0000","last_clean":"2024-05-27T10:16:10.397533+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:16:10.397533+0000","last_undegraded":"2024-05-27T10:16:10.397533+0000","last_fullsized":"2024-05-27T10:16:10.397533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-27T10:16:10.378459+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:16:10.378459+0000","last_peered":"2024-05-27T10:16:10.378459+0000","last_clean":"2024-05-27T10:16:10.378459+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:16:10.378459+0000","last_undegraded":"2024-05-27T10:16:10.378459+0000","last_fullsized":"2024-05-27T10:16:10.378459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284341,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63579,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":850,"num_read_kb":494,"num_write":175,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":428086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874301,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1079004,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":428322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234060,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":428242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430291,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":1020845,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593819,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":427711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":486409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738503,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":428982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":488291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149810,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":428022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":428680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790051,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":1019997,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34527,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:17:01.817 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T10:17:01.817 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T10:17:01.817 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-27T10:17:01.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-27T10:17:01.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:01 smithi042 ceph-mon[31661]: pgmap v471: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-27T10:17:01.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:01 smithi104 ceph-mon[34812]: pgmap v471: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-27T10:17:02.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:01 smithi143 ceph-mon[35545]: pgmap v471: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-27T10:17:02.319 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:02.730 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:02 smithi042 ceph-mon[31661]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:17:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:02 smithi104 ceph-mon[34812]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:17:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:02 smithi143 ceph-mon[35545]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:17:03.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:03 smithi104 ceph-mon[34812]: pgmap v472: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:04.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:03 smithi143 ceph-mon[35545]: pgmap v472: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:04.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:03 smithi042 ceph-mon[31661]: pgmap v472: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:04.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 -- 172.21.15.42:0/3243438234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2238108f10 msgr2=0x7f2238109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:04.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 --2- 172.21.15.42:0/3243438234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2238108f10 0x7f2238109320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f22240099c0 tx=0x7f2224009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 -- 172.21.15.42:0/3243438234 shutdown_connections 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 --2- 172.21.15.42:0/3243438234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2238109860 0x7f22381111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 --2- 172.21.15.42:0/3243438234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22381045e0 0x7f22381049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 --2- 172.21.15.42:0/3243438234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2238108f10 0x7f2238109320 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 -- 172.21.15.42:0/3243438234 >> 172.21.15.42:0/3243438234 conn(0x7f22380ffeb0 msgr2=0x7f2238102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 -- 172.21.15.42:0/3243438234 shutdown_connections 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.825+0000 7f223f200700 1 -- 172.21.15.42:0/3243438234 wait complete. 2024-05-27T10:17:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.827+0000 7f223f200700 1 Processor -- start 2024-05-27T10:17:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.827+0000 7f223f200700 1 -- start start 2024-05-27T10:17:04.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223f200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 0x7f223819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223f200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2238108f10 0x7f223819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223f200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2238109860 0x7f223819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223f200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22381143b0 con 0x7f22381045e0 2024-05-27T10:17:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223f200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2238114230 con 0x7f2238108f10 2024-05-27T10:17:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223f200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2238114530 con 0x7f2238109860 2024-05-27T10:17:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f2237e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 0x7f223819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f2237e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 0x7f223819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60934/0 (socket says 172.21.15.42:60934) 2024-05-27T10:17:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f2237e00700 1 -- 172.21.15.42:0/3241843532 learned_addr learned my addr 172.21.15.42:0/3241843532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f223d200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2238109860 0x7f223819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.828+0000 7f2237400700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2238108f10 0x7f223819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237e00700 1 -- 172.21.15.42:0/3241843532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2238109860 msgr2=0x7f223819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237e00700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2238109860 0x7f223819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237e00700 1 -- 172.21.15.42:0/3241843532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2238108f10 msgr2=0x7f223819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237e00700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2238108f10 0x7f223819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237e00700 1 -- 172.21.15.42:0/3241843532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223819fcb0 con 0x7f22381045e0 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237400700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2238108f10 0x7f223819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2237e00700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 0x7f223819ade0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f222800b530 tx=0x7f222800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.829+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228015020 con 0x7f22381045e0 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.830+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2228011cc0 con 0x7f22381045e0 2024-05-27T10:17:04.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.830+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223819ffa0 con 0x7f22381045e0 2024-05-27T10:17:04.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.830+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228010dc0 con 0x7f22381045e0 2024-05-27T10:17:04.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.830+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2238076af0 con 0x7f22381045e0 2024-05-27T10:17:04.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.831+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2218005330 con 0x7f22381045e0 2024-05-27T10:17:04.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.834+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2228018020 con 0x7f22381045e0 2024-05-27T10:17:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.835+0000 7f2234c00700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f221c0b2880 0x7f221c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.836+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f22280e1f00 con 0x7f22381045e0 2024-05-27T10:17:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.836+0000 7f2237400700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f221c0b2880 0x7f221c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:04.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.837+0000 7f2237400700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f221c0b2880 0x7f221c0b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:04.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:04.838+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f222801f020 con 0x7f22381045e0 2024-05-27T10:17:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.129+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2218005620 con 0x7f22381045e0 2024-05-27T10:17:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.130+0000 7f2234c00700 1 -- 172.21.15.42:0/3241843532 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f22280a9bf0 con 0x7f22381045e0 2024-05-27T10:17:05.130 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:17:05.130 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T10:17:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f221c0b2880 msgr2=0x7f221c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f221c0b2880 0x7f221c0b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224005ab0 comp rx=0 tx=0).stop 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 msgr2=0x7f223819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 0x7f223819ade0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f222800b530 tx=0x7f222800b840 comp rx=0 tx=0).stop 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 shutdown_connections 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2238109860 0x7f223819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22381045e0 0x7f223819ade0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f221c0b2880 0x7f221c0b4d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 --2- 172.21.15.42:0/3241843532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2238108f10 0x7f223819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.133+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 >> 172.21.15.42:0/3241843532 conn(0x7f22380ffeb0 msgr2=0x7f22381022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.134+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 shutdown_connections 2024-05-27T10:17:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:05.134+0000 7f223f200700 1 -- 172.21.15.42:0/3241843532 wait complete. 2024-05-27T10:17:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:05 smithi104 ceph-mon[34812]: pgmap v473: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:05.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:05 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3241843532' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T10:17:05.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:05 smithi042 ceph-mon[31661]: pgmap v473: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:05.999 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:05 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3241843532' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T10:17:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:05 smithi143 ceph-mon[35545]: pgmap v473: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:05 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3241843532' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T10:17:06.061 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-27T10:17:06.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:17:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:07.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:07 smithi104 ceph-mon[34812]: pgmap v474: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:07.975 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:07 smithi042 ceph-mon[31661]: pgmap v474: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:08.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:07 smithi143 ceph-mon[35545]: pgmap v474: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.887+0000 7f2f61e00700 1 -- 172.21.15.42:0/3838456386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10af40 msgr2=0x7f2f5c10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.887+0000 7f2f61e00700 1 --2- 172.21.15.42:0/3838456386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10af40 0x7f2f5c10b350 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f480099c0 tx=0x7f2f48009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.887+0000 7f2f61e00700 1 -- 172.21.15.42:0/3838456386 shutdown_connections 2024-05-27T10:17:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.887+0000 7f2f61e00700 1 --2- 172.21.15.42:0/3838456386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.887+0000 7f2f61e00700 1 --2- 172.21.15.42:0/3838456386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c106610 0x7f2f5c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.887+0000 7f2f61e00700 1 --2- 172.21.15.42:0/3838456386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10af40 0x7f2f5c10b350 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.888+0000 7f2f61e00700 1 -- 172.21.15.42:0/3838456386 >> 172.21.15.42:0/3838456386 conn(0x7f2f5c102020 msgr2=0x7f2f5c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.888+0000 7f2f61e00700 1 -- 172.21.15.42:0/3838456386 shutdown_connections 2024-05-27T10:17:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.888+0000 7f2f61e00700 1 -- 172.21.15.42:0/3838456386 wait complete. 2024-05-27T10:17:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.890+0000 7f2f61e00700 1 Processor -- start 2024-05-27T10:17:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.890+0000 7f2f61e00700 1 -- start start 2024-05-27T10:17:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c106610 0x7f2f5c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:08.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c10af40 0x7f2f5c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:08.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:08.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c116370 con 0x7f2f5c10b890 2024-05-27T10:17:08.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c1161f0 con 0x7f2f5c106610 2024-05-27T10:17:08.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c1164f0 con 0x7f2f5c10af40 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60948/0 (socket says 172.21.15.42:60948) 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f61400700 1 -- 172.21.15.42:0/4216485975 learned_addr learned my addr 172.21.15.42:0/4216485975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f60a00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c106610 0x7f2f5c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.891+0000 7f2f5be00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c10af40 0x7f2f5c19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f61400700 1 -- 172.21.15.42:0/4216485975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c10af40 msgr2=0x7f2f5c19d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f61400700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c10af40 0x7f2f5c19d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f61400700 1 -- 172.21.15.42:0/4216485975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c106610 msgr2=0x7f2f5c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f61400700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c106610 0x7f2f5c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f61400700 1 -- 172.21.15.42:0/4216485975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c1a1c50 con 0x7f2f5c10b890 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f60a00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c106610 0x7f2f5c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f5be00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c10af40 0x7f2f5c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f61400700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1a15b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2f54007ee0 tx=0x7f2f5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.892+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400dbd0 con 0x7f2f5c10b890 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.893+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f5400dd70 con 0x7f2f5c10b890 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.893+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5c1a1f40 con 0x7f2f5c10b890 2024-05-27T10:17:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.893+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f540058d0 con 0x7f2f5c10b890 2024-05-27T10:17:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.893+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5c1a9860 con 0x7f2f5c10b890 2024-05-27T10:17:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.894+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5c0512b0 con 0x7f2f5c10b890 2024-05-27T10:17:08.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.897+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2f5401a020 con 0x7f2f5c10b890 2024-05-27T10:17:08.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.898+0000 7f2f59600700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2f440b2900 0x7f2f440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:08.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.898+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2f540e19d0 con 0x7f2f5c10b890 2024-05-27T10:17:08.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.899+0000 7f2f60a00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2f440b2900 0x7f2f440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:08.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.899+0000 7f2f60a00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2f440b2900 0x7f2f440b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000bcf0 tx=0x7f2f5000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:08.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:08.907+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f54010070 con 0x7f2f5c10b890 2024-05-27T10:17:09.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.197+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f5c02e5e0 con 0x7f2f5c10b890 2024-05-27T10:17:09.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.200+0000 7f2f59600700 1 -- 172.21.15.42:0/4216485975 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f2f540a96c0 con 0x7f2f5c10b890 2024-05-27T10:17:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:17:09.200 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:14:45.975282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:17:09.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2f440b2900 msgr2=0x7f2f440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:09.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2f440b2900 0x7f2f440b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000bcf0 tx=0x7f2f5000b2c0 comp rx=0 tx=0).stop 2024-05-27T10:17:09.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 msgr2=0x7f2f5c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:09.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1a15b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2f54007ee0 tx=0x7f2f5400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 shutdown_connections 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c10b890 0x7f2f5c1a15b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2f440b2900 0x7f2f440b4e10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5c106610 0x7f2f5c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 --2- 172.21.15.42:0/4216485975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c10af40 0x7f2f5c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.203+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 >> 172.21.15.42:0/4216485975 conn(0x7f2f5c102020 msgr2=0x7f2f5c10c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.204+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 shutdown_connections 2024-05-27T10:17:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:09.204+0000 7f2f61e00700 1 -- 172.21.15.42:0/4216485975 wait complete. 2024-05-27T10:17:09.205 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:17:10.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:09 smithi143 ceph-mon[35545]: pgmap v475: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:10.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:09 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4216485975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:10.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:09 smithi042 ceph-mon[31661]: pgmap v475: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:10.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:09 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4216485975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:10.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:17:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:09 smithi104 ceph-mon[34812]: pgmap v475: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:09 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4216485975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:11 smithi143 ceph-mon[35545]: pgmap v476: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:12.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:11 smithi042 ceph-mon[31661]: pgmap v476: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:11 smithi104 ceph-mon[34812]: pgmap v476: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 -- 172.21.15.42:0/2821061927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c10b890 msgr2=0x7f2d6c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 --2- 172.21.15.42:0/2821061927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c10b890 0x7f2d6c1131f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b0d0 tx=0x7f2d6400b3e0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 -- 172.21.15.42:0/2821061927 shutdown_connections 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 --2- 172.21.15.42:0/2821061927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c10b890 0x7f2d6c1131f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 --2- 172.21.15.42:0/2821061927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d6c106610 0x7f2d6c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 --2- 172.21.15.42:0/2821061927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 0x7f2d6c10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 -- 172.21.15.42:0/2821061927 >> 172.21.15.42:0/2821061927 conn(0x7f2d6c102020 msgr2=0x7f2d6c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 -- 172.21.15.42:0/2821061927 shutdown_connections 2024-05-27T10:17:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.196+0000 7f2d70a00700 1 -- 172.21.15.42:0/2821061927 wait complete. 2024-05-27T10:17:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.197+0000 7f2d70a00700 1 Processor -- start 2024-05-27T10:17:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 -- start start 2024-05-27T10:17:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 0x7f2d6c1a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 0x7f2d6c1a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d6c10b890 0x7f2d6c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1163c0 con 0x7f2d6c106610 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c116240 con 0x7f2d6c10af40 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d70a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c116540 con 0x7f2d6c10b890 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d6a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 0x7f2d6c1a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d6a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 0x7f2d6c1a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47438/0 (socket says 172.21.15.42:47438) 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.198+0000 7f2d6a800700 1 -- 172.21.15.42:0/1657354151 learned_addr learned my addr 172.21.15.42:0/1657354151 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6b200700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d6c10b890 0x7f2d6c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d69e00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 0x7f2d6c1a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6a800700 1 -- 172.21.15.42:0/1657354151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d6c10b890 msgr2=0x7f2d6c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6a800700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d6c10b890 0x7f2d6c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6a800700 1 -- 172.21.15.42:0/1657354151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 msgr2=0x7f2d6c1a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6a800700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 0x7f2d6c1a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6a800700 1 -- 172.21.15.42:0/1657354151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c19d150 con 0x7f2d6c106610 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d69e00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 0x7f2d6c1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.199+0000 7f2d6a800700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 0x7f2d6c1a28a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c9b0 tx=0x7f2d5c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.200+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c007b00 con 0x7f2d6c106610 2024-05-27T10:17:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.200+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5c007ca0 con 0x7f2d6c106610 2024-05-27T10:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.200+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c19d440 con 0x7f2d6c106610 2024-05-27T10:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.200+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c018660 con 0x7f2d6c106610 2024-05-27T10:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.200+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c1a9750 con 0x7f2d6c106610 2024-05-27T10:17:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.201+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6c0512b0 con 0x7f2d6c106610 2024-05-27T10:17:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.204+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d5c00f450 con 0x7f2d6c106610 2024-05-27T10:17:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.205+0000 7f2d63400700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2d500b2880 0x7f2d500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.205+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2d5c0e24e0 con 0x7f2d6c106610 2024-05-27T10:17:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.205+0000 7f2d69e00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2d500b2880 0x7f2d500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.206+0000 7f2d69e00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2d500b2880 0x7f2d500b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005fd0 tx=0x7f2d5800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:13.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.208+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d5c014030 con 0x7f2d6c106610 2024-05-27T10:17:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.497+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d6c19dfd0 con 0x7f2d6c106610 2024-05-27T10:17:13.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.499+0000 7f2d63400700 1 -- 172.21.15.42:0/1657354151 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f2d5c0aa260 con 0x7f2d6c106610 2024-05-27T10:17:13.499 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:17:13.499 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:14:45.975282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.502+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2d500b2880 msgr2=0x7f2d500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.502+0000 7f2d70a00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2d500b2880 0x7f2d500b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005fd0 tx=0x7f2d5800a5e0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.502+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 msgr2=0x7f2d6c1a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.502+0000 7f2d70a00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 0x7f2d6c1a28a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c9b0 tx=0x7f2d5c00ccc0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 shutdown_connections 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d6c10b890 0x7f2d6c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d6c106610 0x7f2d6c1a28a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f2d500b2880 0x7f2d500b4d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 --2- 172.21.15.42:0/1657354151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c10af40 0x7f2d6c1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 >> 172.21.15.42:0/1657354151 conn(0x7f2d6c102020 msgr2=0x7f2d6c10a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 shutdown_connections 2024-05-27T10:17:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:13.503+0000 7f2d70a00700 1 -- 172.21.15.42:0/1657354151 wait complete. 2024-05-27T10:17:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:17:13.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:13 smithi042 ceph-mon[31661]: pgmap v477: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:13.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:13 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1657354151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:13 smithi143 ceph-mon[35545]: pgmap v477: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:13 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1657354151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:14.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:13 smithi104 ceph-mon[34812]: pgmap v477: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:14.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:13 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1657354151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:14.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:17:15.010 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:16.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:15 smithi143 ceph-mon[35545]: pgmap v478: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:16.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:15 smithi042 ceph-mon[31661]: pgmap v478: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:16.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:15 smithi104 ceph-mon[34812]: pgmap v478: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 -- 172.21.15.42:0/3988268324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 msgr2=0x7f2a18113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3988268324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a18113350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009990 tx=0x7f2a10009ca0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 -- 172.21.15.42:0/3988268324 shutdown_connections 2024-05-27T10:17:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3988268324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a18113350 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3988268324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a1810c5b0 0x7f2a1810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3988268324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 0x7f2a18106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 -- 172.21.15.42:0/3988268324 >> 172.21.15.42:0/3988268324 conn(0x7f2a18102070 msgr2=0x7f2a181044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 -- 172.21.15.42:0/3988268324 shutdown_connections 2024-05-27T10:17:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.580+0000 7f2a1f000700 1 -- 172.21.15.42:0/3988268324 wait complete. 2024-05-27T10:17:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.582+0000 7f2a1f000700 1 Processor -- start 2024-05-27T10:17:17.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.582+0000 7f2a1f000700 1 -- start start 2024-05-27T10:17:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a1f000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 0x7f2a1819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a1f000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a1819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a1f000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a1810c5b0 0x7f2a181a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a1f000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2a181165e0 con 0x7f2a18106ec0 2024-05-27T10:17:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a1f000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18116460 con 0x7f2a18106570 2024-05-27T10:17:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a1f000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18116760 con 0x7f2a1810c5b0 2024-05-27T10:17:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a17400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a1819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a17400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a1819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47460/0 (socket says 172.21.15.42:47460) 2024-05-27T10:17:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.583+0000 7f2a17400700 1 -- 172.21.15.42:0/3531765838 learned_addr learned my addr 172.21.15.42:0/3531765838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17e00700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 0x7f2a1819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a1d000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a1810c5b0 0x7f2a181a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17400700 1 -- 172.21.15.42:0/3531765838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a1810c5b0 msgr2=0x7f2a181a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17400700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a1810c5b0 0x7f2a181a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17400700 1 -- 172.21.15.42:0/3531765838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 msgr2=0x7f2a1819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17400700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 0x7f2a1819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17400700 1 -- 172.21.15.42:0/3531765838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a181a1d30 con 0x7f2a18106ec0 2024-05-27T10:17:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.584+0000 7f2a17e00700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 0x7f2a1819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.585+0000 7f2a17400700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a1819d4b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c9b0 tx=0x7f2a0000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.585+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00007aa0 con 0x7f2a18106ec0 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.585+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a00007c40 con 0x7f2a18106ec0 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.585+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00018660 con 0x7f2a18106ec0 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.585+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a181a2020 con 0x7f2a18106ec0 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.585+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a181a2510 con 0x7f2a18106ec0 2024-05-27T10:17:17.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.586+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a180512b0 con 0x7f2a18106ec0 2024-05-27T10:17:17.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.591+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2a0000f450 con 0x7f2a18106ec0 2024-05-27T10:17:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.592+0000 7f2a14c00700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29fc0b2900 0x7f29fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.593+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2a000e24f0 con 0x7f2a18106ec0 2024-05-27T10:17:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.593+0000 7f2a17e00700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29fc0b2900 0x7f29fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.594+0000 7f2a17e00700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29fc0b2900 0x7f29fc0b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a08007fd0 tx=0x7f2a08007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.595+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a00014030 con 0x7f2a18106ec0 2024-05-27T10:17:17.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.842+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1802e5e0 con 0x7f2a18106ec0 2024-05-27T10:17:17.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.845+0000 7f2a14c00700 1 -- 172.21.15.42:0/3531765838 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22562 (secure 0 0 0) 0x7f2a000a9c10 con 0x7f2a18106ec0 2024-05-27T10:17:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:17:17.846 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:41.618756+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29fc0b2900 msgr2=0x7f29fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29fc0b2900 0x7f29fc0b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a08007fd0 tx=0x7f2a08007830 comp rx=0 tx=0).stop 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 msgr2=0x7f2a1819d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a1819d4b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c9b0 tx=0x7f2a0000cd70 comp rx=0 tx=0).stop 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 shutdown_connections 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a1810c5b0 0x7f2a181a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29fc0b2900 0x7f29fc0b4e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a18106570 0x7f2a1819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 --2- 172.21.15.42:0/3531765838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a18106ec0 0x7f2a1819d4b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:17.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 >> 172.21.15.42:0/3531765838 conn(0x7f2a18102070 msgr2=0x7f2a18104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:17.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 shutdown_connections 2024-05-27T10:17:17.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:17.848+0000 7f2a1f000700 1 -- 172.21.15.42:0/3531765838 wait complete. 2024-05-27T10:17:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:17 smithi143 ceph-mon[35545]: pgmap v479: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:18.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:17 smithi042 ceph-mon[31661]: pgmap v479: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:18.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:17 smithi104 ceph-mon[34812]: pgmap v479: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:18.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:17:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:18 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3531765838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:17:19.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:18 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3531765838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:17:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:18 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3531765838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:17:19.324 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:20.017 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:19 smithi042 ceph-mon[31661]: pgmap v480: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:20.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:19 smithi143 ceph-mon[35545]: pgmap v480: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:20.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:19 smithi104 ceph-mon[34812]: pgmap v480: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 -- 172.21.15.42:0/1420073312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8075d20 msgr2=0x7fc9b8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 --2- 172.21.15.42:0/1420073312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8075d20 0x7fc9b8076130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009990 tx=0x7fc9a8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 -- 172.21.15.42:0/1420073312 shutdown_connections 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 --2- 172.21.15.42:0/1420073312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 0x7fc9b8112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 --2- 172.21.15.42:0/1420073312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8075d20 0x7fc9b8076130 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 --2- 172.21.15.42:0/1420073312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9b8076670 0x7fc9b8110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.824+0000 7fc9b7200700 1 -- 172.21.15.42:0/1420073312 >> 172.21.15.42:0/1420073312 conn(0x7fc9b8079d00 msgr2=0x7fc9b807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:21.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.825+0000 7fc9b7200700 1 -- 172.21.15.42:0/1420073312 shutdown_connections 2024-05-27T10:17:21.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.825+0000 7fc9b7200700 1 -- 172.21.15.42:0/1420073312 wait complete. 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 Processor -- start 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 -- start start 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9b8075d20 0x7fc9b819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 0x7fc9b819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 0x7fc9b81a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8116340 con 0x7fc9b8076670 2024-05-27T10:17:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81161c0 con 0x7fc9b8075d20 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.826+0000 7fc9b7200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81164c0 con 0x7fc9b81109a0 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 0x7fc9b819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 0x7fc9b819d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47470/0 (socket says 172.21.15.42:47470) 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 -- 172.21.15.42:0/4124663291 learned_addr learned my addr 172.21.15.42:0/4124663291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5e00700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9b8075d20 0x7fc9b819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b6800700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 0x7fc9b81a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 -- 172.21.15.42:0/4124663291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 msgr2=0x7fc9b81a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 0x7fc9b81a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 -- 172.21.15.42:0/4124663291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9b8075d20 msgr2=0x7fc9b819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9b8075d20 0x7fc9b819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b5400700 1 -- 172.21.15.42:0/4124663291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b81a1bb0 con 0x7fc9b8076670 2024-05-27T10:17:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.827+0000 7fc9b6800700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 0x7fc9b81a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:17:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.828+0000 7fc9b5400700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 0x7fc9b819d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c920 tx=0x7fc9a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.828+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4007b00 con 0x7fc9b8076670 2024-05-27T10:17:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.828+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b81a1ea0 con 0x7fc9b8076670 2024-05-27T10:17:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.828+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81a9870 con 0x7fc9b8076670 2024-05-27T10:17:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.828+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a4004cc0 con 0x7fc9b8076670 2024-05-27T10:17:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.828+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40053f0 con 0x7fc9b8076670 2024-05-27T10:17:21.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.830+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc998005330 con 0x7fc9b8076670 2024-05-27T10:17:21.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.833+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc9a4004750 con 0x7fc9b8076670 2024-05-27T10:17:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.834+0000 7fc9aea00700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc99c0b2900 0x7fc99c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.835+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc9a40e1c70 con 0x7fc9b8076670 2024-05-27T10:17:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.835+0000 7fc9b5e00700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc99c0b2900 0x7fc99c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.836+0000 7fc9b5e00700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc99c0b2900 0x7fc99c0b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b5c0 tx=0x7fc9a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:21.840+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9a4014030 con 0x7fc9b8076670 2024-05-27T10:17:22.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:21 smithi143 ceph-mon[35545]: pgmap v481: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:22.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:21 smithi042 ceph-mon[31661]: pgmap v481: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.142+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc998005620 con 0x7fc9b8076670 2024-05-27T10:17:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.144+0000 7fc9aea00700 1 -- 172.21.15.42:0/4124663291 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7fc9a40a9960 con 0x7fc9b8076670 2024-05-27T10:17:22.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:17:22.145 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:14:45.975282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:17:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc99c0b2900 msgr2=0x7fc99c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc99c0b2900 0x7fc99c0b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b5c0 tx=0x7fc9a8005960 comp rx=0 tx=0).stop 2024-05-27T10:17:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 msgr2=0x7fc9b819d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 0x7fc9b819d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c920 tx=0x7fc9a400cc30 comp rx=0 tx=0).stop 2024-05-27T10:17:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 shutdown_connections 2024-05-27T10:17:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b81109a0 0x7fc9b81a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fc99c0b2900 0x7fc99c0b4e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9b8075d20 0x7fc9b819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 --2- 172.21.15.42:0/4124663291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b8076670 0x7fc9b819d330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 >> 172.21.15.42:0/4124663291 conn(0x7fc9b8079d00 msgr2=0x7fc9b8101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.148+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 shutdown_connections 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:22.149+0000 7fc9b7200700 1 -- 172.21.15.42:0/4124663291 wait complete. 2024-05-27T10:17:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:17:22.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:21 smithi104 ceph-mon[34812]: pgmap v481: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:22.875 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:22 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4124663291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:23.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:22 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4124663291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:23.110 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T10:14:18.127208+0000', 'modified': '2024-05-27T10:14:45.975282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34499, 'mds_1': 34466, 'mds_2': 34442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/4166809554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 4166809554}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 4166809554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6839/517039305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 517039305}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 517039305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34421': {'gid': 34421, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/3150585498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 3150585498}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 3150585498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.104:6835/3501163811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 3501163811}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 3501163811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34466': {'gid': 34466, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.104:6837/219118322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 219118322}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 219118322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34499': {'gid': 34499, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.104:6839/3864142797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3864142797}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3864142797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34499, 'qdb_cluster': [34466, 34499, 34442]} 2024-05-27T10:17:23.110 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-27T10:17:23.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T10:17:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:22 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4124663291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:17:23.594 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:24.004 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:23 smithi042 ceph-mon[31661]: pgmap v482: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:24.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:23 smithi143 ceph-mon[35545]: pgmap v482: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:24.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:23 smithi104 ceph-mon[34812]: pgmap v482: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:25.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:24 smithi143 ceph-mon[35545]: pgmap v483: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:25.075 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:24 smithi042 ceph-mon[31661]: pgmap v483: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:25.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:24 smithi104 ceph-mon[34812]: pgmap v483: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 -- 172.21.15.42:0/1727960158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 msgr2=0x7fd464106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 --2- 172.21.15.42:0/1727960158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd464106a40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd4540099c0 tx=0x7fd454009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 -- 172.21.15.42:0/1727960158 shutdown_connections 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 --2- 172.21.15.42:0/1727960158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46410b8b0 0x7fd464113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 --2- 172.21.15.42:0/1727960158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd464106a40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 --2- 172.21.15.42:0/1727960158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd46410af60 0x7fd46410b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 -- 172.21.15.42:0/1727960158 >> 172.21.15.42:0/1727960158 conn(0x7fd464101ec0 msgr2=0x7fd464104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 -- 172.21.15.42:0/1727960158 shutdown_connections 2024-05-27T10:17:26.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.016+0000 7fd469200700 1 -- 172.21.15.42:0/1727960158 wait complete. 2024-05-27T10:17:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.018+0000 7fd469200700 1 Processor -- start 2024-05-27T10:17:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.018+0000 7fd469200700 1 -- start start 2024-05-27T10:17:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.018+0000 7fd469200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd46419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd469200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd46410af60 0x7fd46419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd469200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46410b8b0 0x7fd4641a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd469200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd464116400 con 0x7fd464106630 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd469200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd464116280 con 0x7fd46410b8b0 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd469200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd464116580 con 0x7fd46410af60 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd463400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd46419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd463e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46410b8b0 0x7fd4641a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd463400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd46419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45526/0 (socket says 172.21.15.42:45526) 2024-05-27T10:17:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.019+0000 7fd463400700 1 -- 172.21.15.42:0/2742796438 learned_addr learned my addr 172.21.15.42:0/2742796438 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.020+0000 7fd463400700 1 -- 172.21.15.42:0/2742796438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd46410af60 msgr2=0x7fd46419d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.020+0000 7fd463400700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd46410af60 0x7fd46419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.020+0000 7fd463400700 1 -- 172.21.15.42:0/2742796438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46410b8b0 msgr2=0x7fd4641a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.020+0000 7fd463400700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46410b8b0 0x7fd4641a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.020+0000 7fd463400700 1 -- 172.21.15.42:0/2742796438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641a1bb0 con 0x7fd464106630 2024-05-27T10:17:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.020+0000 7fd463400700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd46419cd90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd454000c00 tx=0x7fd45400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:26.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.021+0000 7fd45be00700 1 -- 172.21.15.42:0/2742796438 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454027070 con 0x7fd464106630 2024-05-27T10:17:26.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.021+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4641a1e40 con 0x7fd464106630 2024-05-27T10:17:26.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.021+0000 7fd45be00700 1 -- 172.21.15.42:0/2742796438 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd45400fda0 con 0x7fd464106630 2024-05-27T10:17:26.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.021+0000 7fd45be00700 1 -- 172.21.15.42:0/2742796438 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454022e50 con 0x7fd464106630 2024-05-27T10:17:26.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.021+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a2320 con 0x7fd464106630 2024-05-27T10:17:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.025+0000 7fd45be00700 1 -- 172.21.15.42:0/2742796438 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd45402a070 con 0x7fd464106630 2024-05-27T10:17:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 Processor -- start 2024-05-27T10:17:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 -- start start 2024-05-27T10:17:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d73b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56454d65cdf0 con 0x56454d69b9a0 2024-05-27T10:17:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56454d64c990 con 0x56454d739270 2024-05-27T10:17:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.026+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56454d6509d0 con 0x56454d5afdf0 2024-05-27T10:17:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd463e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd463e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45538/0 (socket says 172.21.15.42:45538) 2024-05-27T10:17:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd463e00700 1 -- 172.21.15.42:0/444493834 learned_addr learned my addr 172.21.15.42:0/444493834 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd45be00700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd4480b2a20 0x7fd4480b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd463400700 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd462a00700 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d73b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.027+0000 7fd45be00700 1 -- 172.21.15.42:0/2742796438 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd4540e2f10 con 0x7fd464106630 2024-05-27T10:17:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 -- 172.21.15.42:0/444493834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 msgr2=0x56454d72fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 -- 172.21.15.42:0/444493834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 msgr2=0x56454d73b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d73b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 -- 172.21.15.42:0/444493834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454d6153d0 con 0x56454d69b9a0 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd4480b2a20 0x7fd4480b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd463e00700 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00f6d0 tx=0x7fd45c00f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ace324ba3587fa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd458c00700 1 -- 172.21.15.42:0/444493834 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c019070 con 0x56454d69b9a0 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.028+0000 7fd458c00700 1 -- 172.21.15.42:0/444493834 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd45c01fc60 con 0x56454d69b9a0 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.029+0000 7fd458c00700 1 -- 172.21.15.42:0/444493834 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c01ea10 con 0x56454d69b9a0 2024-05-27T10:17:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.029+0000 7fd463e00700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd4480b2a20 0x7fd4480b4f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd45c020000 tx=0x7fd45c01d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.029+0000 7fd4792ecb80 1 -- 172.21.15.42:0/444493834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 msgr2=0x56454d72f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.029+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00f6d0 tx=0x7fd45c00f9e0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 -- 172.21.15.42:0/444493834 shutdown_connections 2024-05-27T10:17:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d73b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/444493834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 -- 172.21.15.42:0/444493834 >> 172.21.15.42:0/444493834 conn(0x56454d649340 msgr2=0x56454d72a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 -- 172.21.15.42:0/444493834 shutdown_connections 2024-05-27T10:17:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 -- 172.21.15.42:0/444493834 wait complete. 2024-05-27T10:17:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.030+0000 7fd4792ecb80 1 -- start start 2024-05-27T10:17:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 Processor -- start 2024-05-27T10:17:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d736210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d69b9a0 0x56454d736750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d736c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56454d552f50 con 0x56454d5afdf0 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56454d5ed710 con 0x56454d739270 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56454d73bdb0 con 0x56454d69b9a0 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.031+0000 7fd4792ecb80 10 client.0 _ll_register_callbacks cb 0x56454d5f6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d736210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d736210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45542/0 (socket says 172.21.15.42:45542) 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 -- 172.21.15.42:0/2308117797 learned_addr learned my addr 172.21.15.42:0/2308117797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd462a00700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d69b9a0 0x56454d736750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463e00700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d736c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 -- 172.21.15.42:0/2308117797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d69b9a0 msgr2=0x56454d736750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d69b9a0 0x56454d736750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 -- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 msgr2=0x56454d736c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d736c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 -- 172.21.15.42:0/2308117797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454d59c880 con 0x56454d5afdf0 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463e00700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d736c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd463400700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d736210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd4540aa520 tx=0x7fd454073e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5779d13e9f4ad7e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.032+0000 7fd44d600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:17:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd44d600700 1 -- 172.21.15.42:0/2308117797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45406ffe0 con 0x56454d5afdf0 2024-05-27T10:17:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd44d600700 1 -- 172.21.15.42:0/2308117797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd4540094b0 con 0x56454d5afdf0 2024-05-27T10:17:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45c01fc60 con 0x56454d5afdf0 2024-05-27T10:17:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd44d600700 1 -- 172.21.15.42:0/2308117797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4540aac90 con 0x56454d5afdf0 2024-05-27T10:17:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd4792ecb80 10 client.14964 fetch_fsmap learned FSMap version 25 2024-05-27T10:17:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56454d61d010 con 0x56454d5afdf0 2024-05-27T10:17:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd44d600700 1 -- 172.21.15.42:0/2308117797 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd45c01fc60 con 0x56454d5afdf0 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.033+0000 7fd44d600700 1 -- 172.21.15.42:0/2308117797 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd4540aa7f0 con 0x56454d5afdf0 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd4792ecb80 10 client.14964 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd4792ecb80 10 client.14964 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd4792ecb80 20 client.14964 populate_metadata read hostname 'smithi042' 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d748d00 0x56454d74a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd4792ecb80 4 client.14964 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454d65ed40 con 0x56454d748d00 2024-05-27T10:17:26.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.034+0000 7fd462a00700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d748d00 0x56454d74a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.035+0000 7fd462a00700 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d748d00 0x56454d74a170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47328b93f20a6469 server_cookie=8a141c9b3475eefb in_seq=0 out_seq=0 2024-05-27T10:17:26.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.035+0000 7fd44d600700 10 client.14964 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:17:26.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.038+0000 7fd44d600700 1 -- 172.21.15.42:0/2308117797 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56454d65ed40 con 0x56454d748d00 2024-05-27T10:17:26.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.038+0000 7fd44d600700 10 client.14964 handle_command_reply: tid=0 2024-05-27T10:17:26.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.038+0000 7fd4792ecb80 1 client.14964 shutdown 2024-05-27T10:17:26.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 msgr2=0x56454d736210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d736210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd4540aa520 tx=0x7fd454073e80 comp rx=0 tx=0).stop 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 shutdown_connections 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d748d00 0x56454d74a170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d736c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d736210 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2308117797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d69b9a0 0x56454d736750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 >> 172.21.15.42:0/2308117797 conn(0x56454d649340 msgr2=0x56454d72f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd44d600700 0 client.14964 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:17:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 shutdown_connections 2024-05-27T10:17:26.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2308117797 wait complete. 2024-05-27T10:17:26.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.039+0000 7fd4792ecb80 20 client.14964 trim_cache size 0 max 16384 2024-05-27T10:17:26.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 Processor -- start 2024-05-27T10:17:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 -- start start 2024-05-27T10:17:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d72fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56454d7484b0 con 0x56454d69b9a0 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56454d748b80 con 0x56454d739270 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56454d749250 con 0x56454d5afdf0 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd462a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.063+0000 7fd462a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45548/0 (socket says 172.21.15.42:45548) 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd462a00700 1 -- 172.21.15.42:0/332998810 learned_addr learned my addr 172.21.15.42:0/332998810 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd463400700 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd463e00700 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d72fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd462a00700 1 -- 172.21.15.42:0/332998810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 msgr2=0x56454d72a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd462a00700 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd462a00700 1 -- 172.21.15.42:0/332998810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 msgr2=0x56454d72fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd462a00700 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d72fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd462a00700 1 -- 172.21.15.42:0/332998810 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454d730180 con 0x56454d69b9a0 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.064+0000 7fd463e00700 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d72fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.065+0000 7fd462a00700 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x56454d7311c0 tx=0x7fd4500058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c92209508ec1637 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.065+0000 7fd44fe00700 1 -- 172.21.15.42:0/332998810 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450013d00 con 0x56454d69b9a0 2024-05-27T10:17:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.065+0000 7fd44fe00700 1 -- 172.21.15.42:0/332998810 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd450004cc0 con 0x56454d69b9a0 2024-05-27T10:17:26.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.065+0000 7fd44fe00700 1 -- 172.21.15.42:0/332998810 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45001b3b0 con 0x56454d69b9a0 2024-05-27T10:17:26.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 -- 172.21.15.42:0/332998810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 msgr2=0x56454d72f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x56454d7311c0 tx=0x7fd4500058e0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 -- 172.21.15.42:0/332998810 shutdown_connections 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d739270 0x56454d72fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d5afdf0 0x56454d72a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/332998810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d69b9a0 0x56454d72f700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 -- 172.21.15.42:0/332998810 >> 172.21.15.42:0/332998810 conn(0x56454d649340 msgr2=0x56454d734b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 -- 172.21.15.42:0/332998810 shutdown_connections 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.066+0000 7fd4792ecb80 1 -- 172.21.15.42:0/332998810 wait complete. 2024-05-27T10:17:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.067+0000 7fd4792ecb80 1 -- start start 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.067+0000 7fd4792ecb80 1 Processor -- start 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d753c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d69b9a0 0x56454d74cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d739270 0x56454d74d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56454d749ff0 con 0x56454d5afdf0 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56454d749920 con 0x56454d69b9a0 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56454d74a6c0 con 0x56454d739270 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd4792ecb80 10 client.0 _ll_register_callbacks cb 0x56454d5f6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd463400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d753c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd463400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d753c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45562/0 (socket says 172.21.15.42:45562) 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd463400700 1 -- 172.21.15.42:0/2883181579 learned_addr learned my addr 172.21.15.42:0/2883181579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:26.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd463400700 1 -- 172.21.15.42:0/2883181579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d739270 msgr2=0x56454d74d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd462a00700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d69b9a0 0x56454d74cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.068+0000 7fd463e00700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d739270 0x56454d74d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd463400700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d739270 0x56454d74d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd463400700 1 -- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d69b9a0 msgr2=0x56454d74cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd463400700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d69b9a0 0x56454d74cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd463400700 1 -- 172.21.15.42:0/2883181579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454d74dc00 con 0x56454d5afdf0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd462a00700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d69b9a0 0x56454d74cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd463400700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d753c30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd4540aa520 tx=0x7fd454009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f225827f33fdd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd44d600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd44d600700 1 -- 172.21.15.42:0/2883181579 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540a98e0 con 0x56454d5afdf0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd44d600700 1 -- 172.21.15.42:0/2883181579 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd4540a9a80 con 0x56454d5afdf0 2024-05-27T10:17:26.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd450004cc0 con 0x56454d5afdf0 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd44d600700 1 -- 172.21.15.42:0/2883181579 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd454070020 con 0x56454d5afdf0 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd4792ecb80 10 client.14970 fetch_fsmap learned FSMap version 25 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.069+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56454d74dc00 con 0x56454d5afdf0 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd44d600700 1 -- 172.21.15.42:0/2883181579 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd450004cc0 con 0x56454d5afdf0 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd44d600700 1 -- 172.21.15.42:0/2883181579 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd4540cb340 con 0x56454d5afdf0 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd4792ecb80 10 client.14970 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd4792ecb80 10 client.14970 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd4792ecb80 20 client.14970 populate_metadata read hostname 'smithi042' 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d7b64d0 0x56454d7b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd4792ecb80 4 client.14970 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.070+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "status"}) -- 0x56454d658520 con 0x56454d7b64d0 2024-05-27T10:17:26.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.071+0000 7fd462a00700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d7b64d0 0x56454d7b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.072+0000 7fd462a00700 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d7b64d0 0x56454d7b7940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9915dc0b6fede665 server_cookie=dbe979018a6c30ff in_seq=0 out_seq=0 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.072+0000 7fd44d600700 10 client.14970 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.072+0000 7fd44d600700 1 -- 172.21.15.42:0/2883181579 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56454d658520 con 0x56454d7b64d0 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.072+0000 7fd44d600700 10 client.14970 handle_command_reply: tid=0 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.072+0000 7fd4792ecb80 1 client.14970 shutdown 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 msgr2=0x56454d753c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d753c30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd4540aa520 tx=0x7fd454009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 shutdown_connections 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56454d7b64d0 0x56454d7b7940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56454d739270 0x56454d74d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56454d5afdf0 0x56454d753c30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 --2- 172.21.15.42:0/2883181579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454d69b9a0 0x56454d74cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 >> 172.21.15.42:0/2883181579 conn(0x56454d649340 msgr2=0x56454d7311f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd44d600700 0 client.14970 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 shutdown_connections 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 1 -- 172.21.15.42:0/2883181579 wait complete. 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.073+0000 7fd4792ecb80 20 client.14970 trim_cache size 0 max 16384 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34442, 2024-05-27T10:17:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 168.482745096, 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 25, 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 81, 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 242.90085528200001 2024-05-27T10:17:26.076 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd4480b2a20 msgr2=0x7fd4480b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd4480b2a20 0x7fd4480b4f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd45c020000 tx=0x7fd45c01d000 comp rx=0 tx=0).stop 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 msgr2=0x7fd46419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd46419cd90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd454000c00 tx=0x7fd45400f460 comp rx=0 tx=0).stop 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 shutdown_connections 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fd4480b2a20 0x7fd4480b4f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46410b8b0 0x7fd4641a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd464106630 0x7fd46419cd90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 --2- 172.21.15.42:0/2742796438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd46410af60 0x7fd46419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 >> 172.21.15.42:0/2742796438 conn(0x7fd464101ec0 msgr2=0x7fd4641042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:26.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 shutdown_connections 2024-05-27T10:17:26.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:26.074+0000 7fd469200700 1 -- 172.21.15.42:0/2742796438 wait complete. 2024-05-27T10:17:27.066 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T10:17:27.643 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:27.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:27 smithi042 ceph-mon[31661]: pgmap v484: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:27 smithi143 ceph-mon[35545]: pgmap v484: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:27.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:27 smithi104 ceph-mon[34812]: pgmap v484: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:29.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:29 smithi042 ceph-mon[31661]: pgmap v485: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:29.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:29 smithi143 ceph-mon[35545]: pgmap v485: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:29.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:29 smithi104 ceph-mon[34812]: pgmap v485: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:30.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 -- 172.21.15.42:0/1544423093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc1043f0 msgr2=0x7f42dc104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 --2- 172.21.15.42:0/1544423093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc1043f0 0x7f42dc104800 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0099c0 tx=0x7f42cc009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 -- 172.21.15.42:0/1544423093 shutdown_connections 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 --2- 172.21.15.42:0/1544423093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42dc109670 0x7f42dc110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 --2- 172.21.15.42:0/1544423093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc1043f0 0x7f42dc104800 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 --2- 172.21.15.42:0/1544423093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc108d20 0x7f42dc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 -- 172.21.15.42:0/1544423093 >> 172.21.15.42:0/1544423093 conn(0x7f42dc079d00 msgr2=0x7f42dc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 -- 172.21.15.42:0/1544423093 shutdown_connections 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.383+0000 7f42db200700 1 -- 172.21.15.42:0/1544423093 wait complete. 2024-05-27T10:17:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.385+0000 7f42db200700 1 Processor -- start 2024-05-27T10:17:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.385+0000 7f42db200700 1 -- start start 2024-05-27T10:17:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42db200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42dc1043f0 0x7f42dc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42db200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 0x7f42dc19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42db200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc109670 0x7f42dc1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42db200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1141a0 con 0x7f42dc108d20 2024-05-27T10:17:30.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42db200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc114020 con 0x7f42dc109670 2024-05-27T10:17:30.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42db200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc114320 con 0x7f42dc1043f0 2024-05-27T10:17:30.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42d9400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 0x7f42dc19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42d9400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 0x7f42dc19d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45584/0 (socket says 172.21.15.42:45584) 2024-05-27T10:17:30.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42d9400700 1 -- 172.21.15.42:0/982763545 learned_addr learned my addr 172.21.15.42:0/982763545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:30.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42d9e00700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42dc1043f0 0x7f42dc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.386+0000 7f42da800700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc109670 0x7f42dc1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d9400700 1 -- 172.21.15.42:0/982763545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42dc1043f0 msgr2=0x7f42dc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d9400700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42dc1043f0 0x7f42dc19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d9400700 1 -- 172.21.15.42:0/982763545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc109670 msgr2=0x7f42dc1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d9400700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc109670 0x7f42dc1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d9400700 1 -- 172.21.15.42:0/982763545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc1a1bd0 con 0x7f42dc108d20 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42da800700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc109670 0x7f42dc1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d9400700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 0x7f42dc19d350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f42c800c9e0 tx=0x7f42c800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d2a00700 1 -- 172.21.15.42:0/982763545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8007aa0 con 0x7f42dc108d20 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d2a00700 1 -- 172.21.15.42:0/982763545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42c8007c40 con 0x7f42dc108d20 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42d2a00700 1 -- 172.21.15.42:0/982763545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8018660 con 0x7f42dc108d20 2024-05-27T10:17:30.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.387+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc1a1ec0 con 0x7f42dc108d20 2024-05-27T10:17:30.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.388+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc1a9890 con 0x7f42dc108d20 2024-05-27T10:17:30.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.392+0000 7f42d2a00700 1 -- 172.21.15.42:0/982763545 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f42c800f450 con 0x7f42dc108d20 2024-05-27T10:17:30.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.392+0000 7f42ef7f4b80 1 Processor -- start 2024-05-27T10:17:30.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 -- start start 2024-05-27T10:17:30.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b49e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b56fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559975a78df0 con 0x5599759cbdf0 2024-05-27T10:17:30.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559975a68990 con 0x559975b4a7d0 2024-05-27T10:17:30.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559975a6c9d0 con 0x559975b4a3c0 2024-05-27T10:17:30.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42da800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b49e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42da800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b49e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45600/0 (socket says 172.21.15.42:45600) 2024-05-27T10:17:30.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.393+0000 7f42da800700 1 -- 172.21.15.42:0/1979295267 learned_addr learned my addr 172.21.15.42:0/1979295267 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:30.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42d9e00700 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42d9400700 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b56fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42da800700 1 -- 172.21.15.42:0/1979295267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 msgr2=0x559975b54a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42da800700 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b54a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42da800700 1 -- 172.21.15.42:0/1979295267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 msgr2=0x559975b56fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42da800700 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b56fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42da800700 1 -- 172.21.15.42:0/1979295267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559975a313d0 con 0x5599759cbdf0 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42d9400700 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42d2a00700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42c00b2900 0x7f42c00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42d2a00700 1 -- 172.21.15.42:0/982763545 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42c80e2160 con 0x7f42dc108d20 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42da800700 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b49e80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009dd0 tx=0x7f42d40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37da0f9cbb452a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.394+0000 7f42d9e00700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42c00b2900 0x7f42c00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.395+0000 7f42c7400700 1 -- 172.21.15.42:0/1979295267 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004bb0 con 0x5599759cbdf0 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.395+0000 7f42c7400700 1 -- 172.21.15.42:0/1979295267 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42d4004d50 con 0x5599759cbdf0 2024-05-27T10:17:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.395+0000 7f42c7400700 1 -- 172.21.15.42:0/1979295267 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4014670 con 0x5599759cbdf0 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/1979295267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 msgr2=0x559975b49e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42d9e00700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42c00b2900 0x7f42c00b4e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42cc000c00 tx=0x7f42cc005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b49e80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009dd0 tx=0x7f42d40077e0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/1979295267 shutdown_connections 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b49e80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/1979295267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/1979295267 >> 172.21.15.42:0/1979295267 conn(0x559975a2bf70 msgr2=0x559975b45ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/1979295267 shutdown_connections 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.396+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/1979295267 wait complete. 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.397+0000 7f42ef7f4b80 1 -- start start 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.397+0000 7f42ef7f4b80 1 Processor -- start 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 0x559975b54ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b55020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55997596ef50 con 0x559975b4a3c0 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559975a09710 con 0x559975b4a7d0 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559975b57940 con 0x5599759cbdf0 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42ef7f4b80 10 client.0 _ll_register_callbacks cb 0x559975a12c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42d9400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 0x559975b54ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42d9400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 0x559975b54ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45602/0 (socket says 172.21.15.42:45602) 2024-05-27T10:17:30.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42d9400700 1 -- 172.21.15.42:0/3498209574 learned_addr learned my addr 172.21.15.42:0/3498209574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:30.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42da800700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b55020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.398+0000 7f42d9400700 1 -- 172.21.15.42:0/3498209574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 msgr2=0x559975b545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42d9400700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42d9400700 1 -- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 msgr2=0x559975b55020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42d9400700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b55020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42d9400700 1 -- 172.21.15.42:0/3498209574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599759b8880 con 0x559975b4a3c0 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42d9400700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 0x559975b54ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42c80a9450 tx=0x7f42c80a9d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15c904802993f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42bbe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42bbe00700 1 -- 172.21.15.42:0/3498209574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c80a9e50 con 0x559975b4a3c0 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42bbe00700 1 -- 172.21.15.42:0/3498209574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42c8091300 con 0x559975b4a3c0 2024-05-27T10:17:30.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d4004d50 con 0x559975b4a3c0 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.399+0000 7f42bbe00700 1 -- 172.21.15.42:0/3498209574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f42c8007df0 con 0x559975b4a3c0 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 10 client.14982 fetch_fsmap learned FSMap version 25 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559975a39010 con 0x559975b4a3c0 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42bbe00700 1 -- 172.21.15.42:0/3498209574 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42d4004d50 con 0x559975b4a3c0 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42bbe00700 1 -- 172.21.15.42:0/3498209574 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f42c80beba0 con 0x559975b4a3c0 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 10 client.14982 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 20 client.14982 populate_metadata read hostname 'smithi042' 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975b648e0 0x559975b65d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 4 client.14982 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.400+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559975a7ad40 con 0x559975b648e0 2024-05-27T10:17:30.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.401+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975b648e0 0x559975b65d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.402+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975b648e0 0x559975b65d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d68f4bf4594d7ec0 server_cookie=e37f218950ec8817 in_seq=0 out_seq=0 2024-05-27T10:17:30.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.402+0000 7f42bbe00700 10 client.14982 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.404+0000 7f42bbe00700 1 -- 172.21.15.42:0/3498209574 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559975a7ad40 con 0x559975b648e0 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.404+0000 7f42bbe00700 10 client.14982 handle_command_reply: tid=0 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.404+0000 7f42ef7f4b80 1 client.14982 shutdown 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 msgr2=0x559975b54ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 0x559975b54ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42c80a9450 tx=0x7f42c80a9d60 comp rx=0 tx=0).stop 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 shutdown_connections 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975b648e0 0x559975b65d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3498209574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a3c0 0x559975b54ae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 >> 172.21.15.42:0/3498209574 conn(0x559975a2bf70 msgr2=0x559975b44fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:30.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42bbe00700 0 client.14982 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:17:30.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 shutdown_connections 2024-05-27T10:17:30.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3498209574 wait complete. 2024-05-27T10:17:30.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.405+0000 7f42ef7f4b80 20 client.14982 trim_cache size 0 max 16384 2024-05-27T10:17:30.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 Processor -- start 2024-05-27T10:17:30.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 -- start start 2024-05-27T10:17:30.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b45e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559975b64090 con 0x5599759cbdf0 2024-05-27T10:17:30.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559975b64760 con 0x559975b4a7d0 2024-05-27T10:17:30.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.429+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559975b64e30 con 0x559975b4a3c0 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b45e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b45e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45608/0 (socket says 172.21.15.42:45608) 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 -- 172.21.15.42:0/3772719219 learned_addr learned my addr 172.21.15.42:0/3772719219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9400700 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42da800700 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 -- 172.21.15.42:0/3772719219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 msgr2=0x559975b3ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b3ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 -- 172.21.15.42:0/3772719219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 msgr2=0x559975b60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b60be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.430+0000 7f42d9e00700 1 -- 172.21.15.42:0/3772719219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559975b47300 con 0x5599759cbdf0 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.431+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b45e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0051c0 tx=0x7f42cc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e779a7f6aa217053 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.431+0000 7f42c6a00700 1 -- 172.21.15.42:0/3772719219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00bbf0 con 0x5599759cbdf0 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.431+0000 7f42c6a00700 1 -- 172.21.15.42:0/3772719219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42cc00bd90 con 0x5599759cbdf0 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.431+0000 7f42c6a00700 1 -- 172.21.15.42:0/3772719219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00f850 con 0x5599759cbdf0 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3772719219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 msgr2=0x559975b45e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b45e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0051c0 tx=0x7f42cc020000 comp rx=0 tx=0).stop 2024-05-27T10:17:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3772719219 shutdown_connections 2024-05-27T10:17:30.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a7d0 0x559975b60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599759cbdf0 0x559975b45e80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3772719219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559975b4a3c0 0x559975b3ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3772719219 >> 172.21.15.42:0/3772719219 conn(0x559975a2bf70 msgr2=0x559975b45770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:30.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3772719219 shutdown_connections 2024-05-27T10:17:30.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.432+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3772719219 wait complete. 2024-05-27T10:17:30.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.433+0000 7f42ef7f4b80 1 -- start start 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.433+0000 7f42ef7f4b80 1 Processor -- start 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.433+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a3c0 0x559975b69c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42ef7f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 0x559975b6dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559975b65bd0 con 0x559975b4a7d0 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559975b65500 con 0x559975b4a3c0 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42ef7f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559975b662a0 con 0x5599759cbdf0 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42ef7f4b80 10 client.0 _ll_register_callbacks cb 0x559975a12c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 0x559975b6dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 0x559975b6dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45610/0 (socket says 172.21.15.42:45610) 2024-05-27T10:17:30.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 -- 172.21.15.42:0/3200302960 learned_addr learned my addr 172.21.15.42:0/3200302960 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42d9400700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a3c0 0x559975b69c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 -- 172.21.15.42:0/3200302960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 msgr2=0x559975b696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 -- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a3c0 msgr2=0x559975b69c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a3c0 0x559975b69c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.434+0000 7f42da800700 1 -- 172.21.15.42:0/3200302960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559975b61120 con 0x559975b4a7d0 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.435+0000 7f42da800700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 0x559975b6dc80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009310 tx=0x7f42d4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba60dbae706c984f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.435+0000 7f42bbe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.435+0000 7f42bbe00700 1 -- 172.21.15.42:0/3200302960 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4023070 con 0x559975b4a7d0 2024-05-27T10:17:30.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.435+0000 7f42bbe00700 1 -- 172.21.15.42:0/3200302960 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42d4005630 con 0x559975b4a7d0 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.435+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42cc00bd90 con 0x559975b4a7d0 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.435+0000 7f42bbe00700 1 -- 172.21.15.42:0/3200302960 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f42d402f070 con 0x559975b4a7d0 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 10 client.14988 fetch_fsmap learned FSMap version 25 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559975bd0690 con 0x559975b4a7d0 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42bbe00700 1 -- 172.21.15.42:0/3200302960 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42cc00bd90 con 0x559975b4a7d0 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42bbe00700 1 -- 172.21.15.42:0/3200302960 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f42d4028cd0 con 0x559975b4a7d0 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 10 client.14988 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 20 client.14988 populate_metadata read hostname 'smithi042' 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975bd2760 0x559975bd3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 4 client.14988 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:17:30.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.436+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "status"}) -- 0x559975a74520 con 0x559975bd2760 2024-05-27T10:17:30.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.437+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975bd2760 0x559975bd3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:30.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.438+0000 7f42d9e00700 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975bd2760 0x559975bd3bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62aed6f4fe8b2f4a server_cookie=1a127bec7ec23442 in_seq=0 out_seq=0 2024-05-27T10:17:30.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.438+0000 7f42bbe00700 10 client.14988 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.438+0000 7f42bbe00700 1 -- 172.21.15.42:0/3200302960 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559975a74520 con 0x559975bd2760 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.438+0000 7f42bbe00700 10 client.14988 handle_command_reply: tid=0 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.438+0000 7f42ef7f4b80 1 client.14988 shutdown 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 msgr2=0x559975b6dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 0x559975b6dc80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009310 tx=0x7f42d4004f90 comp rx=0 tx=0).stop 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 shutdown_connections 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x559975bd2760 0x559975bd3bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559975b4a7d0 0x559975b6dc80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599759cbdf0 0x559975b696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 --2- 172.21.15.42:0/3200302960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559975b4a3c0 0x559975b69c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 >> 172.21.15.42:0/3200302960 conn(0x559975a2bf70 msgr2=0x559975b54560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42bbe00700 0 client.14988 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 shutdown_connections 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 1 -- 172.21.15.42:0/3200302960 wait complete. 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.439+0000 7f42ef7f4b80 20 client.14988 trim_cache size 0 max 16384 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34466, 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:17:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 173.863354759, 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 25, 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 81, 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 234.97783482599999 2024-05-27T10:17:30.444 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42c00b2900 msgr2=0x7f42c00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42c00b2900 0x7f42c00b4e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42cc000c00 tx=0x7f42cc005040 comp rx=0 tx=0).stop 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 msgr2=0x7f42dc19d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 0x7f42dc19d350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f42c800c9e0 tx=0x7f42c800ccf0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 shutdown_connections 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f42c00b2900 0x7f42c00b4e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42dc109670 0x7f42dc1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42dc1043f0 0x7f42dc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 --2- 172.21.15.42:0/982763545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42dc108d20 0x7f42dc19d350 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 >> 172.21.15.42:0/982763545 conn(0x7f42dc079d00 msgr2=0x7f42dc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:30.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 shutdown_connections 2024-05-27T10:17:30.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:30.440+0000 7f42db200700 1 -- 172.21.15.42:0/982763545 wait complete. 2024-05-27T10:17:31.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T10:17:31.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:31 smithi143 ceph-mon[35545]: pgmap v486: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:31.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:31 smithi042 ceph-mon[31661]: pgmap v486: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:31.913 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:31.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:31 smithi104 ceph-mon[34812]: pgmap v486: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:33.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:33 smithi042 ceph-mon[31661]: pgmap v487: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:33.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:33 smithi104 ceph-mon[34812]: pgmap v487: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:34.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:33 smithi143 ceph-mon[35545]: pgmap v487: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:34.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.354+0000 7f6de6000700 1 -- 172.21.15.42:0/2668953149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 msgr2=0x7f6de0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.354+0000 7f6de6000700 1 --2- 172.21.15.42:0/2668953149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de0110090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b0d0 tx=0x7f6dd800b3e0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.354+0000 7f6de6000700 1 -- 172.21.15.42:0/2668953149 shutdown_connections 2024-05-27T10:17:34.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.354+0000 7f6de6000700 1 --2- 172.21.15.42:0/2668953149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de0110090 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.355+0000 7f6de6000700 1 --2- 172.21.15.42:0/2668953149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de01034b0 0x7f6de01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.355+0000 7f6de6000700 1 --2- 172.21.15.42:0/2668953149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de0107de0 0x7f6de01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.355+0000 7f6de6000700 1 -- 172.21.15.42:0/2668953149 >> 172.21.15.42:0/2668953149 conn(0x7f6de00fedb0 msgr2=0x7f6de01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:34.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.355+0000 7f6de6000700 1 -- 172.21.15.42:0/2668953149 shutdown_connections 2024-05-27T10:17:34.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.355+0000 7f6de6000700 1 -- 172.21.15.42:0/2668953149 wait complete. 2024-05-27T10:17:34.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.357+0000 7f6de6000700 1 Processor -- start 2024-05-27T10:17:34.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.357+0000 7f6de6000700 1 -- start start 2024-05-27T10:17:34.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.357+0000 7f6de6000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de01034b0 0x7f6de019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6de6000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de0107de0 0x7f6de019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6de6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6de6000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0113260 con 0x7f6de0108730 2024-05-27T10:17:34.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6de6000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01130e0 con 0x7f6de01034b0 2024-05-27T10:17:34.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6de6000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01133e0 con 0x7f6de0107de0 2024-05-27T10:17:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de019f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35550/0 (socket says 172.21.15.42:35550) 2024-05-27T10:17:34.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 -- 172.21.15.42:0/1732541617 learned_addr learned my addr 172.21.15.42:0/1732541617 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddf400700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de01034b0 0x7f6de019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de0107de0 0x7f6de019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 -- 172.21.15.42:0/1732541617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de0107de0 msgr2=0x7f6de019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de0107de0 0x7f6de019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 -- 172.21.15.42:0/1732541617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de01034b0 msgr2=0x7f6de019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de01034b0 0x7f6de019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.358+0000 7f6ddfe00700 1 -- 172.21.15.42:0/1732541617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de019fc30 con 0x7f6de0108730 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.359+0000 7f6ddfe00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de019f590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8007d10 tx=0x7f6dd8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.359+0000 7f6ddcc00700 1 -- 172.21.15.42:0/1732541617 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801f070 con 0x7f6de0108730 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.359+0000 7f6ddcc00700 1 -- 172.21.15.42:0/1732541617 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd8003e10 con 0x7f6de0108730 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.359+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de019fec0 con 0x7f6de0108730 2024-05-27T10:17:34.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.359+0000 7f6ddcc00700 1 -- 172.21.15.42:0/1732541617 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801b820 con 0x7f6de0108730 2024-05-27T10:17:34.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.359+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a7840 con 0x7f6de0108730 2024-05-27T10:17:34.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.365+0000 7f6ddcc00700 1 -- 172.21.15.42:0/1732541617 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6dd8022070 con 0x7f6de0108730 2024-05-27T10:17:34.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 Processor -- start 2024-05-27T10:17:34.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 -- start start 2024-05-27T10:17:34.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f243f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5569f236ddf0 con 0x5569f243fe20 2024-05-27T10:17:34.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5569f235d990 con 0x5569f22c0df0 2024-05-27T10:17:34.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.366+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5569f23619d0 con 0x5569f23ac9a0 2024-05-27T10:17:34.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35566/0 (socket says 172.21.15.42:35566) 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3722205710 learned_addr learned my addr 172.21.15.42:0/3722205710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6ddcc00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6dc40b2900 0x7f6dc40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6ddf400700 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f243f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6ddfe00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6dc40b2900 0x7f6dc40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6ddcc00700 1 -- 172.21.15.42:0/1732541617 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dd800f020 con 0x7f6de0108730 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3722205710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 msgr2=0x5569f243f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3722205710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 msgr2=0x5569f243f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f243f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3722205710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f23263d0 con 0x5569f243fe20 2024-05-27T10:17:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.367+0000 7f6ddf400700 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f243f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:17:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.368+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244c0c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005e10 tx=0x7f6dc800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4373e5d84e1f4439 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:34.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.368+0000 7f6ddfe00700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6dc40b2900 0x7f6dc40b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009fd0 tx=0x7f6dd802b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.368+0000 7f6dd4c00700 1 -- 172.21.15.42:0/3722205710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8026070 con 0x5569f243fe20 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.368+0000 7f6dd4c00700 1 -- 172.21.15.42:0/3722205710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc8021de0 con 0x5569f243fe20 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.368+0000 7f6dd4c00700 1 -- 172.21.15.42:0/3722205710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8012a80 con 0x5569f243fe20 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3722205710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 msgr2=0x5569f244c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244c0c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005e10 tx=0x7f6dc800c4b0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3722205710 shutdown_connections 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244c0c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3722205710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f243f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.369+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3722205710 >> 172.21.15.42:0/3722205710 conn(0x5569f235a340 msgr2=0x5569f243b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.370+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3722205710 shutdown_connections 2024-05-27T10:17:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.370+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3722205710 wait complete. 2024-05-27T10:17:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.370+0000 7f6df4a47b80 1 -- start start 2024-05-27T10:17:34.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.370+0000 7f6df4a47b80 1 Processor -- start 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f2449660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f2449ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5569f2263f50 con 0x5569f243fe20 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5569f22fe710 con 0x5569f22c0df0 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5569f244cb30 con 0x5569f23ac9a0 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6df4a47b80 10 client.0 _ll_register_callbacks cb 0x5569f2307c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35568/0 (socket says 172.21.15.42:35568) 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.371+0000 7f6dd7e00700 1 -- 172.21.15.42:0/1580332874 learned_addr learned my addr 172.21.15.42:0/1580332874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6ddf400700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f2449ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f2449660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 -- 172.21.15.42:0/1580332874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 msgr2=0x5569f2449ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f2449ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 -- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 msgr2=0x5569f2449660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f2449660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 -- 172.21.15.42:0/1580332874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f22ad880 con 0x5569f243fe20 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244a0e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005c50 tx=0x7f6dc8012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae282d4be6cdbbcd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dcd600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dcd600700 1 -- 172.21.15.42:0/1580332874 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8026070 con 0x5569f243fe20 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6dcd600700 1 -- 172.21.15.42:0/1580332874 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dc8005460 con 0x5569f243fe20 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.372+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc8021de0 con 0x5569f243fe20 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.373+0000 7f6dcd600700 1 -- 172.21.15.42:0/1580332874 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6dc8005660 con 0x5569f243fe20 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.373+0000 7f6df4a47b80 10 client.15000 fetch_fsmap learned FSMap version 25 2024-05-27T10:17:34.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.373+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569f232e010 con 0x5569f243fe20 2024-05-27T10:17:34.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.373+0000 7f6dcd600700 1 -- 172.21.15.42:0/1580332874 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc8021de0 con 0x5569f243fe20 2024-05-27T10:17:34.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.373+0000 7f6dcd600700 1 -- 172.21.15.42:0/1580332874 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6dc80128c0 con 0x5569f243fe20 2024-05-27T10:17:34.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.373+0000 7f6df4a47b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.374+0000 7f6df4a47b80 10 client.15000 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.374+0000 7f6df4a47b80 20 client.15000 populate_metadata read hostname 'smithi042' 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.374+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f2459810 0x5569f245ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.374+0000 7f6df4a47b80 4 client.15000 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.374+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f236fd40 con 0x5569f2459810 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.374+0000 7f6ddf400700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f2459810 0x5569f245ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.375+0000 7f6ddf400700 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f2459810 0x5569f245ac80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5250924f08adbd server_cookie=d440111da34a753b in_seq=0 out_seq=0 2024-05-27T10:17:34.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.375+0000 7f6dcd600700 10 client.15000 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:17:34.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6dcd600700 1 -- 172.21.15.42:0/1580332874 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5569f236fd40 con 0x5569f2459810 2024-05-27T10:17:34.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6dcd600700 10 client.15000 handle_command_reply: tid=0 2024-05-27T10:17:34.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 client.15000 shutdown 2024-05-27T10:17:34.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 msgr2=0x5569f244a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244a0e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005c50 tx=0x7f6dc8012660 comp rx=0 tx=0).stop 2024-05-27T10:17:34.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 shutdown_connections 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f2459810 0x5569f245ac80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f243fe20 0x5569f244a0e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f22c0df0 0x5569f2449660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/1580332874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f2449ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6dcd600700 0 client.15000 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 >> 172.21.15.42:0/1580332874 conn(0x5569f235a340 msgr2=0x5569f243f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 shutdown_connections 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.378+0000 7f6df4a47b80 1 -- 172.21.15.42:0/1580332874 wait complete. 2024-05-27T10:17:34.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.379+0000 7f6df4a47b80 20 client.15000 trim_cache size 0 max 16384 2024-05-27T10:17:34.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 Processor -- start 2024-05-27T10:17:34.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 -- start start 2024-05-27T10:17:34.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 0x5569f243f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f243ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5569f2458fc0 con 0x5569f22c0df0 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5569f2459690 con 0x5569f243fe20 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.402+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5569f2459d60 con 0x5569f23ac9a0 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 0x5569f243f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 0x5569f243f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35572/0 (socket says 172.21.15.42:35572) 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3985081837 learned_addr learned my addr 172.21.15.42:0/3985081837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6ddf400700 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3985081837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 msgr2=0x5569f243f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3985081837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 msgr2=0x5569f243ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f243ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6dd7e00700 1 -- 172.21.15.42:0/3985081837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f2447bd0 con 0x5569f22c0df0 2024-05-27T10:17:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.403+0000 7f6ddf400700 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:17:34.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.404+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 0x5569f243f0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c5f0 tx=0x7f6dc8005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfba3f09ba58b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.404+0000 7f6dcfe00700 1 -- 172.21.15.42:0/3985081837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8026070 con 0x5569f22c0df0 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.404+0000 7f6dcfe00700 1 -- 172.21.15.42:0/3985081837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc800c970 con 0x5569f22c0df0 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.404+0000 7f6dcfe00700 1 -- 172.21.15.42:0/3985081837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8004280 con 0x5569f22c0df0 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3985081837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 msgr2=0x5569f243f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 0x5569f243f0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c5f0 tx=0x7f6dc8005d60 comp rx=0 tx=0).stop 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3985081837 shutdown_connections 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f243ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f22c0df0 0x5569f243f0d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/3985081837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f23ac9a0 0x5569f243f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3985081837 >> 172.21.15.42:0/3985081837 conn(0x5569f235a340 msgr2=0x5569f2447f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3985081837 shutdown_connections 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.405+0000 7f6df4a47b80 1 -- 172.21.15.42:0/3985081837 wait complete. 2024-05-27T10:17:34.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.406+0000 7f6df4a47b80 1 -- start start 2024-05-27T10:17:34.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.406+0000 7f6df4a47b80 1 Processor -- start 2024-05-27T10:17:34.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 0x5569f2464400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f245d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f2464a60 0x5569f245dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5569f245ab00 con 0x5569f23ac9a0 2024-05-27T10:17:34.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5569f245a430 con 0x5569f243fe20 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5569f245b1d0 con 0x5569f2464a60 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6df4a47b80 10 client.0 _ll_register_callbacks cb 0x5569f2307c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 0x5569f2464400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 0x5569f2464400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35588/0 (socket says 172.21.15.42:35588) 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6dd7e00700 1 -- 172.21.15.42:0/2476357710 learned_addr learned my addr 172.21.15.42:0/2476357710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6ddf400700 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f245d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6dd7e00700 1 -- 172.21.15.42:0/2476357710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f2464a60 msgr2=0x5569f245dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.407+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f2464a60 0x5569f245dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.408+0000 7f6dd7e00700 1 -- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 msgr2=0x5569f245d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.408+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f245d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.408+0000 7f6dd7e00700 1 -- 172.21.15.42:0/2476357710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f245e4a0 con 0x5569f23ac9a0 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.408+0000 7f6dd7e00700 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 0x5569f2464400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8012ea0 tx=0x7f6dc80054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4310076c9670bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.408+0000 7f6dcd600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6dcd600700 1 -- 172.21.15.42:0/2476357710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8047020 con 0x5569f23ac9a0 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc800c970 con 0x5569f23ac9a0 2024-05-27T10:17:34.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6dcd600700 1 -- 172.21.15.42:0/2476357710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dc8012500 con 0x5569f23ac9a0 2024-05-27T10:17:34.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6dcd600700 1 -- 172.21.15.42:0/2476357710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6dc8026070 con 0x5569f23ac9a0 2024-05-27T10:17:34.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6df4a47b80 10 client.15006 fetch_fsmap learned FSMap version 25 2024-05-27T10:17:34.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569f24c4fa0 con 0x5569f23ac9a0 2024-05-27T10:17:34.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.409+0000 7f6dcd600700 1 -- 172.21.15.42:0/2476357710 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc800c970 con 0x5569f23ac9a0 2024-05-27T10:17:34.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6dcd600700 1 -- 172.21.15.42:0/2476357710 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6dc8004710 con 0x5569f23ac9a0 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6df4a47b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6df4a47b80 10 client.15006 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6df4a47b80 20 client.15006 populate_metadata read hostname 'smithi042' 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f24c6e60 0x5569f24c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6df4a47b80 4 client.15006 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.410+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "status"}) -- 0x5569f2369520 con 0x5569f24c6e60 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.411+0000 7f6ddf400700 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f24c6e60 0x5569f24c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.411+0000 7f6ddf400700 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f24c6e60 0x5569f24c82d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14324597904bf54 server_cookie=c5dbd349aead0b30 in_seq=0 out_seq=0 2024-05-27T10:17:34.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.411+0000 7f6dcd600700 10 client.15006 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:17:34.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6dcd600700 1 -- 172.21.15.42:0/2476357710 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5569f2369520 con 0x5569f24c6e60 2024-05-27T10:17:34.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6dcd600700 10 client.15006 handle_command_reply: tid=0 2024-05-27T10:17:34.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 client.15006 shutdown 2024-05-27T10:17:34.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 msgr2=0x5569f2464400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 0x5569f2464400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8012ea0 tx=0x7f6dc80054a0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 shutdown_connections 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5569f24c6e60 0x5569f24c82d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569f2464a60 0x5569f245dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5569f23ac9a0 0x5569f2464400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 --2- 172.21.15.42:0/2476357710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5569f243fe20 0x5569f245d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 >> 172.21.15.42:0/2476357710 conn(0x5569f235a340 msgr2=0x5569f243ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.412+0000 7f6dcd600700 0 client.15006 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.413+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 shutdown_connections 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.413+0000 7f6df4a47b80 1 -- 172.21.15.42:0/2476357710 wait complete. 2024-05-27T10:17:34.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.413+0000 7f6df4a47b80 20 client.15006 trim_cache size 0 max 16384 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34499, 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 196.226156288, 2024-05-27T10:17:34.415 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 25, 2024-05-27T10:17:34.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 81, 2024-05-27T10:17:34.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-05-27T10:17:34.416 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 219.53255592900001 2024-05-27T10:17:34.416 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:17:34.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6dc40b2900 msgr2=0x7f6dc40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6dc40b2900 0x7f6dc40b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009fd0 tx=0x7f6dd802b000 comp rx=0 tx=0).stop 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 msgr2=0x7f6de019f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de019f590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8007d10 tx=0x7f6dd8007df0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 shutdown_connections 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de0108730 0x7f6de019f590 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6dc40b2900 0x7f6dc40b4e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de01034b0 0x7f6de019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 --2- 172.21.15.42:0/1732541617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de0107de0 0x7f6de019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 >> 172.21.15.42:0/1732541617 conn(0x7f6de00fedb0 msgr2=0x7f6de006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 shutdown_connections 2024-05-27T10:17:34.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:34.414+0000 7f6de6000700 1 -- 172.21.15.42:0/1732541617 wait complete. 2024-05-27T10:17:35.462 INFO:tasks.cephadm:Setup complete, yielding 2024-05-27T10:17:35.462 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T10:17:35.472 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:17:35.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-27T10:17:35.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:35 smithi042 ceph-mon[31661]: pgmap v488: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:35.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:35 smithi104 ceph-mon[34812]: pgmap v488: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:36.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:35 smithi143 ceph-mon[35545]: pgmap v488: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:37.807 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:37 smithi042 ceph-mon[31661]: pgmap v489: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:37.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:37 smithi104 ceph-mon[34812]: pgmap v489: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:38.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:37 smithi143 ceph-mon[35545]: pgmap v489: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.490+0000 7f4001200700 1 -- 172.21.15.42:0/1071802121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc107690 msgr2=0x7f3ffc107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.490+0000 7f4001200700 1 --2- 172.21.15.42:0/1071802121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc107690 0x7f3ffc107aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009990 tx=0x7f3fe8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 -- 172.21.15.42:0/1071802121 shutdown_connections 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 --2- 172.21.15.42:0/1071802121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ffc108030 0x7f3ffc10e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 --2- 172.21.15.42:0/1071802121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ffc106560 0x7f3ffc106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 --2- 172.21.15.42:0/1071802121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc107690 0x7f3ffc107aa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 -- 172.21.15.42:0/1071802121 >> 172.21.15.42:0/1071802121 conn(0x7f3ffc101bc0 msgr2=0x7f3ffc104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 -- 172.21.15.42:0/1071802121 shutdown_connections 2024-05-27T10:17:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.491+0000 7f4001200700 1 -- 172.21.15.42:0/1071802121 wait complete. 2024-05-27T10:17:38.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.492+0000 7f4001200700 1 Processor -- start 2024-05-27T10:17:38.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.492+0000 7f4001200700 1 -- start start 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f4001200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ffc106560 0x7f3ffc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f4001200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ffc107690 0x7f3ffc19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f4001200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 0x7f3ffc1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f4001200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc111ed0 con 0x7f3ffc108030 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f4001200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc111d50 con 0x7f3ffc106560 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f4001200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc112050 con 0x7f3ffc107690 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f3ffa800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 0x7f3ffc1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f3ffa800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 0x7f3ffc1a3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35600/0 (socket says 172.21.15.42:35600) 2024-05-27T10:17:38.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f3ffa800700 1 -- 172.21.15.42:0/3079261902 learned_addr learned my addr 172.21.15.42:0/3079261902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f3ff9400700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ffc107690 0x7f3ffc19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.493+0000 7f3ff9e00700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ffc106560 0x7f3ffc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ffa800700 1 -- 172.21.15.42:0/3079261902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ffc107690 msgr2=0x7f3ffc19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ffa800700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ffc107690 0x7f3ffc19d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ffa800700 1 -- 172.21.15.42:0/3079261902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ffc106560 msgr2=0x7f3ffc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ffa800700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ffc106560 0x7f3ffc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ffa800700 1 -- 172.21.15.42:0/3079261902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc1a45d0 con 0x7f3ffc108030 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ffa800700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 0x7f3ffc1a3fe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4007ee0 tx=0x7f3ff400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.494+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4004bb0 con 0x7f3ffc108030 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.495+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff4004d50 con 0x7f3ffc108030 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.495+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc1a48c0 con 0x7f3ffc108030 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.495+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40058b0 con 0x7f3ffc108030 2024-05-27T10:17:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.495+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a4e00 con 0x7f3ffc108030 2024-05-27T10:17:38.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.496+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc0512b0 con 0x7f3ffc108030 2024-05-27T10:17:38.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.499+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3ff401a020 con 0x7f3ffc108030 2024-05-27T10:17:38.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.500+0000 7f3ff2a00700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f3fe00b2900 0x7f3fe00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:38.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.500+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3ff40e1eb0 con 0x7f3ffc108030 2024-05-27T10:17:38.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.500+0000 7f3ff9e00700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f3fe00b2900 0x7f3fe00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:38.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.501+0000 7f3ff9e00700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f3fe00b2900 0x7f3fe00b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc107590 tx=0x7f3fec00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:38.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.503+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ff4010070 con 0x7f3ffc108030 2024-05-27T10:17:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.760+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3ffc063bf0 con 0x7f3fe00b2900 2024-05-27T10:17:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.762+0000 7f3ff2a00700 1 -- 172.21.15.42:0/3079261902 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3ffc063bf0 con 0x7f3fe00b2900 2024-05-27T10:17:38.762 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-05-27T10:17:38.762 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-05-27T10:17:38.762 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-05-27T10:17:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f3fe00b2900 msgr2=0x7f3fe00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f3fe00b2900 0x7f3fe00b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc107590 tx=0x7f3fec00a3b0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 msgr2=0x7f3ffc1a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 0x7f3ffc1a3fe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4007ee0 tx=0x7f3ff400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 shutdown_connections 2024-05-27T10:17:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ffc108030 0x7f3ffc1a3fe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f3fe00b2900 0x7f3fe00b4e10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ffc106560 0x7f3ffc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 --2- 172.21.15.42:0/3079261902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ffc107690 0x7f3ffc19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 >> 172.21.15.42:0/3079261902 conn(0x7f3ffc101bc0 msgr2=0x7f3ffc10a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 shutdown_connections 2024-05-27T10:17:38.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:38.765+0000 7f4001200700 1 -- 172.21.15.42:0/3079261902 wait complete. 2024-05-27T10:17:39.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-27T10:17:39.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:39 smithi042 ceph-mon[31661]: pgmap v490: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:39.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:39 smithi104 ceph-mon[34812]: pgmap v490: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:40.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:39 smithi143 ceph-mon[35545]: pgmap v490: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:40.209 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:40 smithi042 ceph-mon[31661]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:40.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:40 smithi104 ceph-mon[34812]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:40 smithi143 ceph-mon[35545]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:41.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:41 smithi042 ceph-mon[31661]: pgmap v491: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:41.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:41 smithi104 ceph-mon[34812]: pgmap v491: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:42.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:41 smithi143 ceph-mon[35545]: pgmap v491: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.935+0000 7f759f000700 1 -- 172.21.15.42:0/4278998617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598104140 msgr2=0x7f7598104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.935+0000 7f759f000700 1 --2- 172.21.15.42:0/4278998617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598104140 0x7f7598104550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f758c009990 tx=0x7f758c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:17:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 -- 172.21.15.42:0/4278998617 shutdown_connections 2024-05-27T10:17:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 --2- 172.21.15.42:0/4278998617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7598105c10 0x7f759810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 --2- 172.21.15.42:0/4278998617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598104140 0x7f7598104550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 --2- 172.21.15.42:0/4278998617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7598105270 0x7f7598105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 -- 172.21.15.42:0/4278998617 >> 172.21.15.42:0/4278998617 conn(0x7f75980ff7a0 msgr2=0x7f7598101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 -- 172.21.15.42:0/4278998617 shutdown_connections 2024-05-27T10:17:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.936+0000 7f759f000700 1 -- 172.21.15.42:0/4278998617 wait complete. 2024-05-27T10:17:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.937+0000 7f759f000700 1 Processor -- start 2024-05-27T10:17:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 -- start start 2024-05-27T10:17:42.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7598104140 0x7f759819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:42.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 0x7f759819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7598105c10 0x7f75981a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f759810fab0 con 0x7f7598105270 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f759810f930 con 0x7f7598105c10 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.938+0000 7f759f000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f759810fc30 con 0x7f7598104140 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 0x7f759819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 0x7f759819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42612/0 (socket says 172.21.15.42:42612) 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 -- 172.21.15.42:0/3538978533 learned_addr learned my addr 172.21.15.42:0/3538978533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759dc00700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7598104140 0x7f759819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:42.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759e600700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7598105c10 0x7f75981a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 -- 172.21.15.42:0/3538978533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7598104140 msgr2=0x7f759819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7598104140 0x7f759819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 -- 172.21.15.42:0/3538978533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7598105c10 msgr2=0x7f75981a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7598105c10 0x7f75981a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.939+0000 7f759d200700 1 -- 172.21.15.42:0/3538978533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75981a21e0 con 0x7f7598105270 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.940+0000 7f759d200700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 0x7f759819abc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f758400b530 tx=0x7f758400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.940+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584015020 con 0x7f7598105270 2024-05-27T10:17:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.940+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7584011cc0 con 0x7f7598105270 2024-05-27T10:17:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.940+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75981a24d0 con 0x7f7598105270 2024-05-27T10:17:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.940+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584010dc0 con 0x7f7598105270 2024-05-27T10:17:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.940+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75981a2ae0 con 0x7f7598105270 2024-05-27T10:17:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.942+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75980512b0 con 0x7f7598105270 2024-05-27T10:17:42.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.944+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7584018020 con 0x7f7598105270 2024-05-27T10:17:42.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.946+0000 7f7596000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f75800b2900 0x7f75800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:42.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.946+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f75840e1fe0 con 0x7f7598105270 2024-05-27T10:17:42.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.946+0000 7f759dc00700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f75800b2900 0x7f75800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:42.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.947+0000 7f759dc00700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f75800b2900 0x7f75800b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098e0 tx=0x7f758c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:42.949+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f758401f020 con 0x7f7598105270 2024-05-27T10:17:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.198+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7598063bf0 con 0x7f75800b2900 2024-05-27T10:17:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.220+0000 7f7596000700 1 -- 172.21.15.42:0/3538978533 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7598063bf0 con 0x7f75800b2900 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.a smithi042 running (3m) 51s ago 3m 80.5M - 19.0.0-3868-ga6507388 850a06a84d28 2c4b962aeb59 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.b smithi104 running (3m) 85s ago 3m 93.3M - 19.0.0-3868-ga6507388 850a06a84d28 46fd50cc7f41 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.c smithi143 running (4m) 85s ago 4m 85.0M - 19.0.0-3868-ga6507388 850a06a84d28 98f783de499c 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.d smithi042 running (4m) 51s ago 4m 80.7M - 19.0.0-3868-ga6507388 850a06a84d28 868cb46713fb 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.e smithi104 running (4m) 85s ago 4m 85.7M - 19.0.0-3868-ga6507388 850a06a84d28 6315d348d2d7 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.f smithi143 running (4m) 85s ago 4m 80.6M - 19.0.0-3868-ga6507388 850a06a84d28 ec67033ec54c 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.g smithi042 running (3m) 51s ago 3m 80.5M - 19.0.0-3868-ga6507388 850a06a84d28 860b52c20f48 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.h smithi104 running (4m) 85s ago 4m 85.8M - 19.0.0-3868-ga6507388 850a06a84d28 9faec2d5ac52 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.i smithi143 running (4m) 85s ago 4m 84.9M - 19.0.0-3868-ga6507388 850a06a84d28 b1f10c310b47 2024-05-27T10:17:43.221 INFO:teuthology.orchestra.run.smithi042.stdout:mds.j smithi042 running (4m) 51s ago 4m 80.7M - 19.0.0-3868-ga6507388 850a06a84d28 9f6a3772c279 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mds.k smithi104 running (4m) 85s ago 4m 86.4M - 19.0.0-3868-ga6507388 850a06a84d28 53fa28eed621 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mds.l smithi143 running (4m) 85s ago 4m 80.8M - 19.0.0-3868-ga6507388 850a06a84d28 c29cc86ecba1 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.x smithi042 *:9283,8765 running (15m) 51s ago 15m 600M - 19.0.0-3868-ga6507388 850a06a84d28 f8155dda74fd 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.y smithi104 *:8443,8765 running (12m) 85s ago 12m 537M - 19.0.0-3868-ga6507388 850a06a84d28 59bbe45dca08 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.z smithi143 *:8443,8765 running (12m) 85s ago 12m 534M - 19.0.0-3868-ga6507388 850a06a84d28 aa8a932cc3b7 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a smithi042 running (15m) 51s ago 15m 119M 2048M 19.0.0-3868-ga6507388 850a06a84d28 c143ba0d02a3 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b smithi104 running (12m) 85s ago 12m 108M 2048M 19.0.0-3868-ga6507388 850a06a84d28 407d38b71fe4 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c smithi143 running (12m) 85s ago 12m 108M 2048M 19.0.0-3868-ga6507388 850a06a84d28 945848b5fdc3 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (11m) 51s ago 11m 113M 4273M 19.0.0-3868-ga6507388 850a06a84d28 30a2e5345c6a 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi104 running (11m) 85s ago 11m 125M 4272M 19.0.0-3868-ga6507388 850a06a84d28 9a1fb83c415d 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi143 running (10m) 85s ago 10m 114M 4272M 19.0.0-3868-ga6507388 850a06a84d28 e58a9248a36d 2024-05-27T10:17:43.222 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi042 running (10m) 51s ago 10m 111M 4273M 19.0.0-3868-ga6507388 850a06a84d28 2f951c3640de 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi104 running (9m) 85s ago 9m 122M 4272M 19.0.0-3868-ga6507388 850a06a84d28 46d9451537b0 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi143 running (8m) 85s ago 8m 122M 4272M 19.0.0-3868-ga6507388 850a06a84d28 d622eec71dcc 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 smithi042 running (8m) 51s ago 8m 124M 4273M 19.0.0-3868-ga6507388 850a06a84d28 49a4ff8a6736 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 smithi104 running (7m) 85s ago 7m 111M 4272M 19.0.0-3868-ga6507388 850a06a84d28 d0c51dcd1de3 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 smithi143 running (6m) 85s ago 6m 122M 4272M 19.0.0-3868-ga6507388 850a06a84d28 49e274f21a1c 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 smithi042 running (6m) 51s ago 6m 110M 4273M 19.0.0-3868-ga6507388 850a06a84d28 d84a9fdc6d0d 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 smithi104 running (5m) 85s ago 5m 111M 4272M 19.0.0-3868-ga6507388 850a06a84d28 94a673679b56 2024-05-27T10:17:43.223 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 smithi143 running (5m) 85s ago 5m 109M 4272M 19.0.0-3868-ga6507388 850a06a84d28 aa09ab002a64 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f75800b2900 msgr2=0x7f75800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f75800b2900 0x7f75800b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098e0 tx=0x7f758c01a040 comp rx=0 tx=0).stop 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 msgr2=0x7f759819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 0x7f759819abc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f758400b530 tx=0x7f758400b840 comp rx=0 tx=0).stop 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 shutdown_connections 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f75800b2900 0x7f75800b4e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7598105c10 0x7f75981a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7598104140 0x7f759819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.223+0000 7f759f000700 1 --2- 172.21.15.42:0/3538978533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7598105270 0x7f759819abc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.224+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 >> 172.21.15.42:0/3538978533 conn(0x7f75980ff7a0 msgr2=0x7f75981084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.224+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 shutdown_connections 2024-05-27T10:17:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:43.224+0000 7f759f000700 1 -- 172.21.15.42:0/3538978533 wait complete. 2024-05-27T10:17:43.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:43 smithi104 ceph-mon[34812]: pgmap v492: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:44.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:43 smithi143 ceph-mon[35545]: pgmap v492: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:44.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:43 smithi042 ceph-mon[31661]: pgmap v492: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:44.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-27T10:17:44.759 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:44.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:44 smithi042 ceph-mon[31661]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:44.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:44 smithi104 ceph-mon[34812]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:45.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:44 smithi143 ceph-mon[35545]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:45.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:45 smithi042 ceph-mon[31661]: pgmap v493: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:45.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:45 smithi104 ceph-mon[34812]: pgmap v493: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:46.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:45 smithi143 ceph-mon[35545]: pgmap v493: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:47.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.212+0000 7fb2d0600700 1 -- 172.21.15.42:0/3218314019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80787e0 msgr2=0x7fb2c8076cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:47.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.212+0000 7fb2d0600700 1 --2- 172.21.15.42:0/3218314019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80787e0 0x7fb2c8076cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0099c0 tx=0x7fb2bc009cd0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 -- 172.21.15.42:0/3218314019 shutdown_connections 2024-05-27T10:17:47.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 --2- 172.21.15.42:0/3218314019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 0x7fb2c8112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 --2- 172.21.15.42:0/3218314019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80787e0 0x7fb2c8076cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 --2- 172.21.15.42:0/3218314019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c80771f0 0x7fb2c8077680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 -- 172.21.15.42:0/3218314019 >> 172.21.15.42:0/3218314019 conn(0x7fb2c80ff9c0 msgr2=0x7fb2c8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:47.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 -- 172.21.15.42:0/3218314019 shutdown_connections 2024-05-27T10:17:47.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.213+0000 7fb2d0600700 1 -- 172.21.15.42:0/3218314019 wait complete. 2024-05-27T10:17:47.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.214+0000 7fb2d0600700 1 Processor -- start 2024-05-27T10:17:47.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.214+0000 7fb2d0600700 1 -- start start 2024-05-27T10:17:47.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2d0600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 0x7fb2c81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2d0600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c80787e0 0x7fb2c81a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2d0600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 0x7fb2c81a8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2d0600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8116250 con 0x7fb2c80771f0 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2d0600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81160d0 con 0x7fb2c81109b0 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2d0600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81163d0 con 0x7fb2c80787e0 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 0x7fb2c81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 0x7fb2c81a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42624/0 (socket says 172.21.15.42:42624) 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 -- 172.21.15.42:0/4213470362 learned_addr learned my addr 172.21.15.42:0/4213470362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cd200700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c80787e0 0x7fb2c81a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2ce600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 0x7fb2c81a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 -- 172.21.15.42:0/4213470362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c80787e0 msgr2=0x7fb2c81a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:47.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c80787e0 0x7fb2c81a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 -- 172.21.15.42:0/4213470362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 msgr2=0x7fb2c81a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:47.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.215+0000 7fb2cdc00700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 0x7fb2c81a8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2cdc00700 1 -- 172.21.15.42:0/4213470362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81a8980 con 0x7fb2c80771f0 2024-05-27T10:17:47.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2ce600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 0x7fb2c81a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:17:47.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2cdc00700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 0x7fb2c81a0e20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009990 tx=0x7fb2bc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:47.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc027070 con 0x7fb2c80771f0 2024-05-27T10:17:47.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2bc00bd30 con 0x7fb2c80771f0 2024-05-27T10:17:47.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81a8c10 con 0x7fb2c80771f0 2024-05-27T10:17:47.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc022d90 con 0x7fb2c80771f0 2024-05-27T10:17:47.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.216+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a9010 con 0x7fb2c80771f0 2024-05-27T10:17:47.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.218+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c80512b0 con 0x7fb2c80771f0 2024-05-27T10:17:47.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.222+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb2bc02a070 con 0x7fb2c80771f0 2024-05-27T10:17:47.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.223+0000 7fb2c6000700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b00b2880 0x7fb2b00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:47.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.224+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb2bc0e3400 con 0x7fb2c80771f0 2024-05-27T10:17:47.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.224+0000 7fb2cd200700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b00b2880 0x7fb2b00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:47.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.225+0000 7fb2cd200700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b00b2880 0x7fb2b00b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005950 tx=0x7fb2b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:47.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.226+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2bc013070 con 0x7fb2c80771f0 2024-05-27T10:17:47.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.469+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb2c8063bf0 con 0x7fb2b00b2880 2024-05-27T10:17:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.487+0000 7fb2c6000700 1 -- 172.21.15.42:0/4213470362 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb2c8063bf0 con 0x7fb2b00b2880 2024-05-27T10:17:47.487 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-27T10:17:47.487 INFO:teuthology.orchestra.run.smithi042.stdout:mds.all 12/12 89s ago 4m smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-27T10:17:47.487 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 3/3 89s ago 12m smithi042=x;smithi104=y;smithi143=z;count:3 2024-05-27T10:17:47.487 INFO:teuthology.orchestra.run.smithi042.stdout:mon 3/3 89s ago 13m smithi042:172.21.15.42=a;smithi104:172.21.15.104=b;smithi143:172.21.15.143=c;count:3 2024-05-27T10:17:47.488 INFO:teuthology.orchestra.run.smithi042.stdout:osd 12 89s ago - 2024-05-27T10:17:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b00b2880 msgr2=0x7fb2b00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b00b2880 0x7fb2b00b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005950 tx=0x7fb2b40058e0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 msgr2=0x7fb2c81a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 0x7fb2c81a0e20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009990 tx=0x7fb2bc00f740 comp rx=0 tx=0).stop 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 shutdown_connections 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fb2b00b2880 0x7fb2b00b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c81109b0 0x7fb2c81a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2c80771f0 0x7fb2c81a0e20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 --2- 172.21.15.42:0/4213470362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2c80787e0 0x7fb2c81a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.490+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 >> 172.21.15.42:0/4213470362 conn(0x7fb2c80ff9c0 msgr2=0x7fb2c8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.491+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 shutdown_connections 2024-05-27T10:17:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:47.491+0000 7fb2d0600700 1 -- 172.21.15.42:0/4213470362 wait complete. 2024-05-27T10:17:47.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:47 smithi042 ceph-mon[31661]: pgmap v494: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:47.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:47 smithi104 ceph-mon[34812]: pgmap v494: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:48.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:47 smithi143 ceph-mon[35545]: pgmap v494: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:48.563 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-27T10:17:48.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:48 smithi042 ceph-mon[31661]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:48.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:48 smithi104 ceph-mon[34812]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:49.030 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:49.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:48 smithi143 ceph-mon[35545]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:49.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:49 smithi042 ceph-mon[31661]: pgmap v495: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:49 smithi104 ceph-mon[34812]: pgmap v495: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:50.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:49 smithi143 ceph-mon[35545]: pgmap v495: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 -- 172.21.15.42:0/1911775320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 msgr2=0x7fbf1c105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 --2- 172.21.15.42:0/1911775320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c105870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009990 tx=0x7fbf08009ca0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 -- 172.21.15.42:0/1911775320 shutdown_connections 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 --2- 172.21.15.42:0/1911775320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf1c105e00 0x7fbf1c10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 --2- 172.21.15.42:0/1911775320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf1c104330 0x7fbf1c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 --2- 172.21.15.42:0/1911775320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c105870 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 -- 172.21.15.42:0/1911775320 >> 172.21.15.42:0/1911775320 conn(0x7fbf1c0ff9b0 msgr2=0x7fbf1c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:51.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.676+0000 7fbf23800700 1 -- 172.21.15.42:0/1911775320 shutdown_connections 2024-05-27T10:17:51.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.677+0000 7fbf23800700 1 -- 172.21.15.42:0/1911775320 wait complete. 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 Processor -- start 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 -- start start 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf1c104330 0x7fbf1c077400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf1c105e00 0x7fbf1c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10fc50 con 0x7fbf1c105460 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10fad0 con 0x7fbf1c105e00 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.678+0000 7fbf23800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10fdd0 con 0x7fbf1c104330 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42648/0 (socket says 172.21.15.42:42648) 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 -- 172.21.15.42:0/2789651962 learned_addr learned my addr 172.21.15.42:0/2789651962 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:51.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf20e00700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf1c104330 0x7fbf1c077400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:51.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf21800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf1c105e00 0x7fbf1c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:51.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 -- 172.21.15.42:0/2789651962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf1c104330 msgr2=0x7fbf1c077400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:51.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf1c104330 0x7fbf1c077400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 -- 172.21.15.42:0/2789651962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf1c105e00 msgr2=0x7fbf1c076040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:51.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf1c105e00 0x7fbf1c076040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.679+0000 7fbf1be00700 1 -- 172.21.15.42:0/2789651962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c076750 con 0x7fbf1c105460 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.680+0000 7fbf1be00700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c075b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf080098e0 tx=0x7fbf080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.680+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08004830 con 0x7fbf1c105460 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.680+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf08005020 con 0x7fbf1c105460 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.680+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c1a4920 con 0x7fbf1c105460 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.680+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08003d30 con 0x7fbf1c105460 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.680+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c1a4cc0 con 0x7fbf1c105460 2024-05-27T10:17:51.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.682+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c0512b0 con 0x7fbf1c105460 2024-05-27T10:17:51.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.684+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbf080054d0 con 0x7fbf1c105460 2024-05-27T10:17:51.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.687+0000 7fbf19600700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbf040b2900 0x7fbf040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:51.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.687+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbf080e3560 con 0x7fbf1c105460 2024-05-27T10:17:51.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.687+0000 7fbf20e00700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbf040b2900 0x7fbf040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.688+0000 7fbf20e00700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbf040b2900 0x7fbf040b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c105360 tx=0x7fbf1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.690+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf08013070 con 0x7fbf1c105460 2024-05-27T10:17:51.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:51 smithi042 ceph-mon[31661]: pgmap v496: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.938+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbf1c063bf0 con 0x7fbf040b2900 2024-05-27T10:17:51.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.941+0000 7fbf19600700 1 -- 172.21.15.42:0/2789651962 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fbf1c063bf0 con 0x7fbf040b2900 2024-05-27T10:17:51.941 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-05-27T10:17:51.941 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-05-27T10:17:51.941 INFO:teuthology.orchestra.run.smithi042.stdout:smithi104 172.21.15.104 2024-05-27T10:17:51.941 INFO:teuthology.orchestra.run.smithi042.stdout:smithi143 172.21.15.143 2024-05-27T10:17:51.941 INFO:teuthology.orchestra.run.smithi042.stdout:3 hosts in cluster 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.944+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbf040b2900 msgr2=0x7fbf040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbf040b2900 0x7fbf040b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c105360 tx=0x7fbf1000b410 comp rx=0 tx=0).stop 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 msgr2=0x7fbf1c075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c075b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf080098e0 tx=0x7fbf080047b0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 shutdown_connections 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fbf040b2900 0x7fbf040b4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf1c105e00 0x7fbf1c076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf1c104330 0x7fbf1c077400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 --2- 172.21.15.42:0/2789651962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c105460 0x7fbf1c075b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 >> 172.21.15.42:0/2789651962 conn(0x7fbf1c0ff9b0 msgr2=0x7fbf1c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 shutdown_connections 2024-05-27T10:17:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:51.945+0000 7fbf23800700 1 -- 172.21.15.42:0/2789651962 wait complete. 2024-05-27T10:17:51.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:51 smithi104 ceph-mon[34812]: pgmap v496: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:52.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:51 smithi143 ceph-mon[35545]: pgmap v496: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:17:53.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-27T10:17:53.582 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:53.920 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:53 smithi042 ceph-mon[31661]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:53.921 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:53 smithi042 ceph-mon[31661]: pgmap v497: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:53.921 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:53 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:17:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:53 smithi104 ceph-mon[34812]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:53 smithi104 ceph-mon[34812]: pgmap v497: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:53.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:53 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:17:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:53 smithi143 ceph-mon[35545]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:53 smithi143 ceph-mon[35545]: pgmap v497: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:17:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:53 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:17:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:17:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:17:54.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:54 smithi104 ceph-mon[34812]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:17:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:17:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:17:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:54 smithi143 ceph-mon[35545]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:17:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:17:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:17:55.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:54 smithi042 ceph-mon[31661]: from='mgr.14152 172.21.15.42:0/4134055836' entity='mgr.x' 2024-05-27T10:17:55.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:55 smithi042 ceph-mon[31661]: pgmap v498: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:55.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:55 smithi104 ceph-mon[34812]: pgmap v498: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:56.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:55 smithi143 ceph-mon[35545]: pgmap v498: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.059+0000 7f6d8c400700 1 -- 172.21.15.42:0/1164434265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84106560 msgr2=0x7f6d84106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.059+0000 7f6d8c400700 1 --2- 172.21.15.42:0/1164434265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84106560 0x7f6d84106970 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009990 tx=0x7f6d78009ca0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 -- 172.21.15.42:0/1164434265 shutdown_connections 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 --2- 172.21.15.42:0/1164434265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d84108030 0x7f6d8410e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 --2- 172.21.15.42:0/1164434265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84106560 0x7f6d84106970 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 --2- 172.21.15.42:0/1164434265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107690 0x7f6d84107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 -- 172.21.15.42:0/1164434265 >> 172.21.15.42:0/1164434265 conn(0x7f6d84101bc0 msgr2=0x7f6d84104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 -- 172.21.15.42:0/1164434265 shutdown_connections 2024-05-27T10:17:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.060+0000 7f6d8c400700 1 -- 172.21.15.42:0/1164434265 wait complete. 2024-05-27T10:17:56.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.061+0000 7f6d8c400700 1 Processor -- start 2024-05-27T10:17:56.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.061+0000 7f6d8c400700 1 -- start start 2024-05-27T10:17:56.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8c400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d84106560 0x7f6d8419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:56.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8c400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 0x7f6d8419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:56.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8c400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84108030 0x7f6d841a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:56.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8c400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84111e60 con 0x7f6d84107690 2024-05-27T10:17:56.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8c400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84111ce0 con 0x7f6d84108030 2024-05-27T10:17:56.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8c400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84111fe0 con 0x7f6d84106560 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d89000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 0x7f6d8419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d89000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 0x7f6d8419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52482/0 (socket says 172.21.15.42:52482) 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d89000700 1 -- 172.21.15.42:0/2629929566 learned_addr learned my addr 172.21.15.42:0/2629929566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d89a00700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d84106560 0x7f6d8419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.062+0000 7f6d8a400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84108030 0x7f6d841a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d89000700 1 -- 172.21.15.42:0/2629929566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d84106560 msgr2=0x7f6d8419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d89000700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d84106560 0x7f6d8419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d89000700 1 -- 172.21.15.42:0/2629929566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84108030 msgr2=0x7f6d841a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d89000700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84108030 0x7f6d841a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d89000700 1 -- 172.21.15.42:0/2629929566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841a44a0 con 0x7f6d84107690 2024-05-27T10:17:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d89000700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 0x7f6d8419cf10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c9b0 tx=0x7f6d7000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.063+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70007b00 con 0x7f6d84107690 2024-05-27T10:17:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.064+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d70007ca0 con 0x7f6d84107690 2024-05-27T10:17:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.064+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841a4790 con 0x7f6d84107690 2024-05-27T10:17:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.064+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70018660 con 0x7f6d84107690 2024-05-27T10:17:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.064+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841a4d20 con 0x7f6d84107690 2024-05-27T10:17:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.065+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d840512b0 con 0x7f6d84107690 2024-05-27T10:17:56.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.069+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6d7000f450 con 0x7f6d84107690 2024-05-27T10:17:56.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.071+0000 7f6d82000700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6d6c0b2900 0x7f6d6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:17:56.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.071+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6d700e2410 con 0x7f6d84107690 2024-05-27T10:17:56.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.071+0000 7f6d89a00700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6d6c0b2900 0x7f6d6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:17:56.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.072+0000 7f6d89a00700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6d6c0b2900 0x7f6d6c0b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d78006010 tx=0x7f6d7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:17:56.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.074+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d70014030 con 0x7f6d84107690 2024-05-27T10:17:56.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.320+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 --> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6d84063bf0 con 0x7f6d6c0b2900 2024-05-27T10:17:56.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.324+0000 7f6d82000700 1 -- 172.21.15.42:0/2629929566 <== mgr.14152 v2:172.21.15.42:6800/2062230330 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6d84063bf0 con 0x7f6d6c0b2900 2024-05-27T10:17:56.325 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-27T10:17:56.325 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 6m ago Has a FileSystem, LVM detected 2024-05-27T10:17:56.325 INFO:teuthology.orchestra.run.smithi042.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-27T10:17:56.325 INFO:teuthology.orchestra.run.smithi042.stdout:smithi143 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330073400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.327+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6d6c0b2900 msgr2=0x7f6d6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.327+0000 7f6d8c400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6d6c0b2900 0x7f6d6c0b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d78006010 tx=0x7f6d7801a040 comp rx=0 tx=0).stop 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.327+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 msgr2=0x7f6d8419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.327+0000 7f6d8c400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 0x7f6d8419cf10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c9b0 tx=0x7f6d7000ccc0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 shutdown_connections 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f6d6c0b2900 0x7f6d6c0b4e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84108030 0x7f6d841a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d84106560 0x7f6d8419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 --2- 172.21.15.42:0/2629929566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d84107690 0x7f6d8419cf10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 >> 172.21.15.42:0/2629929566 conn(0x7f6d84101bc0 msgr2=0x7f6d8410a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 shutdown_connections 2024-05-27T10:17:56.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:17:56.328+0000 7f6d8c400700 1 -- 172.21.15.42:0/2629929566 wait complete. 2024-05-27T10:17:57.393 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T10:17:57.401 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:17:57.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T10:17:57.880 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:17:58.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:57 smithi143 ceph-mon[35545]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:58.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:57 smithi143 ceph-mon[35545]: pgmap v499: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:58.297 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:57 smithi104 ceph-mon[34812]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:58.297 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:57 smithi104 ceph-mon[34812]: pgmap v499: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:58.297 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:57 smithi042 ceph-mon[31661]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:17:58.297 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:57 smithi042 ceph-mon[31661]: pgmap v499: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:17:59.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:17:59 smithi104 ceph-mon[34812]: pgmap v500: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:17:59.970 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:17:59 smithi042 ceph-mon[31661]: pgmap v500: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:00.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:17:59 smithi143 ceph-mon[35545]: pgmap v500: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:00.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.371+0000 7fe43f400700 1 -- 172.21.15.42:0/2354675427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 msgr2=0x7fe43810c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:00.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.371+0000 7fe43f400700 1 --2- 172.21.15.42:0/2354675427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe43810c6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe43000b0a0 tx=0x7fe43000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 -- 172.21.15.42:0/2354675427 shutdown_connections 2024-05-27T10:18:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 --2- 172.21.15.42:0/2354675427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe43810c6b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 --2- 172.21.15.42:0/2354675427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe438104330 0x7fe438104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 --2- 172.21.15.42:0/2354675427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 0x7fe438105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 -- 172.21.15.42:0/2354675427 >> 172.21.15.42:0/2354675427 conn(0x7fe4380ff9b0 msgr2=0x7fe438101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 -- 172.21.15.42:0/2354675427 shutdown_connections 2024-05-27T10:18:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.372+0000 7fe43f400700 1 -- 172.21.15.42:0/2354675427 wait complete. 2024-05-27T10:18:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.373+0000 7fe43f400700 1 Processor -- start 2024-05-27T10:18:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.373+0000 7fe43f400700 1 -- start start 2024-05-27T10:18:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.373+0000 7fe43f400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe438104330 0x7fe438077450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43f400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 0x7fe438075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe438076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43f400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe43810fc90 con 0x7fe438105e00 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43f400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe43810fb10 con 0x7fe438104330 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43f400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe43810fe10 con 0x7fe438105460 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe438076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe438076040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52492/0 (socket says 172.21.15.42:52492) 2024-05-27T10:18:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43d400700 1 -- 172.21.15.42:0/2993216808 learned_addr learned my addr 172.21.15.42:0/2993216808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe437e00700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 0x7fe438075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.374+0000 7fe43ca00700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe438104330 0x7fe438077450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43d400700 1 -- 172.21.15.42:0/2993216808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 msgr2=0x7fe438075b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43d400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 0x7fe438075b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43d400700 1 -- 172.21.15.42:0/2993216808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe438104330 msgr2=0x7fe438077450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43d400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe438104330 0x7fe438077450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43d400700 1 -- 172.21.15.42:0/2993216808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380767a0 con 0x7fe438105e00 2024-05-27T10:18:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe437e00700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 0x7fe438075b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43d400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe438076040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe43000b830 tx=0x7fe43000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430021070 con 0x7fe438105e00 2024-05-27T10:18:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe430007c80 con 0x7fe438105e00 2024-05-27T10:18:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43001b7b0 con 0x7fe438105e00 2024-05-27T10:18:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4381a48d0 con 0x7fe438105e00 2024-05-27T10:18:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.375+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4381a4c70 con 0x7fe438105e00 2024-05-27T10:18:00.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.377+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4380512b0 con 0x7fe438105e00 2024-05-27T10:18:00.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.379+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe430024070 con 0x7fe438105e00 2024-05-27T10:18:00.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.381+0000 7fe435600700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe4200b2900 0x7fe4200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:00.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.381+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe43000f020 con 0x7fe438105e00 2024-05-27T10:18:00.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.381+0000 7fe43ca00700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe4200b2900 0x7fe4200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:00.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.382+0000 7fe43ca00700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe4200b2900 0x7fe4200b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005950 tx=0x7fe42c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:00.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.383+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe43002b020 con 0x7fe438105e00 2024-05-27T10:18:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.670+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe43802e5e0 con 0x7fe438105e00 2024-05-27T10:18:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.671+0000 7fe435600700 1 -- 172.21.15.42:0/2993216808 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3054 (secure 0 0 0) 0x7fe4300ab7c0 con 0x7fe438105e00 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:e25 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 25 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-27T10:18:00.673 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-05-27T10:14:18.127208+0000 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-05-27T10:14:45.975282+0000 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 300 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T10:18:00.674 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 3 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1,2 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34499,1=34466,2=34442} 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 34499 members: 34466,34499,34442 2024-05-27T10:18:00.675 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b{0:34499} state up:active seq 7 addr [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.k{0:34421} state up:standby-replay seq 1 addr [v2:172.21.15.104:6832/3150585498,v1:172.21.15.104:6833/3150585498] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.h{1:34466} state up:active seq 17 addr [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.i{1:24496} state up:standby-replay seq 1 addr [v2:172.21.15.143:6838/517039305,v1:172.21.15.143:6839/517039305] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e{2:34442} state up:active seq 20 addr [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c{2:24475} state up:standby-replay seq 1 addr [v2:172.21.15.143:6836/4166809554,v1:172.21.15.143:6837/4166809554] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.42:6834/781814611,v1:172.21.15.42:6835/781814611] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.42:6836/1839056289,v1:172.21.15.42:6837/1839056289] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.42:6838/4181502432,v1:172.21.15.42:6839/4181502432] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a{-1:14604} state up:standby seq 1 addr [v2:172.21.15.42:6840/2889621451,v1:172.21.15.42:6841/2889621451] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.677 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.143:6832/3395724894,v1:172.21.15.143:6833/3395724894] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.677 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f{-1:24469} state up:standby seq 1 addr [v2:172.21.15.143:6834/3892720784,v1:172.21.15.143:6835/3892720784] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T10:18:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.674+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe4200b2900 msgr2=0x7fe4200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.674+0000 7fe43f400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe4200b2900 0x7fe4200b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005950 tx=0x7fe42c0058e0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.674+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 msgr2=0x7fe438076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.674+0000 7fe43f400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe438076040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe43000b830 tx=0x7fe43000bd70 comp rx=0 tx=0).stop 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 shutdown_connections 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438105e00 0x7fe438076040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fe4200b2900 0x7fe4200b4e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe438104330 0x7fe438077450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 --2- 172.21.15.42:0/2993216808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe438105460 0x7fe438075b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 >> 172.21.15.42:0/2993216808 conn(0x7fe4380ff9b0 msgr2=0x7fe438108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 shutdown_connections 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:00.675+0000 7fe43f400700 1 -- 172.21.15.42:0/2993216808 wait complete. 2024-05-27T10:18:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:18:00.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:00 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2993216808' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T10:18:01.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:00 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2993216808' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T10:18:01.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:00 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2993216808' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T10:18:01.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-27T10:18:01.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:01 smithi042 ceph-mon[31661]: pgmap v501: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:02.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:01 smithi143 ceph-mon[35545]: pgmap v501: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:02.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:01 smithi104 ceph-mon[34812]: pgmap v501: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:04.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:03 smithi042 ceph-mon[31661]: pgmap v502: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:04.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:03 smithi143 ceph-mon[35545]: pgmap v502: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:04.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:03 smithi104 ceph-mon[34812]: pgmap v502: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.667+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2217074326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 msgr2=0x7f1bd4106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.667+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2217074326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd4106790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009990 tx=0x7f1bc4009ca0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2217074326 shutdown_connections 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2217074326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd4107e50 0x7f1bd410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2217074326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd4106790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2217074326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd41074b0 0x7f1bd41078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2217074326 >> 172.21.15.42:0/2217074326 conn(0x7f1bd4101a00 msgr2=0x7f1bd4103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2217074326 shutdown_connections 2024-05-27T10:18:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.668+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2217074326 wait complete. 2024-05-27T10:18:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.669+0000 7f1bd3e00700 1 Processor -- start 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.669+0000 7f1bd3e00700 1 -- start start 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd41074b0 0x7f1bd419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4107e50 0x7f1bd41a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4111c80 con 0x7f1bd4106380 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4111b00 con 0x7f1bd4107e50 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4111e00 con 0x7f1bd41074b0 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37860/0 (socket says 172.21.15.42:37860) 2024-05-27T10:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 -- 172.21.15.42:0/2444542656 learned_addr learned my addr 172.21.15.42:0/2444542656 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bca000700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd41074b0 0x7f1bd419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3400700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4107e50 0x7f1bd41a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 -- 172.21.15.42:0/2444542656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd41074b0 msgr2=0x7f1bd419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd41074b0 0x7f1bd419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 -- 172.21.15.42:0/2444542656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4107e50 msgr2=0x7f1bd41a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4107e50 0x7f1bd41a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd2a00700 1 -- 172.21.15.42:0/2444542656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd41a43b0 con 0x7f1bd4106380 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bca000700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd41074b0 0x7f1bd419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.670+0000 7f1bd3400700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4107e50 0x7f1bd41a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.671+0000 7f1bd2a00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd419c850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4006010 tx=0x7f1bc40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.671+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003c10 con 0x7f1bd4106380 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.671+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bc4004830 con 0x7f1bd4106380 2024-05-27T10:18:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.671+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd41a4640 con 0x7f1bd4106380 2024-05-27T10:18:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.671+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4005370 con 0x7f1bd4106380 2024-05-27T10:18:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.671+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd41a4a90 con 0x7f1bd4106380 2024-05-27T10:18:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.673+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd40512b0 con 0x7f1bd4106380 2024-05-27T10:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.675+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1bc40049e0 con 0x7f1bd4106380 2024-05-27T10:18:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.677+0000 7f1bcbe00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1bb80b2900 0x7f1bb80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.677+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1bc40e3c70 con 0x7f1bd4106380 2024-05-27T10:18:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.677+0000 7f1bca000700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1bb80b2900 0x7f1bb80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.678+0000 7f1bca000700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1bb80b2900 0x7f1bb80b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc005fd0 tx=0x7f1bbc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.679+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bc4030070 con 0x7f1bd4106380 2024-05-27T10:18:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.921+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f1bd402e5e0 con 0x7f1bd4106380 2024-05-27T10:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.923+0000 7f1bcbe00700 1 -- 172.21.15.42:0/2444542656 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5262 (secure 0 0 0) 0x7f1bc4013070 con 0x7f1bd4106380 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 81 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-05-27T10:02:27.892323+0000 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-05-27T10:14:41.618756+0000 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 26 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-05-27T10:18:04.925 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client luminous 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 12 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6802/3193665133,v1:172.21.15.42:6803/3193665133] [v2:172.21.15.42:6804/3193665133,v1:172.21.15.42:6805/3193665133] exists,up 7f20f91e-7aa0-41e1-bff3-c8a8c7e24321 2024-05-27T10:18:04.926 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/2575126270,v1:172.21.15.104:6801/2575126270] [v2:172.21.15.104:6802/2575126270,v1:172.21.15.104:6803/2575126270] exists,up 7d3ff39d-4276-43ec-8f9f-41a4b970afaf 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 18 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6800/793524838,v1:172.21.15.143:6801/793524838] [v2:172.21.15.143:6802/793524838,v1:172.21.15.143:6803/793524838] exists,up 72adfe76-c970-4f68-baea-1d04215301bf 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 26 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6810/2776846288,v1:172.21.15.42:6811/2776846288] [v2:172.21.15.42:6812/2776846288,v1:172.21.15.42:6813/2776846288] exists,up 11c122a0-b307-4f56-8109-52475745da14 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 32 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/3588429935,v1:172.21.15.104:6809/3588429935] [v2:172.21.15.104:6810/3588429935,v1:172.21.15.104:6811/3588429935] exists,up 78330de0-74fb-49b7-8de6-d694077e8ea8 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 37 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6808/658500194,v1:172.21.15.143:6809/658500194] [v2:172.21.15.143:6810/658500194,v1:172.21.15.143:6811/658500194] exists,up 6aba47e2-0c53-46c1-bd90-d7cf86d14f3a 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 43 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6818/2880152722,v1:172.21.15.42:6819/2880152722] [v2:172.21.15.42:6820/2880152722,v1:172.21.15.42:6821/2880152722] exists,up 1cc50666-edea-46b9-a6c0-0d8c27799fe6 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 48 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/2687473765,v1:172.21.15.104:6817/2687473765] [v2:172.21.15.104:6818/2687473765,v1:172.21.15.104:6819/2687473765] exists,up a8c58d76-682d-48b2-a195-ab8ec1b47d3f 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 54 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6816/3963187375,v1:172.21.15.143:6817/3963187375] [v2:172.21.15.143:6818/3963187375,v1:172.21.15.143:6819/3963187375] exists,up e59bbe26-23f9-4728-957e-7ca5e03e6bd5 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 60 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6826/2335274703,v1:172.21.15.42:6827/2335274703] [v2:172.21.15.42:6828/2335274703,v1:172.21.15.42:6829/2335274703] exists,up 951d4d17-d8c3-495b-8295-667f0bfaaef3 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 65 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/1167441183,v1:172.21.15.104:6825/1167441183] [v2:172.21.15.104:6826/1167441183,v1:172.21.15.104:6827/1167441183] exists,up d114e0d1-848d-453c-a61d-65262d9c404e 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 71 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6824/1326686581,v1:172.21.15.143:6825/1326686581] [v2:172.21.15.143:6826/1326686581,v1:172.21.15.143:6827/1326686581] exists,up a925b882-f488-4723-a316-04230d44cfb7 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.6 [8,11] 2024-05-27T10:18:04.927 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.a [2,11] 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.b [2,4] 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.10 [5,9] 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.27 [3,9] 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.3a [4,8] 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.3c [4,9] 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/4102007980 expires 2024-05-28T10:03:40.179244+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2381100937 expires 2024-05-28T10:03:40.179244+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/3098170734 expires 2024-05-28T10:03:40.179244+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1584316049 expires 2024-05-28T10:03:40.179244+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/3950108791 expires 2024-05-28T10:03:03.499453+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/3950108791 expires 2024-05-28T10:03:03.499453+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/274704256 expires 2024-05-28T10:03:03.499453+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/4159042453 expires 2024-05-28T10:03:03.499453+0000 2024-05-27T10:18:04.928 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2156747670 expires 2024-05-28T10:03:03.499453+0000 2024-05-27T10:18:04.929 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/3098170734 expires 2024-05-28T10:03:40.179244+0000 2024-05-27T10:18:04.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1bb80b2900 msgr2=0x7f1bb80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:04.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1bb80b2900 0x7f1bb80b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc005fd0 tx=0x7f1bbc005f00 comp rx=0 tx=0).stop 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 msgr2=0x7f1bd419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd419c850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4006010 tx=0x7f1bc40047b0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 shutdown_connections 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f1bb80b2900 0x7f1bb80b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4107e50 0x7f1bd41a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd4106380 0x7f1bd419c850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 --2- 172.21.15.42:0/2444542656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bd41074b0 0x7f1bd419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 >> 172.21.15.42:0/2444542656 conn(0x7f1bd4101a00 msgr2=0x7f1bd410a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.928+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 shutdown_connections 2024-05-27T10:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:04.929+0000 7f1bd3e00700 1 -- 172.21.15.42:0/2444542656 wait complete. 2024-05-27T10:18:06.001 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:05 smithi104 ceph-mon[34812]: pgmap v503: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:06.001 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:05 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2444542656' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T10:18:06.016 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:05 smithi042 ceph-mon[31661]: pgmap v503: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:06.016 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:05 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2444542656' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T10:18:06.020 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-27T10:18:06.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:18:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:05 smithi143 ceph-mon[35545]: pgmap v503: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:05 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2444542656' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T10:18:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:08.031 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:07 smithi042 ceph-mon[31661]: pgmap v504: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:08.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:07 smithi143 ceph-mon[35545]: pgmap v504: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:08.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:07 smithi104 ceph-mon[34812]: pgmap v504: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 -- 172.21.15.42:0/2934015265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82080764a0 msgr2=0x7f8208114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 --2- 172.21.15.42:0/2934015265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82080764a0 0x7f8208114c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009990 tx=0x7f81f4009ca0 comp rx=0 tx=0).stop 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 -- 172.21.15.42:0/2934015265 shutdown_connections 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 --2- 172.21.15.42:0/2934015265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8208115170 0x7f8208117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 --2- 172.21.15.42:0/2934015265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8208075b50 0x7f8208075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 --2- 172.21.15.42:0/2934015265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82080764a0 0x7f8208114c30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 -- 172.21.15.42:0/2934015265 >> 172.21.15.42:0/2934015265 conn(0x7f8208079d00 msgr2=0x7f820807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 -- 172.21.15.42:0/2934015265 shutdown_connections 2024-05-27T10:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.972+0000 7f820ec00700 1 -- 172.21.15.42:0/2934015265 wait complete. 2024-05-27T10:18:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 Processor -- start 2024-05-27T10:18:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 -- start start 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 0x7f82081a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82080764a0 0x7f82081a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8208115170 0x7f82081a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f820811a780 con 0x7f8208075b50 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f820811a600 con 0x7f82080764a0 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.974+0000 7f820ec00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f820811a900 con 0x7f8208115170 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 0x7f82081a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 0x7f82081a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37876/0 (socket says 172.21.15.42:37876) 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 -- 172.21.15.42:0/4007154541 learned_addr learned my addr 172.21.15.42:0/4007154541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820ce00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82080764a0 0x7f82081a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820e200700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8208115170 0x7f82081a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:08.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 -- 172.21.15.42:0/4007154541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8208115170 msgr2=0x7f82081a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:08.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8208115170 0x7f82081a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:08.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 -- 172.21.15.42:0/4007154541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82080764a0 msgr2=0x7f82081a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:08.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82080764a0 0x7f82081a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820d800700 1 -- 172.21.15.42:0/4007154541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82081a6050 con 0x7f8208075b50 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.975+0000 7f820ce00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82080764a0 0x7f82081a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.976+0000 7f820d800700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 0x7f82081a1230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81fc007e20 tx=0x7f81fc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.976+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc016020 con 0x7f8208075b50 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.976+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81fc00ad40 con 0x7f8208075b50 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.976+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82081a6340 con 0x7f8208075b50 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.976+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0115e0 con 0x7f8208075b50 2024-05-27T10:18:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.976+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82081add10 con 0x7f8208075b50 2024-05-27T10:18:08.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.977+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8208077290 con 0x7f8208075b50 2024-05-27T10:18:08.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.980+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f81fc019020 con 0x7f8208075b50 2024-05-27T10:18:08.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.981+0000 7f8206000700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f81f00b2900 0x7f81f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:08.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.982+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f81fc0e23a0 con 0x7f8208075b50 2024-05-27T10:18:08.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.982+0000 7f820ce00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f81f00b2900 0x7f81f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:08.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.983+0000 7f820ce00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f81f00b2900 0x7f81f00b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81f4003760 tx=0x7f81f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:08.985+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81fc020070 con 0x7f8208075b50 2024-05-27T10:18:09.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.276+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82081a2590 con 0x7f8208075b50 2024-05-27T10:18:09.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.277+0000 7f8206000700 1 -- 172.21.15.42:0/4007154541 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f81fc0aa090 con 0x7f8208075b50 2024-05-27T10:18:09.282 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:09.282 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:14:45.975282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f81f00b2900 msgr2=0x7f81f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f81f00b2900 0x7f81f00b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81f4003760 tx=0x7f81f4005fd0 comp rx=0 tx=0).stop 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 msgr2=0x7f82081a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 0x7f82081a1230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81fc007e20 tx=0x7f81fc00a340 comp rx=0 tx=0).stop 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 shutdown_connections 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8208115170 0x7f82081a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8208075b50 0x7f82081a1230 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f81f00b2900 0x7f81f00b4e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 --2- 172.21.15.42:0/4007154541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82080764a0 0x7f82081a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.281+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 >> 172.21.15.42:0/4007154541 conn(0x7f8208079d00 msgr2=0x7f8208101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.282+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 shutdown_connections 2024-05-27T10:18:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:09.282+0000 7f820ec00700 1 -- 172.21.15.42:0/4007154541 wait complete. 2024-05-27T10:18:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:18:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:09 smithi143 ceph-mon[35545]: pgmap v505: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:10.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:09 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4007154541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:10.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:09 smithi042 ceph-mon[31661]: pgmap v505: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:10.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:09 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4007154541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:09 smithi104 ceph-mon[34812]: pgmap v505: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:10.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:09 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4007154541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:10.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:18:10.753 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:12.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:11 smithi143 ceph-mon[35545]: pgmap v506: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:12.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:11 smithi042 ceph-mon[31661]: pgmap v506: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:11 smithi104 ceph-mon[34812]: pgmap v506: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 -- 172.21.15.42:0/1793731747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 msgr2=0x7f8f981111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/1793731747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981111a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000b0d0 tx=0x7f8f9000b3e0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 -- 172.21.15.42:0/1793731747 shutdown_connections 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/1793731747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981111a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/1793731747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f981045c0 0x7f8f981049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/1793731747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f98108ef0 0x7f8f98109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 -- 172.21.15.42:0/1793731747 >> 172.21.15.42:0/1793731747 conn(0x7f8f980ffe90 msgr2=0x7f8f981022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.419+0000 7f8f9ce00700 1 -- 172.21.15.42:0/1793731747 shutdown_connections 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.420+0000 7f8f9ce00700 1 -- 172.21.15.42:0/1793731747 wait complete. 2024-05-27T10:18:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.421+0000 7f8f9ce00700 1 Processor -- start 2024-05-27T10:18:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.421+0000 7f8f9ce00700 1 -- start start 2024-05-27T10:18:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.422+0000 7f8f9ce00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f981045c0 0x7f8f9819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.422+0000 7f8f9ce00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f98108ef0 0x7f8f9819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.422+0000 7f8f9ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.422+0000 7f8f9ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98114370 con 0x7f8f98109840 2024-05-27T10:18:13.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.422+0000 7f8f9ce00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981141f0 con 0x7f8f981045c0 2024-05-27T10:18:13.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.422+0000 7f8f9ce00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981144f0 con 0x7f8f98108ef0 2024-05-27T10:18:13.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:13.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43686/0 (socket says 172.21.15.42:43686) 2024-05-27T10:18:13.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 -- 172.21.15.42:0/949530478 learned_addr learned my addr 172.21.15.42:0/949530478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:13.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f95400700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f98108ef0 0x7f8f9819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:13.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f95e00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f981045c0 0x7f8f9819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 -- 172.21.15.42:0/949530478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f98108ef0 msgr2=0x7f8f9819d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f98108ef0 0x7f8f9819d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 -- 172.21.15.42:0/949530478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f981045c0 msgr2=0x7f8f9819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f981045c0 0x7f8f9819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.423+0000 7f8f96800700 1 -- 172.21.15.42:0/949530478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f981a1e10 con 0x7f8f98109840 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.424+0000 7f8f96800700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981a1770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f90007d10 tx=0x7f8f90007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.424+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9001f070 con 0x7f8f98109840 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.424+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f90003e10 con 0x7f8f98109840 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.424+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9001b720 con 0x7f8f98109840 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.424+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f981a20a0 con 0x7f8f98109840 2024-05-27T10:18:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.424+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f981a9a20 con 0x7f8f98109840 2024-05-27T10:18:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.426+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f980512b0 con 0x7f8f98109840 2024-05-27T10:18:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.428+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f90022070 con 0x7f8f98109840 2024-05-27T10:18:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.431+0000 7f8f8ea00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8f7c0b2900 0x7f8f7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.431+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8f9000f020 con 0x7f8f98109840 2024-05-27T10:18:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.431+0000 7f8f95e00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8f7c0b2900 0x7f8f7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.432+0000 7f8f95e00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8f7c0b2900 0x7f8f7c0b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f88007be0 tx=0x7f8f880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:13.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.433+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f90029020 con 0x7f8f98109840 2024-05-27T10:18:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.721+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f981a9bf0 con 0x7f8f98109840 2024-05-27T10:18:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.722+0000 7f8f8ea00700 1 -- 172.21.15.42:0/949530478 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f8f900ab740 con 0x7f8f98109840 2024-05-27T10:18:13.723 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:13.723 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:14:45.975282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:18:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8f7c0b2900 msgr2=0x7f8f7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8f7c0b2900 0x7f8f7c0b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f88007be0 tx=0x7f8f880073d0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 msgr2=0x7f8f981a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981a1770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f90007d10 tx=0x7f8f90007d40 comp rx=0 tx=0).stop 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 shutdown_connections 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f98109840 0x7f8f981a1770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f8f7c0b2900 0x7f8f7c0b4e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f981045c0 0x7f8f9819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 --2- 172.21.15.42:0/949530478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f98108ef0 0x7f8f9819d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 >> 172.21.15.42:0/949530478 conn(0x7f8f980ffe90 msgr2=0x7f8f9810a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.725+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 shutdown_connections 2024-05-27T10:18:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:13.726+0000 7f8f9ce00700 1 -- 172.21.15.42:0/949530478 wait complete. 2024-05-27T10:18:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-05-27T10:18:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:13 smithi143 ceph-mon[35545]: pgmap v507: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:14.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:13 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/949530478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:14.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:13 smithi042 ceph-mon[31661]: pgmap v507: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:14.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:13 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/949530478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:14.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:13 smithi104 ceph-mon[34812]: pgmap v507: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:14.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:13 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/949530478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:14.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:18:15.205 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:15.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:15 smithi042 ceph-mon[31661]: pgmap v508: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:16.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:15 smithi143 ceph-mon[35545]: pgmap v508: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:16.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:15 smithi104 ceph-mon[34812]: pgmap v508: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:17.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.863+0000 7f0d1e800700 1 -- 172.21.15.42:0/2351687019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d181045e0 msgr2=0x7f0d181049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:17.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.863+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2351687019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d181045e0 0x7f0d181049f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800b0d0 tx=0x7f0d0800b3e0 comp rx=0 tx=0).stop 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 -- 172.21.15.42:0/2351687019 shutdown_connections 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2351687019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d18109860 0x7f0d181111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2351687019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d181045e0 0x7f0d181049f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2351687019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d18108f10 0x7f0d18109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 -- 172.21.15.42:0/2351687019 >> 172.21.15.42:0/2351687019 conn(0x7f0d18100030 msgr2=0x7f0d18102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 -- 172.21.15.42:0/2351687019 shutdown_connections 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.864+0000 7f0d1e800700 1 -- 172.21.15.42:0/2351687019 wait complete. 2024-05-27T10:18:17.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.865+0000 7f0d1e800700 1 Processor -- start 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.865+0000 7f0d1e800700 1 -- start start 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d1e800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d181045e0 0x7f0d1819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d1e800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 0x7f0d1819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d1e800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d18109860 0x7f0d181a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d1e800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18114340 con 0x7f0d18108f10 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d1e800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181141c0 con 0x7f0d18109860 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d1e800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181144c0 con 0x7f0d181045e0 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d0ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 0x7f0d1819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:17.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d0ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 0x7f0d1819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43718/0 (socket says 172.21.15.42:43718) 2024-05-27T10:18:17.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d0ea00700 1 -- 172.21.15.42:0/2919058194 learned_addr learned my addr 172.21.15.42:0/2919058194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:17.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.866+0000 7f0d17400700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d181045e0 0x7f0d1819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:17.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d0ea00700 1 -- 172.21.15.42:0/2919058194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d181045e0 msgr2=0x7f0d1819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:17.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d0ea00700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d181045e0 0x7f0d1819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:17.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d0ea00700 1 -- 172.21.15.42:0/2919058194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d18109860 msgr2=0x7f0d181a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d0ea00700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d18109860 0x7f0d181a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d0ea00700 1 -- 172.21.15.42:0/2919058194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d181a1d90 con 0x7f0d18108f10 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d0ea00700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 0x7f0d1819d510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000d980 tx=0x7f0d0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00017070 con 0x7f0d18108f10 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d181a2080 con 0x7f0d18108f10 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.867+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d00009dc0 con 0x7f0d18108f10 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.868+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000f3c0 con 0x7f0d18108f10 2024-05-27T10:18:17.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.868+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d181a9a50 con 0x7f0d18108f10 2024-05-27T10:18:17.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.869+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d180512b0 con 0x7f0d18108f10 2024-05-27T10:18:17.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.871+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0d00009930 con 0x7f0d18108f10 2024-05-27T10:18:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.873+0000 7f0d14c00700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0cfc0b2900 0x7f0cfc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:17.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.873+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0d000e1cc0 con 0x7f0d18108f10 2024-05-27T10:18:17.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.873+0000 7f0d17400700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0cfc0b2900 0x7f0cfc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:17.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.874+0000 7f0d17400700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0cfc0b2900 0x7f0cfc0b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009fd0 tx=0x7f0d0800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:17.878+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d000e8080 con 0x7f0d18108f10 2024-05-27T10:18:18.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:17 smithi143 ceph-mon[35545]: pgmap v509: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:18.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:17 smithi042 ceph-mon[31661]: pgmap v509: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:18.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.119+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d181a9c20 con 0x7f0d18108f10 2024-05-27T10:18:18.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.122+0000 7f0d14c00700 1 -- 172.21.15.42:0/2919058194 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22562 (secure 0 0 0) 0x7f0d000a99b0 con 0x7f0d18108f10 2024-05-27T10:18:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:18.123 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:14:41.618756+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0cfc0b2900 msgr2=0x7f0cfc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0cfc0b2900 0x7f0cfc0b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009fd0 tx=0x7f0d0800b040 comp rx=0 tx=0).stop 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 msgr2=0x7f0d1819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 0x7f0d1819d510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000d980 tx=0x7f0d0000dc90 comp rx=0 tx=0).stop 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 shutdown_connections 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0cfc0b2900 0x7f0cfc0b4e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d18109860 0x7f0d181a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d181045e0 0x7f0d1819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 --2- 172.21.15.42:0/2919058194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d18108f10 0x7f0d1819d510 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:18.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 >> 172.21.15.42:0/2919058194 conn(0x7f0d18100030 msgr2=0x7f0d18100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:18.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 shutdown_connections 2024-05-27T10:18:18.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:18.125+0000 7f0d1e800700 1 -- 172.21.15.42:0/2919058194 wait complete. 2024-05-27T10:18:18.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:17 smithi104 ceph-mon[34812]: pgmap v509: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:18 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2919058194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:18:19.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:18 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2919058194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:18:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:18 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2919058194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:18:19.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T10:14:18.127208+0000', 'modified': '2024-05-27T10:14:45.975282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34499, 'mds_1': 34466, 'mds_2': 34442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/4166809554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 4166809554}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 4166809554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6839/517039305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 517039305}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 517039305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34421': {'gid': 34421, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/3150585498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 3150585498}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 3150585498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.104:6835/3501163811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 3501163811}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 3501163811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34466': {'gid': 34466, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.104:6837/219118322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 219118322}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 219118322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34499': {'gid': 34499, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.104:6839/3864142797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3864142797}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3864142797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34499, 'qdb_cluster': [34466, 34499, 34442]} 2024-05-27T10:18:19.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-27T10:18:19.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T10:18:19.665 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:20.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:19 smithi042 ceph-mon[31661]: pgmap v510: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:20.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:19 smithi143 ceph-mon[35545]: pgmap v510: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:20.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:19 smithi104 ceph-mon[34812]: pgmap v510: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:21.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:20 smithi143 ceph-mon[35545]: pgmap v511: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:21.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:20 smithi042 ceph-mon[31661]: pgmap v511: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:21.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:20 smithi104 ceph-mon[34812]: pgmap v511: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:22.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.240+0000 7f61fe800700 1 -- 172.21.15.42:0/1210939051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 msgr2=0x7f61f8110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.240+0000 7f61fe800700 1 --2- 172.21.15.42:0/1210939051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f8110bd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009990 tx=0x7f61f0009ca0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 -- 172.21.15.42:0/1210939051 shutdown_connections 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 --2- 172.21.15.42:0/1210939051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f8110bd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 --2- 172.21.15.42:0/1210939051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61f8105300 0x7f61f8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 --2- 172.21.15.42:0/1210939051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f8105c50 0x7f61f810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 -- 172.21.15.42:0/1210939051 >> 172.21.15.42:0/1210939051 conn(0x7f61f8078ab0 msgr2=0x7f61f8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 -- 172.21.15.42:0/1210939051 shutdown_connections 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.241+0000 7f61fe800700 1 -- 172.21.15.42:0/1210939051 wait complete. 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.242+0000 7f61fe800700 1 Processor -- start 2024-05-27T10:18:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.242+0000 7f61fe800700 1 -- start start 2024-05-27T10:18:22.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fe800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f8105300 0x7f61f81a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fe800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61f8105c50 0x7f61f81a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fe800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fe800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8114150 con 0x7f61f810e730 2024-05-27T10:18:22.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fe800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8113fd0 con 0x7f61f8105c50 2024-05-27T10:18:22.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fe800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81142d0 con 0x7f61f8105300 2024-05-27T10:18:22.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43736/0 (socket says 172.21.15.42:43736) 2024-05-27T10:18:22.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.243+0000 7f61fde00700 1 -- 172.21.15.42:0/977798570 learned_addr learned my addr 172.21.15.42:0/977798570 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:22.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fd400700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f8105300 0x7f61f81a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fca00700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61f8105c50 0x7f61f81a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fde00700 1 -- 172.21.15.42:0/977798570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f8105300 msgr2=0x7f61f81a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fde00700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f8105300 0x7f61f81a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fde00700 1 -- 172.21.15.42:0/977798570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61f8105c50 msgr2=0x7f61f81a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fde00700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61f8105c50 0x7f61f81a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fde00700 1 -- 172.21.15.42:0/977798570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f819d160 con 0x7f61f810e730 2024-05-27T10:18:22.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.244+0000 7f61fde00700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f819caf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005b40 tx=0x7f61f00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:22.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.245+0000 7f61f6000700 1 -- 172.21.15.42:0/977798570 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0027070 con 0x7f61f810e730 2024-05-27T10:18:22.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.245+0000 7f61f6000700 1 -- 172.21.15.42:0/977798570 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61f000fc00 con 0x7f61f810e730 2024-05-27T10:18:22.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.245+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f819d3f0 con 0x7f61f810e730 2024-05-27T10:18:22.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.245+0000 7f61f6000700 1 -- 172.21.15.42:0/977798570 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0022c40 con 0x7f61f810e730 2024-05-27T10:18:22.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.245+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f819d920 con 0x7f61f810e730 2024-05-27T10:18:22.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.249+0000 7f61f6000700 1 -- 172.21.15.42:0/977798570 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f61f002a070 con 0x7f61f810e730 2024-05-27T10:18:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.250+0000 7f620e989b80 1 Processor -- start 2024-05-27T10:18:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.250+0000 7f620e989b80 1 -- start start 2024-05-27T10:18:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61f6000700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61e00b2900 0x7f61e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f620e989b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7e9f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f620e989b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f620e989b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea02fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f620e989b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c7e924df0 con 0x558c7e877df0 2024-05-27T10:18:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f620e989b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c7e914990 con 0x558c7e9f67d0 2024-05-27T10:18:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f620e989b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c7e9189d0 con 0x558c7e9f63c0 2024-05-27T10:18:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61f6000700 1 -- 172.21.15.42:0/977798570 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f61f00e3180 con 0x7f61f810e730 2024-05-27T10:18:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61fd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7e9f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61fd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7e9f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43738/0 (socket says 172.21.15.42:43738) 2024-05-27T10:18:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61fd400700 1 -- 172.21.15.42:0/704277876 learned_addr learned my addr 172.21.15.42:0/704277876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61fca00700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61e00b2900 0x7f61e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.251+0000 7f61fde00700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea02fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fca00700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fd400700 1 -- 172.21.15.42:0/704277876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f63c0 msgr2=0x558c7ea00a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fd400700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fd400700 1 -- 172.21.15.42:0/704277876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 msgr2=0x558c7ea02fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fd400700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea02fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fd400700 1 -- 172.21.15.42:0/704277876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7e8dd3d0 con 0x558c7e877df0 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fca00700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fde00700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea02fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fd400700 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7e9f5e80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00f570 tx=0x7f61ec00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141960e0cbe97a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:22.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.252+0000 7f61fca00700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61e00b2900 0x7f61e00b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x558c7e9f73e0 tx=0x7f61e4005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:22.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.253+0000 7f61eaa00700 1 -- 172.21.15.42:0/704277876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec018020 con 0x558c7e877df0 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.253+0000 7f61eaa00700 1 -- 172.21.15.42:0/704277876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61ec014b10 con 0x558c7e877df0 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.253+0000 7f61eaa00700 1 -- 172.21.15.42:0/704277876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec013ae0 con 0x558c7e877df0 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 -- 172.21.15.42:0/704277876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 msgr2=0x558c7e9f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7e9f5e80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00f570 tx=0x7f61ec00f880 comp rx=0 tx=0).stop 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 -- 172.21.15.42:0/704277876 shutdown_connections 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea02fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7e9f5e80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 --2- 172.21.15.42:0/704277876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 -- 172.21.15.42:0/704277876 >> 172.21.15.42:0/704277876 conn(0x558c7e8d7f70 msgr2=0x558c7e9f1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 -- 172.21.15.42:0/704277876 shutdown_connections 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.254+0000 7f620e989b80 1 -- 172.21.15.42:0/704277876 wait complete. 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.255+0000 7f620e989b80 1 -- start start 2024-05-27T10:18:22.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.255+0000 7f620e989b80 1 Processor -- start 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7ea004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7ea00f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c7e81af50 con 0x558c7e877df0 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c7e8b5710 con 0x558c7e9f63c0 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea03940 con 0x558c7e9f67d0 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f620e989b80 10 client.0 _ll_register_callbacks cb 0x558c7e8bec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:18:22.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7ea004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7ea004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43742/0 (socket says 172.21.15.42:43742) 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fd400700 1 -- 172.21.15.42:0/4077218938 learned_addr learned my addr 172.21.15.42:0/4077218938 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fca00700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fde00700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7ea00f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fd400700 1 -- 172.21.15.42:0/4077218938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 msgr2=0x558c7ea00f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.256+0000 7f61fd400700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7ea00f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61fd400700 1 -- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f63c0 msgr2=0x558c7ea00a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61fd400700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61fd400700 1 -- 172.21.15.42:0/4077218938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7e864880 con 0x558c7e877df0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61fd400700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7ea004e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0060b0 tx=0x7f61ec014730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440357e65ef366d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61df400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61df400700 1 -- 172.21.15.42:0/4077218938 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec018030 con 0x558c7e877df0 2024-05-27T10:18:22.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f61df400700 1 -- 172.21.15.42:0/4077218938 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f61ec013760 con 0x558c7e877df0 2024-05-27T10:18:22.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.257+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61ec014b10 con 0x558c7e877df0 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f61df400700 1 -- 172.21.15.42:0/4077218938 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61ec013960 con 0x558c7e877df0 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f620e989b80 10 client.15078 fetch_fsmap learned FSMap version 25 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c7e8e5010 con 0x558c7e877df0 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f61df400700 1 -- 172.21.15.42:0/4077218938 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61ec014b10 con 0x558c7e877df0 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f61df400700 1 -- 172.21.15.42:0/4077218938 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f61ec031a00 con 0x558c7e877df0 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f620e989b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f620e989b80 10 client.15078 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.258+0000 7f620e989b80 20 client.15078 populate_metadata read hostname 'smithi042' 2024-05-27T10:18:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.259+0000 7f620e989b80 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea103c0 0x558c7ea11830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.259+0000 7f620e989b80 4 client.15078 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:18:22.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.259+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c7e926d40 con 0x558c7ea103c0 2024-05-27T10:18:22.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.259+0000 7f61fca00700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea103c0 0x558c7ea11830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.260+0000 7f61fca00700 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea103c0 0x558c7ea11830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8158568997d3b077 server_cookie=a00820056abe81c9 in_seq=0 out_seq=0 2024-05-27T10:18:22.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.260+0000 7f61df400700 10 client.15078 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:18:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.262+0000 7f61df400700 1 -- 172.21.15.42:0/4077218938 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558c7e926d40 con 0x558c7ea103c0 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.262+0000 7f61df400700 10 client.15078 handle_command_reply: tid=0 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 client.15078 shutdown 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 msgr2=0x558c7ea004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7ea004e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0060b0 tx=0x7f61ec014730 comp rx=0 tx=0).stop 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 shutdown_connections 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea103c0 0x558c7ea11830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7ea00f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e877df0 0x558c7ea004e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 --2- 172.21.15.42:0/4077218938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f63c0 0x558c7ea00a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 >> 172.21.15.42:0/4077218938 conn(0x558c7e8d7f70 msgr2=0x558c7e9f0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f61df400700 0 client.15078 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:18:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 shutdown_connections 2024-05-27T10:18:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 1 -- 172.21.15.42:0/4077218938 wait complete. 2024-05-27T10:18:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.263+0000 7f620e989b80 20 client.15078 trim_cache size 0 max 16384 2024-05-27T10:18:22.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 Processor -- start 2024-05-27T10:18:22.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 -- start start 2024-05-27T10:18:22.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e877df0 0x558c7e9fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea0cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7e9ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea0fb70 con 0x558c7e9f63c0 2024-05-27T10:18:22.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea10240 con 0x558c7e877df0 2024-05-27T10:18:22.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.287+0000 7f620e989b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea10910 con 0x558c7e9f67d0 2024-05-27T10:18:22.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea0cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea0cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43752/0 (socket says 172.21.15.42:43752) 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 -- 172.21.15.42:0/515218572 learned_addr learned my addr 172.21.15.42:0/515218572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fd400700 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e877df0 0x558c7e9fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fde00700 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7e9ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 -- 172.21.15.42:0/515218572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 msgr2=0x558c7e9ff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7e9ff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 -- 172.21.15.42:0/515218572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e877df0 msgr2=0x558c7e9fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e877df0 0x558c7e9fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fca00700 1 -- 172.21.15.42:0/515218572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7e9f3300 con 0x558c7e9f63c0 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.288+0000 7f61fd400700 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e877df0 0x558c7e9fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:22.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.289+0000 7f61fca00700 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea0cc10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x558c7e9f3000 tx=0x7f61e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9657f04af5643c70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.289+0000 7f61ea000700 1 -- 172.21.15.42:0/515218572 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4003940 con 0x558c7e9f63c0 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.289+0000 7f61ea000700 1 -- 172.21.15.42:0/515218572 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e402b070 con 0x558c7e9f63c0 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.289+0000 7f61ea000700 1 -- 172.21.15.42:0/515218572 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4026bf0 con 0x558c7e9f63c0 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 -- 172.21.15.42:0/515218572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 msgr2=0x558c7ea0cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea0cc10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x558c7e9f3000 tx=0x7f61e4014000 comp rx=0 tx=0).stop 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 -- 172.21.15.42:0/515218572 shutdown_connections 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e9f67d0 0x558c7e9ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e877df0 0x558c7e9fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 --2- 172.21.15.42:0/515218572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea0cc10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 -- 172.21.15.42:0/515218572 >> 172.21.15.42:0/515218572 conn(0x558c7e8d7f70 msgr2=0x558c7e9fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 -- 172.21.15.42:0/515218572 shutdown_connections 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.290+0000 7f620e989b80 1 -- 172.21.15.42:0/515218572 wait complete. 2024-05-27T10:18:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.291+0000 7f620e989b80 1 -- start start 2024-05-27T10:18:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.291+0000 7f620e989b80 1 Processor -- start 2024-05-27T10:18:22.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.291+0000 7f620e989b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e877df0 0x558c7ea1b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.291+0000 7f620e989b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.291+0000 7f620e989b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea14f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f620e989b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea116b0 con 0x558c7e9f63c0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f620e989b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea10fe0 con 0x558c7e9f67d0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f620e989b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c7ea11d80 con 0x558c7e877df0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f620e989b80 10 client.0 _ll_register_callbacks cb 0x558c7e8bec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea14870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43768/0 (socket says 172.21.15.42:43768) 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 -- 172.21.15.42:0/3885101750 learned_addr learned my addr 172.21.15.42:0/3885101750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fd400700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e877df0 0x558c7ea1b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fde00700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea14f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 -- 172.21.15.42:0/3885101750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e877df0 msgr2=0x558c7ea1b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e877df0 0x558c7ea1b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 -- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 msgr2=0x558c7ea14f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea14f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fca00700 1 -- 172.21.15.42:0/3885101750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7ea012d0 con 0x558c7e9f63c0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.292+0000 7f61fd400700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e877df0 0x558c7ea1b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f61fca00700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea14870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61e40059d0 tx=0x7f61e40265c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b018dff821cf6782 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f61df400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f61df400700 1 -- 172.21.15.42:0/3885101750 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4035020 con 0x558c7e9f63c0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f61df400700 1 -- 172.21.15.42:0/3885101750 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f61e4003740 con 0x558c7e9f63c0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e402b070 con 0x558c7e9f63c0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f61df400700 1 -- 172.21.15.42:0/3885101750 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61e4003940 con 0x558c7e9f63c0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f620e989b80 10 client.15084 fetch_fsmap learned FSMap version 25 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.293+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c7ea7bb50 con 0x558c7e9f63c0 2024-05-27T10:18:22.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f61df400700 1 -- 172.21.15.42:0/3885101750 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e402b070 con 0x558c7e9f63c0 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f61df400700 1 -- 172.21.15.42:0/3885101750 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f61e400f040 con 0x558c7e9f63c0 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f620e989b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f620e989b80 10 client.15084 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f620e989b80 20 client.15084 populate_metadata read hostname 'smithi042' 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f620e989b80 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea7dc00 0x558c7ea7f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f620e989b80 4 client.15084 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:18:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.294+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "status"}) -- 0x558c7e920520 con 0x558c7ea7dc00 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.295+0000 7f61fd400700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea7dc00 0x558c7ea7f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.296+0000 7f61fd400700 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea7dc00 0x558c7ea7f070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70217e328d440d5c server_cookie=dbc3ef59ca21afee in_seq=0 out_seq=0 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.296+0000 7f61df400700 10 client.15084 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.296+0000 7f61df400700 1 -- 172.21.15.42:0/3885101750 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558c7e920520 con 0x558c7ea7dc00 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.296+0000 7f61df400700 10 client.15084 handle_command_reply: tid=0 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.296+0000 7f620e989b80 1 client.15084 shutdown 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 msgr2=0x558c7ea14870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea14870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61e40059d0 tx=0x7f61e40265c0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 shutdown_connections 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x558c7ea7dc00 0x558c7ea7f070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c7e9f67d0 0x558c7ea14f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c7e877df0 0x558c7ea1b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f61df400700 0 client.15084 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:18:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 --2- 172.21.15.42:0/3885101750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c7e9f63c0 0x558c7ea14870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 >> 172.21.15.42:0/3885101750 conn(0x558c7e8d7f70 msgr2=0x558c7ea002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 shutdown_connections 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 1 -- 172.21.15.42:0/3885101750 wait complete. 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.297+0000 7f620e989b80 20 client.15084 trim_cache size 0 max 16384 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34442, 2024-05-27T10:18:22.300 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 224.70770015400001, 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 25, 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 81, 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 299.12581063099998 2024-05-27T10:18:22.301 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61e00b2900 msgr2=0x7f61e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61e00b2900 0x7f61e00b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x558c7e9f73e0 tx=0x7f61e4005f70 comp rx=0 tx=0).stop 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 msgr2=0x7f61f819caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f819caf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005b40 tx=0x7f61f00057c0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 shutdown_connections 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f810e730 0x7f61f819caf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f8105300 0x7f61f81a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f61e00b2900 0x7f61e00b4e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 --2- 172.21.15.42:0/977798570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61f8105c50 0x7f61f81a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 >> 172.21.15.42:0/977798570 conn(0x7f61f8078ab0 msgr2=0x7f61f8101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 shutdown_connections 2024-05-27T10:18:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:22.298+0000 7f61fe800700 1 -- 172.21.15.42:0/977798570 wait complete. 2024-05-27T10:18:23.312 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T10:18:23.765 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:23.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:23 smithi042 ceph-mon[31661]: pgmap v512: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:23.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:23 smithi104 ceph-mon[34812]: pgmap v512: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:24.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:23 smithi143 ceph-mon[35545]: pgmap v512: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T10:18:25.708 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:25 smithi042 ceph-mon[31661]: pgmap v513: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:25.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:25 smithi104 ceph-mon[34812]: pgmap v513: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:26.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:25 smithi143 ceph-mon[35545]: pgmap v513: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:18:26.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 -- 172.21.15.42:0/3516609711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 msgr2=0x7fac3010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 --2- 172.21.15.42:0/3516609711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac3010e120 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009990 tx=0x7fac1c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 -- 172.21.15.42:0/3516609711 shutdown_connections 2024-05-27T10:18:26.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 --2- 172.21.15.42:0/3516609711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3010e6f0 0x7fac30110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 --2- 172.21.15.42:0/3516609711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac30105280 0x7fac30105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 --2- 172.21.15.42:0/3516609711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac3010e120 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.404+0000 7fac36a00700 1 -- 172.21.15.42:0/3516609711 >> 172.21.15.42:0/3516609711 conn(0x7fac30078af0 msgr2=0x7fac30078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.405+0000 7fac36a00700 1 -- 172.21.15.42:0/3516609711 shutdown_connections 2024-05-27T10:18:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.405+0000 7fac36a00700 1 -- 172.21.15.42:0/3516609711 wait complete. 2024-05-27T10:18:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 Processor -- start 2024-05-27T10:18:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 -- start start 2024-05-27T10:18:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac30105280 0x7fac30102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac301034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac3010e6f0 0x7fac301039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac30114100 con 0x7fac30105bd0 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac30113f80 con 0x7fac30105280 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.406+0000 7fac36a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fac30114280 con 0x7fac3010e6f0 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac301034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac301034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52544/0 (socket says 172.21.15.42:52544) 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 -- 172.21.15.42:0/3488914750 learned_addr learned my addr 172.21.15.42:0/3488914750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac35600700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac30105280 0x7fac30102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac36000700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac3010e6f0 0x7fac301039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 -- 172.21.15.42:0/3488914750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac3010e6f0 msgr2=0x7fac301039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac3010e6f0 0x7fac301039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 -- 172.21.15.42:0/3488914750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac30105280 msgr2=0x7fac30102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac30105280 0x7fac30102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.407+0000 7fac34c00700 1 -- 172.21.15.42:0/3488914750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac301a8f00 con 0x7fac30105bd0 2024-05-27T10:18:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.408+0000 7fac34c00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac301034b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.408+0000 7fac2e000700 1 -- 172.21.15.42:0/3488914750 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c027070 con 0x7fac30105bd0 2024-05-27T10:18:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.408+0000 7fac2e000700 1 -- 172.21.15.42:0/3488914750 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac1c00fb80 con 0x7fac30105bd0 2024-05-27T10:18:26.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.408+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac301a9190 con 0x7fac30105bd0 2024-05-27T10:18:26.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.408+0000 7fac2e000700 1 -- 172.21.15.42:0/3488914750 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c022d00 con 0x7fac30105bd0 2024-05-27T10:18:26.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.408+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301a95e0 con 0x7fac30105bd0 2024-05-27T10:18:26.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.412+0000 7fac2e000700 1 -- 172.21.15.42:0/3488914750 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fac1c02a070 con 0x7fac30105bd0 2024-05-27T10:18:26.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 Processor -- start 2024-05-27T10:18:26.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 -- start start 2024-05-27T10:18:26.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac2e000700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fac180b2880 0x7fac180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac2e000700 1 -- 172.21.15.42:0/3488914750 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fac1c0e3250 con 0x7fac30105bd0 2024-05-27T10:18:26.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff3959a0 0x55faff429700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac36000700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fac180b2880 0x7fac180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff2a9df0 0x55faff429c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 0x55faff435690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55faff356df0 con 0x55faff433270 2024-05-27T10:18:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.414+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55faff346990 con 0x55faff3959a0 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55faff34a9d0 con 0x55faff2a9df0 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac36000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 0x55faff435690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac36000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 0x55faff435690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52560/0 (socket says 172.21.15.42:52560) 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac36000700 1 -- 172.21.15.42:0/705815755 learned_addr learned my addr 172.21.15.42:0/705815755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac35600700 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff3959a0 0x55faff429700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac34c00700 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff2a9df0 0x55faff429c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac36000700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fac180b2880 0x7fac180b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac28012770 tx=0x7fac28012b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac36000700 1 -- 172.21.15.42:0/705815755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff2a9df0 msgr2=0x55faff429c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.415+0000 7fac36000700 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff2a9df0 0x55faff429c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac36000700 1 -- 172.21.15.42:0/705815755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff3959a0 msgr2=0x55faff429700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac36000700 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff3959a0 0x55faff429700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac36000700 1 -- 172.21.15.42:0/705815755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faff30f3d0 con 0x55faff433270 2024-05-27T10:18:26.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac34c00700 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff2a9df0 0x55faff429c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac36000700 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 0x55faff435690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fac28012f30 tx=0x7fac2801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4680b99ac0310513 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac22a00700 1 -- 172.21.15.42:0/705815755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28015e80 con 0x55faff433270 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac22a00700 1 -- 172.21.15.42:0/705815755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac28011040 con 0x55faff433270 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.416+0000 7fac22a00700 1 -- 172.21.15.42:0/705815755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28021660 con 0x55faff433270 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.417+0000 7fac46aaeb80 1 -- 172.21.15.42:0/705815755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 msgr2=0x55faff435690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.417+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 0x55faff435690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fac28012f30 tx=0x7fac2801c000 comp rx=0 tx=0).stop 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 -- 172.21.15.42:0/705815755 shutdown_connections 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff433270 0x55faff435690 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff3959a0 0x55faff429700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/705815755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff2a9df0 0x55faff429c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 -- 172.21.15.42:0/705815755 >> 172.21.15.42:0/705815755 conn(0x55faff343340 msgr2=0x55faff4246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 -- 172.21.15.42:0/705815755 shutdown_connections 2024-05-27T10:18:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 -- 172.21.15.42:0/705815755 wait complete. 2024-05-27T10:18:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 -- start start 2024-05-27T10:18:26.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.418+0000 7fac46aaeb80 1 Processor -- start 2024-05-27T10:18:26.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff4301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff3959a0 0x55faff430710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff430c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55faff24cf50 con 0x55faff2a9df0 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55faff2e7710 con 0x55faff433270 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55faff435d50 con 0x55faff3959a0 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac46aaeb80 10 client.0 _ll_register_callbacks cb 0x55faff2f0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac35600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff4301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac35600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff4301d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52576/0 (socket says 172.21.15.42:52576) 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.419+0000 7fac35600700 1 -- 172.21.15.42:0/2401808706 learned_addr learned my addr 172.21.15.42:0/2401808706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac34c00700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff3959a0 0x55faff430710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac36000700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff430c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac35600700 1 -- 172.21.15.42:0/2401808706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff3959a0 msgr2=0x55faff430710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac35600700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff3959a0 0x55faff430710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac35600700 1 -- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 msgr2=0x55faff430c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac35600700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff430c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac35600700 1 -- 172.21.15.42:0/2401808706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faff296880 con 0x55faff2a9df0 2024-05-27T10:18:26.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac35600700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff4301d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac2400bb70 tx=0x7fac2400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5b7043e8e4b020 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:26.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.420+0000 7fac17400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:18:26.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac17400700 1 -- 172.21.15.42:0/2401808706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24003ca0 con 0x55faff2a9df0 2024-05-27T10:18:26.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac17400700 1 -- 172.21.15.42:0/2401808706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fac24004830 con 0x55faff2a9df0 2024-05-27T10:18:26.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac28011040 con 0x55faff2a9df0 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac17400700 1 -- 172.21.15.42:0/2401808706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac24022070 con 0x55faff2a9df0 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac46aaeb80 10 client.15096 fetch_fsmap learned FSMap version 25 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faff317010 con 0x55faff2a9df0 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.421+0000 7fac17400700 1 -- 172.21.15.42:0/2401808706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac28011040 con 0x55faff2a9df0 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac17400700 1 -- 172.21.15.42:0/2401808706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fac24011440 con 0x55faff2a9df0 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac46aaeb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac46aaeb80 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T10:18:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac46aaeb80 20 client.15096 populate_metadata read hostname 'smithi042' 2024-05-27T10:18:26.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff442c90 0x55faff444100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac46aaeb80 4 client.15096 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:18:26.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faff358d40 con 0x55faff442c90 2024-05-27T10:18:26.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.422+0000 7fac34c00700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff442c90 0x55faff444100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.424+0000 7fac34c00700 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff442c90 0x55faff444100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=854f73c73138a0cf server_cookie=cf0c8e7246892978 in_seq=0 out_seq=0 2024-05-27T10:18:26.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.424+0000 7fac17400700 10 client.15096 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:18:26.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac17400700 1 -- 172.21.15.42:0/2401808706 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55faff358d40 con 0x55faff442c90 2024-05-27T10:18:26.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac17400700 10 client.15096 handle_command_reply: tid=0 2024-05-27T10:18:26.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 client.15096 shutdown 2024-05-27T10:18:26.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 msgr2=0x55faff4301d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff4301d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac2400bb70 tx=0x7fac2400bf30 comp rx=0 tx=0).stop 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 shutdown_connections 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff442c90 0x55faff444100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff430c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff4301d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/2401808706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff3959a0 0x55faff430710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 >> 172.21.15.42:0/2401808706 conn(0x55faff343340 msgr2=0x55faff429700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac17400700 0 client.15096 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 shutdown_connections 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.426+0000 7fac46aaeb80 1 -- 172.21.15.42:0/2401808706 wait complete. 2024-05-27T10:18:26.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.427+0000 7fac46aaeb80 20 client.15096 trim_cache size 0 max 16384 2024-05-27T10:18:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.450+0000 7fac46aaeb80 1 Processor -- start 2024-05-27T10:18:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.450+0000 7fac46aaeb80 1 -- start start 2024-05-27T10:18:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff2a9df0 0x55faff43f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 0x55faff4240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff433270 0x55faff424600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55faff442440 con 0x55faff3959a0 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55faff442b10 con 0x55faff2a9df0 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55faff4431e0 con 0x55faff433270 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 0x55faff4240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 0x55faff4240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52580/0 (socket says 172.21.15.42:52580) 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 -- 172.21.15.42:0/3370456967 learned_addr learned my addr 172.21.15.42:0/3370456967 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac36000700 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff433270 0x55faff424600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac35600700 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff2a9df0 0x55faff43f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 -- 172.21.15.42:0/3370456967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff433270 msgr2=0x55faff424600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff433270 0x55faff424600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 -- 172.21.15.42:0/3370456967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff2a9df0 msgr2=0x55faff43f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.451+0000 7fac34c00700 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff2a9df0 0x55faff43f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.452+0000 7fac34c00700 1 -- 172.21.15.42:0/3370456967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faff432f70 con 0x55faff3959a0 2024-05-27T10:18:26.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.452+0000 7fac34c00700 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 0x55faff4240c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0a6950 tx=0x7fac1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458b6dc273798fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.452+0000 7fac22000700 1 -- 172.21.15.42:0/3370456967 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0b36a0 con 0x55faff3959a0 2024-05-27T10:18:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.452+0000 7fac22000700 1 -- 172.21.15.42:0/3370456967 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac1c0e0e30 con 0x55faff3959a0 2024-05-27T10:18:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.452+0000 7fac22000700 1 -- 172.21.15.42:0/3370456967 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0aad90 con 0x55faff3959a0 2024-05-27T10:18:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 -- 172.21.15.42:0/3370456967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 msgr2=0x55faff4240c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 0x55faff4240c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0a6950 tx=0x7fac1c009310 comp rx=0 tx=0).stop 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 -- 172.21.15.42:0/3370456967 shutdown_connections 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff433270 0x55faff424600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff2a9df0 0x55faff43f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/3370456967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff3959a0 0x55faff4240c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 -- 172.21.15.42:0/3370456967 >> 172.21.15.42:0/3370456967 conn(0x55faff343340 msgr2=0x55faff4236e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 -- 172.21.15.42:0/3370456967 shutdown_connections 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.453+0000 7fac46aaeb80 1 -- 172.21.15.42:0/3370456967 wait complete. 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.454+0000 7fac46aaeb80 1 -- start start 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.454+0000 7fac46aaeb80 1 Processor -- start 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.454+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff44dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff447000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac46aaeb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff447540 0x55faff447a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55faff443f80 con 0x55faff2a9df0 2024-05-27T10:18:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55faff4438b0 con 0x55faff433270 2024-05-27T10:18:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac46aaeb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55faff444650 con 0x55faff447540 2024-05-27T10:18:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac46aaeb80 10 client.0 _ll_register_callbacks cb 0x55faff2f0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac35600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff44dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac35600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff44dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52590/0 (socket says 172.21.15.42:52590) 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac35600700 1 -- 172.21.15.42:0/1967213058 learned_addr learned my addr 172.21.15.42:0/1967213058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac36000700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff447540 0x55faff447a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac35600700 1 -- 172.21.15.42:0/1967213058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff447540 msgr2=0x55faff447a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac35600700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff447540 0x55faff447a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac35600700 1 -- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 msgr2=0x55faff447000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.455+0000 7fac34c00700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff447000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac35600700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff447000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac35600700 1 -- 172.21.15.42:0/1967213058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faff44b9b0 con 0x55faff2a9df0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac36000700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff447540 0x55faff447a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac34c00700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff447000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac35600700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff44dcc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac24000c00 tx=0x7fac2400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4293951843f5abd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac17400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac17400700 1 -- 172.21.15.42:0/1967213058 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24011650 con 0x55faff2a9df0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac17400700 1 -- 172.21.15.42:0/1967213058 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fac24017040 con 0x55faff2a9df0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.456+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1c0e0e30 con 0x55faff2a9df0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac17400700 1 -- 172.21.15.42:0/1967213058 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac24005040 con 0x55faff2a9df0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faff4ae860 con 0x55faff2a9df0 2024-05-27T10:18:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac17400700 1 -- 172.21.15.42:0/1967213058 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac1c0e0e30 con 0x55faff2a9df0 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac17400700 1 -- 172.21.15.42:0/1967213058 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fac240049f0 con 0x55faff2a9df0 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 20 client.15102 populate_metadata read hostname 'smithi042' 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff4b0ce0 0x55faff4b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 4 client.15102 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.457+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "status"}) -- 0x55faff352520 con 0x55faff4b0ce0 2024-05-27T10:18:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.458+0000 7fac34c00700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff4b0ce0 0x55faff4b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:26.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.459+0000 7fac34c00700 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff4b0ce0 0x55faff4b2150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcf3ef1ef1e2caf8 server_cookie=f53c13aaa171bc78 in_seq=0 out_seq=0 2024-05-27T10:18:26.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.459+0000 7fac17400700 10 client.15102 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:18:26.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.459+0000 7fac17400700 1 -- 172.21.15.42:0/1967213058 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55faff352520 con 0x55faff4b0ce0 2024-05-27T10:18:26.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.459+0000 7fac17400700 10 client.15102 handle_command_reply: tid=0 2024-05-27T10:18:26.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.459+0000 7fac46aaeb80 1 client.15102 shutdown 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 msgr2=0x55faff44dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff44dcc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac24000c00 tx=0x7fac2400b5c0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 shutdown_connections 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55faff4b0ce0 0x55faff4b2150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55faff447540 0x55faff447a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55faff2a9df0 0x55faff44dcc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 --2- 172.21.15.42:0/1967213058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faff433270 0x55faff447000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 >> 172.21.15.42:0/1967213058 conn(0x55faff343340 msgr2=0x55faff42eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac17400700 0 client.15102 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 shutdown_connections 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 1 -- 172.21.15.42:0/1967213058 wait complete. 2024-05-27T10:18:26.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.460+0000 7fac46aaeb80 20 client.15102 trim_cache size 0 max 16384 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34466, 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 229.885196452, 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 25, 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 81, 2024-05-27T10:18:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-05-27T10:18:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 290.999676706 2024-05-27T10:18:26.465 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:18:26.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fac180b2880 msgr2=0x7fac180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fac180b2880 0x7fac180b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac28012770 tx=0x7fac28012b30 comp rx=0 tx=0).stop 2024-05-27T10:18:26.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 msgr2=0x7fac301034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:26.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac301034b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c0047b0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 shutdown_connections 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac3010e6f0 0x7fac301039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7fac180b2880 0x7fac180b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac30105280 0x7fac30102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 --2- 172.21.15.42:0/3488914750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac30105bd0 0x7fac301034b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 >> 172.21.15.42:0/3488914750 conn(0x7fac30078af0 msgr2=0x7fac30101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.461+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 shutdown_connections 2024-05-27T10:18:26.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:26.462+0000 7fac36a00700 1 -- 172.21.15.42:0/3488914750 wait complete. 2024-05-27T10:18:27.595 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T10:18:27.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:27 smithi042 ceph-mon[31661]: pgmap v514: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:27.859 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:27 smithi143 ceph-mon[35545]: pgmap v514: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:27 smithi104 ceph-mon[34812]: pgmap v514: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:18:28.051 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:29.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:29 smithi042 ceph-mon[31661]: pgmap v515: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:29.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:29 smithi104 ceph-mon[34812]: pgmap v515: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:30.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:29 smithi143 ceph-mon[35545]: pgmap v515: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T10:18:30.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 -- 172.21.15.42:0/2348009359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 msgr2=0x7f8478110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 --2- 172.21.15.42:0/2348009359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f8478110fd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f84700099c0 tx=0x7f8470009cd0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 -- 172.21.15.42:0/2348009359 shutdown_connections 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 --2- 172.21.15.42:0/2348009359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f8478110fd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 --2- 172.21.15.42:0/2348009359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84781043f0 0x7f8478104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 --2- 172.21.15.42:0/2348009359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478108d20 0x7f8478109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 -- 172.21.15.42:0/2348009359 >> 172.21.15.42:0/2348009359 conn(0x7f8478079d00 msgr2=0x7f847807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 -- 172.21.15.42:0/2348009359 shutdown_connections 2024-05-27T10:18:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.558+0000 7f847ce00700 1 -- 172.21.15.42:0/2348009359 wait complete. 2024-05-27T10:18:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 Processor -- start 2024-05-27T10:18:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 -- start start 2024-05-27T10:18:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84781043f0 0x7f847819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478108d20 0x7f847819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f84781a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84781141a0 con 0x7f8478109670 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8478114020 con 0x7f8478108d20 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.560+0000 7f847ce00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8478114320 con 0x7f84781043f0 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f84781a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f84781a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52610/0 (socket says 172.21.15.42:52610) 2024-05-27T10:18:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 -- 172.21.15.42:0/3678505910 learned_addr learned my addr 172.21.15.42:0/3678505910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477400700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84781043f0 0x7f847819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f846ea00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478108d20 0x7f847819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 -- 172.21.15.42:0/3678505910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84781043f0 msgr2=0x7f847819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84781043f0 0x7f847819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 -- 172.21.15.42:0/3678505910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478108d20 msgr2=0x7f847819d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478108d20 0x7f847819d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 -- 172.21.15.42:0/3678505910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781a1ce0 con 0x7f8478109670 2024-05-27T10:18:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.561+0000 7f8477e00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f84781a1640 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f847000b580 tx=0x7f847000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:30.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.562+0000 7f8474c00700 1 -- 172.21.15.42:0/3678505910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470027070 con 0x7f8478109670 2024-05-27T10:18:30.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.562+0000 7f8474c00700 1 -- 172.21.15.42:0/3678505910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f847000fdd0 con 0x7f8478109670 2024-05-27T10:18:30.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.562+0000 7f8474c00700 1 -- 172.21.15.42:0/3678505910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470022e10 con 0x7f8478109670 2024-05-27T10:18:30.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.562+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84781a1f70 con 0x7f8478109670 2024-05-27T10:18:30.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.562+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84781a98f0 con 0x7f8478109670 2024-05-27T10:18:30.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.568+0000 7f8474c00700 1 -- 172.21.15.42:0/3678505910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f847002a070 con 0x7f8478109670 2024-05-27T10:18:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.568+0000 7f848cfb1b80 1 Processor -- start 2024-05-27T10:18:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.568+0000 7f848cfb1b80 1 -- start start 2024-05-27T10:18:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c60570e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605717d0 0x558c6057dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c6049fdf0 con 0x558c605713c0 2024-05-27T10:18:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c6048f990 con 0x558c603f2df0 2024-05-27T10:18:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c604939d0 con 0x558c605717d0 2024-05-27T10:18:30.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f8477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f8477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52618/0 (socket says 172.21.15.42:52618) 2024-05-27T10:18:30.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f8477400700 1 -- 172.21.15.42:0/3206736527 learned_addr learned my addr 172.21.15.42:0/3206736527 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:30.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.569+0000 7f8474c00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f845c0b2900 0x7f845c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f846ea00700 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c60570e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f846ea00700 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605717d0 0x558c6057dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477400700 1 -- 172.21.15.42:0/3206736527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605717d0 msgr2=0x558c6057dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477400700 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605717d0 0x558c6057dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477400700 1 -- 172.21.15.42:0/3206736527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 msgr2=0x558c60570e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477400700 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c60570e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477400700 1 -- 172.21.15.42:0/3206736527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c604583d0 con 0x558c605713c0 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f846ea00700 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605717d0 0x558c6057dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477e00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f845c0b2900 0x7f845c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8474c00700 1 -- 172.21.15.42:0/3678505910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f84700e2ff0 con 0x7f8478109670 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.570+0000 7f8477400700 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057ba80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e10 tx=0x7f84680077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf8bc90e86783f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.571+0000 7f846cc00700 1 -- 172.21.15.42:0/3206736527 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468014610 con 0x558c605713c0 2024-05-27T10:18:30.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.571+0000 7f846cc00700 1 -- 172.21.15.42:0/3206736527 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8468019070 con 0x558c605713c0 2024-05-27T10:18:30.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.571+0000 7f846cc00700 1 -- 172.21.15.42:0/3206736527 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468013970 con 0x558c605713c0 2024-05-27T10:18:30.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 -- 172.21.15.42:0/3206736527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 msgr2=0x558c6057ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057ba80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e10 tx=0x7f84680077e0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f8477e00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f845c0b2900 0x7f845c0b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84700aa3d0 tx=0x7f84700cd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 -- 172.21.15.42:0/3206736527 shutdown_connections 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605717d0 0x558c6057dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c60570e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/3206736527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057ba80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 -- 172.21.15.42:0/3206736527 >> 172.21.15.42:0/3206736527 conn(0x558c60452f70 msgr2=0x558c6056cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 -- 172.21.15.42:0/3206736527 shutdown_connections 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.572+0000 7f848cfb1b80 1 -- 172.21.15.42:0/3206736527 wait complete. 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.573+0000 7f848cfb1b80 1 -- start start 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.573+0000 7f848cfb1b80 1 Processor -- start 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c6057b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605717d0 0x558c6057bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c60395f50 con 0x558c605713c0 2024-05-27T10:18:30.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c60430710 con 0x558c605717d0 2024-05-27T10:18:30.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c6057e940 con 0x558c603f2df0 2024-05-27T10:18:30.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f848cfb1b80 10 client.0 _ll_register_callbacks cb 0x558c60439c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52620/0 (socket says 172.21.15.42:52620) 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 -- 172.21.15.42:0/1360590437 learned_addr learned my addr 172.21.15.42:0/1360590437 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f846ea00700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c6057b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f846ea00700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605717d0 0x558c6057bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 -- 172.21.15.42:0/1360590437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 msgr2=0x558c6057b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c6057b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 -- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605717d0 msgr2=0x558c6057bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.574+0000 7f8477400700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605717d0 0x558c6057bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f8477400700 1 -- 172.21.15.42:0/1360590437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c603df880 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f846ea00700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605717d0 0x558c6057bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f8477400700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057b9f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8468000f80 tx=0x7f8468004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c8be4df16a4916 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f8465600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f8465600700 1 -- 172.21.15.42:0/1360590437 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468014410 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f8465600700 1 -- 172.21.15.42:0/1360590437 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f84680145b0 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.575+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8468019070 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f8465600700 1 -- 172.21.15.42:0/1360590437 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84680147b0 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 10 client.15114 fetch_fsmap learned FSMap version 25 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c60460010 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f8465600700 1 -- 172.21.15.42:0/1360590437 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8468019070 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f8465600700 1 -- 172.21.15.42:0/1360590437 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8468005520 con 0x558c605713c0 2024-05-27T10:18:30.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 20 client.15114 populate_metadata read hostname 'smithi042' 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c6058b420 0x558c6058c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 4 client.15114 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.576+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c604a1d40 con 0x558c6058b420 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.577+0000 7f846ea00700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c6058b420 0x558c6058c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.578+0000 7f846ea00700 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c6058b420 0x558c6058c890 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e4e80d2cec60ad server_cookie=531e8d6d2935045d in_seq=0 out_seq=0 2024-05-27T10:18:30.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.578+0000 7f8465600700 10 client.15114 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.580+0000 7f8465600700 1 -- 172.21.15.42:0/1360590437 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558c604a1d40 con 0x558c6058b420 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.580+0000 7f8465600700 10 client.15114 handle_command_reply: tid=0 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.580+0000 7f848cfb1b80 1 client.15114 shutdown 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 msgr2=0x558c6057b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057b9f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8468000f80 tx=0x7f8468004f90 comp rx=0 tx=0).stop 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 shutdown_connections 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c6058b420 0x558c6058c890 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605717d0 0x558c6057bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c6057b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1360590437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605713c0 0x558c6057b9f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 >> 172.21.15.42:0/1360590437 conn(0x558c60452f70 msgr2=0x558c6056bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:30.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f8465600700 0 client.15114 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:18:30.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 shutdown_connections 2024-05-27T10:18:30.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1360590437 wait complete. 2024-05-27T10:18:30.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.581+0000 7f848cfb1b80 20 client.15114 trim_cache size 0 max 16384 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.606+0000 7f848cfb1b80 1 Processor -- start 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 -- start start 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c60576ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605713c0 0x558c60587c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c6057d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c6058abd0 con 0x558c605717d0 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c6058b2a0 con 0x558c605713c0 2024-05-27T10:18:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c6058b970 con 0x558c603f2df0 2024-05-27T10:18:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.607+0000 7f846ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c6057d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c6057d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52632/0 (socket says 172.21.15.42:52632) 2024-05-27T10:18:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 -- 172.21.15.42:0/4227993380 learned_addr learned my addr 172.21.15.42:0/4227993380 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c60576ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f8477400700 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605713c0 0x558c60587c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 -- 172.21.15.42:0/4227993380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 msgr2=0x558c60576ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c60576ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 -- 172.21.15.42:0/4227993380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605713c0 msgr2=0x558c60587c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605713c0 0x558c60587c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 -- 172.21.15.42:0/4227993380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6056e300 con 0x558c605717d0 2024-05-27T10:18:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f8477400700 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605713c0 0x558c60587c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.608+0000 7f846ea00700 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c6057d810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f846000c9a0 tx=0x7f846000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799012ed5d56c617 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:30.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.611+0000 7f8467e00700 1 -- 172.21.15.42:0/4227993380 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460004230 con 0x558c605717d0 2024-05-27T10:18:30.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.611+0000 7f8467e00700 1 -- 172.21.15.42:0/4227993380 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84600043d0 con 0x558c605717d0 2024-05-27T10:18:30.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.611+0000 7f8467e00700 1 -- 172.21.15.42:0/4227993380 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460004dc0 con 0x558c605717d0 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.612+0000 7f848cfb1b80 1 -- 172.21.15.42:0/4227993380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 msgr2=0x558c6057d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.612+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c6057d810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f846000c9a0 tx=0x7f846000c450 comp rx=0 tx=0).stop 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 -- 172.21.15.42:0/4227993380 shutdown_connections 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c6057d810 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c603f2df0 0x558c60576ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/4227993380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c605713c0 0x558c60587c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 -- 172.21.15.42:0/4227993380 >> 172.21.15.42:0/4227993380 conn(0x558c60452f70 msgr2=0x558c6056c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:30.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 -- 172.21.15.42:0/4227993380 shutdown_connections 2024-05-27T10:18:30.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.613+0000 7f848cfb1b80 1 -- 172.21.15.42:0/4227993380 wait complete. 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 -- start start 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 Processor -- start 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c6058f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605713c0 0x558c6058ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c60590470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c6058c710 con 0x558c605717d0 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c6058c040 con 0x558c603f2df0 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.614+0000 7f848cfb1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c6058cde0 con 0x558c605713c0 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.615+0000 7f848cfb1b80 10 client.0 _ll_register_callbacks cb 0x558c60439c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:18:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.615+0000 7f846ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c60590470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.615+0000 7f846ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c60590470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52646/0 (socket says 172.21.15.42:52646) 2024-05-27T10:18:30.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.615+0000 7f846ea00700 1 -- 172.21.15.42:0/1764606674 learned_addr learned my addr 172.21.15.42:0/1764606674 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:30.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.615+0000 7f8477400700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605713c0 0x558c6058ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.615+0000 7f846ea00700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c6058f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f846ea00700 1 -- 172.21.15.42:0/1764606674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605713c0 msgr2=0x558c6058ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f846ea00700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605713c0 0x558c6058ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f846ea00700 1 -- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 msgr2=0x558c6058f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f846ea00700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c6058f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f846ea00700 1 -- 172.21.15.42:0/1764606674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c60594160 con 0x558c605717d0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f846ea00700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c60590470 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f846000cb30 tx=0x7f846000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c252ef341c6418d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f8465600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f8465600700 1 -- 172.21.15.42:0/1764606674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460034040 con 0x558c605717d0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f8465600700 1 -- 172.21.15.42:0/1764606674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8460013820 con 0x558c605717d0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.616+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84600043d0 con 0x558c605717d0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f8465600700 1 -- 172.21.15.42:0/1764606674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8460040050 con 0x558c605717d0 2024-05-27T10:18:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 10 client.15120 fetch_fsmap learned FSMap version 25 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c605f6d00 con 0x558c605717d0 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f8465600700 1 -- 172.21.15.42:0/1764606674 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84600043d0 con 0x558c605717d0 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f8465600700 1 -- 172.21.15.42:0/1764606674 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8460043020 con 0x558c605717d0 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 25 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 20 client.15120 populate_metadata read hostname 'smithi042' 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c605f8ef0 0x558c605fa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 4 client.15120 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.617+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "status"}) -- 0x558c6049b520 con 0x558c605f8ef0 2024-05-27T10:18:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.618+0000 7f8477400700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c605f8ef0 0x558c605fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:30.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.619+0000 7f8477400700 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c605f8ef0 0x558c605fa360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fea7b01f6ba5970 server_cookie=b513a711662f067d in_seq=0 out_seq=0 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.619+0000 7f8465600700 10 client.15120 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.619+0000 7f8465600700 1 -- 172.21.15.42:0/1764606674 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558c6049b520 con 0x558c605f8ef0 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f8465600700 10 client.15120 handle_command_reply: tid=0 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 client.15120 shutdown 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 msgr2=0x558c60590470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c60590470 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f846000cb30 tx=0x7f846000ca60 comp rx=0 tx=0).stop 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 shutdown_connections 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558c605f8ef0 0x558c605fa360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c605717d0 0x558c60590470 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c603f2df0 0x558c6058f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 --2- 172.21.15.42:0/1764606674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c605713c0 0x558c6058ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 >> 172.21.15.42:0/1764606674 conn(0x558c60452f70 msgr2=0x558c6057b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f8465600700 0 client.15120 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 shutdown_connections 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 1 -- 172.21.15.42:0/1764606674 wait complete. 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.620+0000 7f848cfb1b80 20 client.15120 trim_cache size 0 max 16384 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34499, 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:18:30.623 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 252.43468441799999, 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 25, 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 81, 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 275.74108545600001 2024-05-27T10:18:30.624 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f845c0b2900 msgr2=0x7f845c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f845c0b2900 0x7f845c0b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84700aa3d0 tx=0x7f84700cd040 comp rx=0 tx=0).stop 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 msgr2=0x7f84781a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f84781a1640 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f847000b580 tx=0x7f847000f510 comp rx=0 tx=0).stop 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 shutdown_connections 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8478109670 0x7f84781a1640 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84781043f0 0x7f847819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f845c0b2900 0x7f845c0b4e10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 --2- 172.21.15.42:0/3678505910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478108d20 0x7f847819d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 >> 172.21.15.42:0/3678505910 conn(0x7f8478079d00 msgr2=0x7f847810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.621+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 shutdown_connections 2024-05-27T10:18:30.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:30.622+0000 7f847ce00700 1 -- 172.21.15.42:0/3678505910 wait complete. 2024-05-27T10:18:31.794 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T10:18:31.804 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T10:18:31.804 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T10:18:31.804 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T10:18:31.804 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-27T10:18:31.805 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T10:18:31.806 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:18:31.806 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-27T10:18:31.886 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:31 smithi042 ceph-mon[31661]: pgmap v516: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:31.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:31 smithi104 ceph-mon[34812]: pgmap v516: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:32.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:31 smithi143 ceph-mon[35545]: pgmap v516: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 -- 172.21.15.42:0/97416701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10f860 msgr2=0x7f29dc10fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 --2- 172.21.15.42:0/97416701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10f860 0x7f29dc10fc70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009900 tx=0x7f29cc02a3f0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 -- 172.21.15.42:0/97416701 shutdown_connections 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 --2- 172.21.15.42:0/97416701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc114560 0x7f29dc116980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 --2- 172.21.15.42:0/97416701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10f860 0x7f29dc10fc70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 --2- 172.21.15.42:0/97416701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f29dc10e620 0x7f29dc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 -- 172.21.15.42:0/97416701 >> 172.21.15.42:0/97416701 conn(0x7f29dc109cc0 msgr2=0x7f29dc10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:32.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 -- 172.21.15.42:0/97416701 shutdown_connections 2024-05-27T10:18:32.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.098+0000 7f29e4200640 1 -- 172.21.15.42:0/97416701 wait complete. 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.099+0000 7f29e4200640 1 Processor -- start 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.099+0000 7f29e4200640 1 -- start start 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e4200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 0x7f29dc1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e4200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f29dc10f860 0x7f29dc1a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e4200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc114560 0x7f29dc1ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e4200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc11a000 con 0x7f29dc10e620 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e4200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc119e80 con 0x7f29dc10f860 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e4200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc11a180 con 0x7f29dc114560 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 0x7f29dc1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 0x7f29dc1a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52654/0 (socket says 172.21.15.42:52654) 2024-05-27T10:18:32.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 -- 172.21.15.42:0/2555997314 learned_addr learned my addr 172.21.15.42:0/2555997314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:32.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e0e00640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f29dc10f860 0x7f29dc1a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:32.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 -- 172.21.15.42:0/2555997314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc114560 msgr2=0x7f29dc1ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:32.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc114560 0x7f29dc1ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 -- 172.21.15.42:0/2555997314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f29dc10f860 msgr2=0x7f29dc1a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:32.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f29dc10f860 0x7f29dc1a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.100+0000 7f29e1800640 1 -- 172.21.15.42:0/2555997314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc1ac7b0 con 0x7f29dc10e620 2024-05-27T10:18:32.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.101+0000 7f29e1800640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 0x7f29dc1a4c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f29d000d740 tx=0x7f29d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:32.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.101+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0017070 con 0x7f29dc10e620 2024-05-27T10:18:32.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.101+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29d000de50 con 0x7f29dc10e620 2024-05-27T10:18:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.101+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc1acaa0 con 0x7f29dc10e620 2024-05-27T10:18:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.101+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0003bb0 con 0x7f29dc10e620 2024-05-27T10:18:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.101+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc1ad040 con 0x7f29dc10e620 2024-05-27T10:18:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.103+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc113370 con 0x7f29dc10e620 2024-05-27T10:18:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.105+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f29d0003d50 con 0x7f29dc10e620 2024-05-27T10:18:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.107+0000 7f29da000640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29c40b2860 0x7f29c40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.107+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f29d00e1ff0 con 0x7f29dc10e620 2024-05-27T10:18:32.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.107+0000 7f29e0e00640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29c40b2860 0x7f29c40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:32.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.108+0000 7f29e0e00640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29c40b2860 0x7f29c40b4d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f29cc02a910 tx=0x7f29cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.109+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29d00a9dc0 con 0x7f29dc10e620 2024-05-27T10:18:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.376+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f29dc0638c0 con 0x7f29dc10e620 2024-05-27T10:18:32.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.637+0000 7f29da000640 1 -- 172.21.15.42:0/2555997314 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f29d00a96c0 con 0x7f29dc10e620 2024-05-27T10:18:32.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.641+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29c40b2860 msgr2=0x7f29c40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:32.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.641+0000 7f29e4200640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29c40b2860 0x7f29c40b4d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f29cc02a910 tx=0x7f29cc000bc0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.641+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 msgr2=0x7f29dc1a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:32.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.641+0000 7f29e4200640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 0x7f29dc1a4c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f29d000d740 tx=0x7f29d000dc00 comp rx=0 tx=0).stop 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 shutdown_connections 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f29c40b2860 0x7f29c40b4d70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc114560 0x7f29dc1ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f29dc10f860 0x7f29dc1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 --2- 172.21.15.42:0/2555997314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29dc10e620 0x7f29dc1a4c80 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f29d000d740 tx=0x7f29d000dc00 comp rx=0 tx=0).stop 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 >> 172.21.15.42:0/2555997314 conn(0x7f29dc109cc0 msgr2=0x7f29dc110540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 shutdown_connections 2024-05-27T10:18:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:32.642+0000 7f29e4200640 1 -- 172.21.15.42:0/2555997314 wait complete. 2024-05-27T10:18:32.684 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-27T10:18:32.713 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-27T10:18:32.713 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-27T10:18:32.713 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-27T10:18:32.713 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T10:18:32.713 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:32.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:32.714 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-27T10:18:32.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:32.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-05-27T10:18:32.848 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-05-27T10:18:32.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:18:32.851 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-27T10:18:32.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T10:18:32.906 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: ignoring --setuser ceph since I am not root 2024-05-27T10:18:32.907 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: ignoring --setgroup ceph since I am not root 2024-05-27T10:18:32.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:32 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2555997314' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T10:18:32.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:32 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2555997314' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T10:18:32.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: ignoring --setuser ceph since I am not root 2024-05-27T10:18:32.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: ignoring --setgroup ceph since I am not root 2024-05-27T10:18:32.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:32 smithi104 ceph-mgr[36186]: -- 172.21.15.104:0/1162601413 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a27821c5a0 con 0x55a27cfa6480 2024-05-27T10:18:33.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:32 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2555997314' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T10:18:33.050 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: ignoring --setuser ceph since I am not root 2024-05-27T10:18:33.050 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: ignoring --setgroup ceph since I am not root 2024-05-27T10:18:33.050 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:32 smithi143 ceph-mgr[37486]: -- 172.21.15.143:0/1204094405 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f57546c5a0 con 0x55f57a200900 2024-05-27T10:18:33.285 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:33.295 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:33.036+0000 7f04f57a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:18:33.305 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:33.048+0000 7f6908550200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:18:33.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:33.058+0000 7f95e5a86200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T10:18:33.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:33.304+0000 7f6908550200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:18:33.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:33.437+0000 7f6908550200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:18:33.607 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:33.342+0000 7f95e5a86200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:18:33.607 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:33.485+0000 7f95e5a86200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:18:33.640 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:33.294+0000 7f04f57a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:18:33.641 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:33.428+0000 7f04f57a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T10:18:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:33 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2555997314' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T10:18:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:33 smithi042 ceph-mon[31661]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-27T10:18:33.945 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:33.828+0000 7f04f57a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:18:33.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:33 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2555997314' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T10:18:33.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:33 smithi104 ceph-mon[34812]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-27T10:18:33.952 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:33.836+0000 7f6908550200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:18:33.953 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:33 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2555997314' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T10:18:33.953 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:33 smithi143 ceph-mon[35545]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-27T10:18:34.198 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:33.943+0000 7f04f57a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:18:34.307 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:33.951+0000 7f6908550200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:18:34.311 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:33.910+0000 7f95e5a86200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T10:18:34.311 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:34.032+0000 7f95e5a86200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T10:18:35.807 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:35.483+0000 7f6908550200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:18:35.807 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:35.710+0000 7f6908550200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:18:35.836 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:35.694+0000 7f95e5a86200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:18:35.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 -- 172.21.15.42:0/3176422410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8075d20 msgr2=0x7f0df8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 --2- 172.21.15.42:0/3176422410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8075d20 0x7f0df8076130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00b0a0 tx=0x7f0dec00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 -- 172.21.15.42:0/3176422410 shutdown_connections 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 --2- 172.21.15.42:0/3176422410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0df8110b90 0x7f0df8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 --2- 172.21.15.42:0/3176422410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8075d20 0x7f0df8076130 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 --2- 172.21.15.42:0/3176422410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0df8076670 0x7f0df8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 -- 172.21.15.42:0/3176422410 >> 172.21.15.42:0/3176422410 conn(0x7f0df8100030 msgr2=0x7f0df8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 -- 172.21.15.42:0/3176422410 shutdown_connections 2024-05-27T10:18:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.925+0000 7f0dffc00700 1 -- 172.21.15.42:0/3176422410 wait complete. 2024-05-27T10:18:35.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 Processor -- start 2024-05-27T10:18:35.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 -- start start 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0df8075d20 0x7f0df819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 0x7f0df819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0df8110b90 0x7f0df81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8116580 con 0x7f0df8076670 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8116400 con 0x7f0df8110b90 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0dffc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8116700 con 0x7f0df8075d20 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.927+0000 7f0df7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 0x7f0df819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 0x7f0df819d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36736/0 (socket says 172.21.15.42:36736) 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 -- 172.21.15.42:0/212033457 learned_addr learned my addr 172.21.15.42:0/212033457 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0dfd200700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0df8075d20 0x7f0df819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0dfdc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0df8110b90 0x7f0df81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:35.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 -- 172.21.15.42:0/212033457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0df8075d20 msgr2=0x7f0df819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:35.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0df8075d20 0x7f0df819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:35.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 -- 172.21.15.42:0/212033457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0df8110b90 msgr2=0x7f0df81a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:35.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0df8110b90 0x7f0df81a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 -- 172.21.15.42:0/212033457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df81a1e20 con 0x7f0df8076670 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.928+0000 7f0df7e00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 0x7f0df819d4f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0de400d950 tx=0x7f0de400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.929+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4016070 con 0x7f0df8076670 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.929+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0de4009dd0 con 0x7f0df8076670 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.929+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df81a20b0 con 0x7f0df8076670 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.929+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de400f3c0 con 0x7f0df8076670 2024-05-27T10:18:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.929+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df81a9a30 con 0x7f0df8076670 2024-05-27T10:18:35.931 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:35.479+0000 7f04f57a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T10:18:35.932 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:35.711+0000 7f04f57a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:18:35.932 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:35.930+0000 7f04f57a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:18:35.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.930+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df80512b0 con 0x7f0df8076670 2024-05-27T10:18:35.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.933+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f0de4019070 con 0x7f0df8076670 2024-05-27T10:18:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.934+0000 7f0df5600700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 0x7f0de00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.934+0000 7f0dfd200700 1 -- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 msgr2=0x7f0de00b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2062230330 2024-05-27T10:18:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.934+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0de40e1e60 con 0x7f0df8076670 2024-05-27T10:18:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.934+0000 7f0dfd200700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 0x7f0de00b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:18:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:35.937+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0de40a9b50 con 0x7f0df8076670 2024-05-27T10:18:36.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:35.946+0000 7f95e5a86200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T10:18:36.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.136+0000 7f0dfd200700 1 -- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 msgr2=0x7f0de00b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2062230330 2024-05-27T10:18:36.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.136+0000 7f0dfd200700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 0x7f0de00b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T10:18:36.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.221+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0df802e5e0 con 0x7f0df8076670 2024-05-27T10:18:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.222+0000 7f0df5600700 1 -- 172.21.15.42:0/212033457 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f0de40a9450 con 0x7f0df8076670 2024-05-27T10:18:36.223 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T10:18:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 msgr2=0x7f0de00b4e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:18:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 0x7f0de00b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 msgr2=0x7f0df819d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 0x7f0df819d4f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0de400d950 tx=0x7f0de400dc60 comp rx=0 tx=0).stop 2024-05-27T10:18:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 shutdown_connections 2024-05-27T10:18:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f0de00b2950 0x7f0de00b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0df8110b90 0x7f0df81a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0df8075d20 0x7f0df819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 --2- 172.21.15.42:0/212033457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df8076670 0x7f0df819d4f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 >> 172.21.15.42:0/212033457 conn(0x7f0df8100030 msgr2=0x7f0df81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 shutdown_connections 2024-05-27T10:18:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:36.225+0000 7f0dffc00700 1 -- 172.21.15.42:0/212033457 wait complete. 2024-05-27T10:18:36.276 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:35.928+0000 7f6908550200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:18:36.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:36 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/212033457' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T10:18:36.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:36.317+0000 7f6908550200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:18:36.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:36.443+0000 7f6908550200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:18:36.582 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:36.173+0000 7f95e5a86200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T10:18:36.582 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:36 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/212033457' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T10:18:36.697 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:36.316+0000 7f04f57a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:18:36.697 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:36.440+0000 7f04f57a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:18:36.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:36 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/212033457' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T10:18:36.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:36.610+0000 7f95e5a86200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T10:18:36.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:36.745+0000 7f95e5a86200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T10:18:36.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:36.695+0000 7f04f57a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:18:36.960 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:36.700+0000 7f6908550200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:18:36.960 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:36.958+0000 7f6908550200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:18:37.277 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:37.013+0000 7f95e5a86200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T10:18:37.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T10:18:37.281 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T10:18:37.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:37.281 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-27T10:18:37.323 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T10:18:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T10:18:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T10:18:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-27T10:18:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-27T10:18:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41573sec preferred_lft 41573sec 2024-05-27T10:18:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-05-27T10:18:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-27T10:18:37.325 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T10:18:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-27T10:18:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-27T10:18:37.325 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T10:18:37.326 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-27T10:18:37.380 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:36.965+0000 7f04f57a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:18:37.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T10:18:37.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:37.295+0000 7f95e5a86200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T10:18:37.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T10:18:37.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:37.851 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T10:18:37.919 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-27T10:18:37.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:37.923 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-27T10:18:37.973 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-27T10:18:37.973 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T10:18:37.974 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-27T10:18:37.974 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-27T10:18:37.974 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T10:18:37.974 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T10:18:37.974 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-27T10:18:37.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-27T10:18:37.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-27T10:18:37.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-27T10:18:37.975 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-27T10:18:38.049 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:37.755+0000 7f6908550200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:18:38.049 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:38.047+0000 7f6908550200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:18:38.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T10:18:38.098 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:37.795+0000 7f04f57a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:18:38.098 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:38.097+0000 7f04f57a3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:18:38.363 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:38.158+0000 7f95e5a86200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T10:18:38.757 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:38.476+0000 7f95e5a86200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T10:18:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T10:18:38.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:38.775 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-27T10:18:38.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:38.830 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-05-27T10:18:38.882 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T10:18:38.882 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-27T10:18:38.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:38.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T10:18:38.883 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-27T10:18:38.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:38.666+0000 7f04f57a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:18:38.948 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:38.795+0000 7f04f57a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:18:38.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T10:18:39.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T10:18:39.056 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T10:18:39.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T10:18:39.057 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-27T10:18:39.057 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:38.610+0000 7f6908550200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:18:39.057 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:38.739+0000 7f6908550200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:18:39.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T10:18:39.263 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:39.113+0000 7f95e5a86200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T10:18:39.264 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:39.258+0000 7f95e5a86200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T10:18:39.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T10:18:39.358 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T10:18:39.358 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-27T10:18:39.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-27T10:18:39.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T10:18:39.359 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-27T10:18:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T10:18:39.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T10:18:39.557 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:39.220+0000 7f6908550200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:18:39.566 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-27T10:18:39.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T10:18:39.566 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:39.606 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T10:18:39.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T10:18:39.607 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:39.698 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:39.272+0000 7f04f57a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:18:39.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-27T10:18:39.769 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T10:18:39.817 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-27T10:18:39.817 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-27T10:18:39.817 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-27T10:18:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c12c44e1539439b003fae886e8f7ad6ced9c10dd60dc117bb717a109edb068f1/merged 2024-05-27T10:18:39.819 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5f6fadbcf5c40667ee5a9ddfd28fab82c96d2b88e4ce4201d8167aa5bed4901e/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/07c8dbcb64731d2fd12d8c9aabfdb09aa8136defadf94100a0519a15ee169572/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4468bdb621a2c2645cf874d7d3c9836a2b46b4bbcccb959c72525d3a63644354/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0dda29587a3f3be2dba61f3c7dc6669aabc7ae979b9ca7715806296d5be90219/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ad0569f59742d20ad3ffea394332084bc6376aab6acb42f10ec488f853f1635d/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ddbbbe2093479466dd780eaf7592f870c31e92768509bb33ca8860c96b8f7e4c/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/45779bf3e4db981d57b3864a08b7670c9fbd0c5d4065846bc92ac3a8d48a0d6f/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/76db39b7ffc4ca582cb3f084a03a56c6b3b9d8ff3d6c03de3ec98480ef05a398/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/af65b4e131a7b36de19d7ddada960d2658e42507841ca586cc224c4871da21b3/merged 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:39.820 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:39.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:39.821 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-27T10:18:39.913 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T10:18:39.914 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63 --client_fs=cephfs) 2024-05-27T10:18:39.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-27T10:18:39.980 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T10:18:40.020 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-27T10:18:40.020 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-27T10:18:40.020 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-27T10:18:40.020 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-27T10:18:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-05-27T10:18:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c12c44e1539439b003fae886e8f7ad6ced9c10dd60dc117bb717a109edb068f1/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5f6fadbcf5c40667ee5a9ddfd28fab82c96d2b88e4ce4201d8167aa5bed4901e/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/07c8dbcb64731d2fd12d8c9aabfdb09aa8136defadf94100a0519a15ee169572/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4468bdb621a2c2645cf874d7d3c9836a2b46b4bbcccb959c72525d3a63644354/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0dda29587a3f3be2dba61f3c7dc6669aabc7ae979b9ca7715806296d5be90219/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ad0569f59742d20ad3ffea394332084bc6376aab6acb42f10ec488f853f1635d/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ddbbbe2093479466dd780eaf7592f870c31e92768509bb33ca8860c96b8f7e4c/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/45779bf3e4db981d57b3864a08b7670c9fbd0c5d4065846bc92ac3a8d48a0d6f/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/76db39b7ffc4ca582cb3f084a03a56c6b3b9d8ff3d6c03de3ec98480ef05a398/merged 2024-05-27T10:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/af65b4e131a7b36de19d7ddada960d2658e42507841ca586cc224c4871da21b3/merged 2024-05-27T10:18:40.024 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-27T10:18:40.024 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:40.024 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:40.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:40.025 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-27T10:18:40.057 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:39.682+0000 7f6908550200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:18:40.057 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:39.810+0000 7f6908550200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:18:40.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:39.764+0000 7f95e5a86200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T10:18:40.181 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:39.727+0000 7f04f57a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:18:40.181 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:39.851+0000 7f04f57a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:18:40.181 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:40.179+0000 7f04f57a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:18:40.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-27T10:18:40.207+0000 7fccea20c480 -1 init, newargv = 0x7fccd4004df0 newargc=15 2024-05-27T10:18:40.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[88898]: starting ceph client 2024-05-27T10:18:40.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[88898]: starting fuse 2024-05-27T10:18:40.448 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:40.367+0000 7f04f57a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:18:40.458 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:40.140+0000 7f6908550200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:18:40.459 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:40.325+0000 7f6908550200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:18:40.459 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:40.457+0000 7f6908550200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:18:40.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:40.285+0000 7f95e5a86200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T10:18:40.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:40.422+0000 7f95e5a86200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T10:18:40.903 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:40.499+0000 7f04f57a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:18:41.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:40.766+0000 7f95e5a86200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T10:18:41.094 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:40.958+0000 7f95e5a86200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T10:18:41.197 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:40.901+0000 7f04f57a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:18:41.198 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:41.029+0000 7f04f57a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:18:41.211 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-27T10:18:41.296 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T10:18:41.307 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:40.862+0000 7f6908550200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:18:41.307 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:40.987+0000 7f6908550200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:18:41.349 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-27T10:18:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T10:18:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c12c44e1539439b003fae886e8f7ad6ced9c10dd60dc117bb717a109edb068f1/merged 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5f6fadbcf5c40667ee5a9ddfd28fab82c96d2b88e4ce4201d8167aa5bed4901e/merged 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/07c8dbcb64731d2fd12d8c9aabfdb09aa8136defadf94100a0519a15ee169572/merged 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4468bdb621a2c2645cf874d7d3c9836a2b46b4bbcccb959c72525d3a63644354/merged 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0dda29587a3f3be2dba61f3c7dc6669aabc7ae979b9ca7715806296d5be90219/merged 2024-05-27T10:18:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ad0569f59742d20ad3ffea394332084bc6376aab6acb42f10ec488f853f1635d/merged 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ddbbbe2093479466dd780eaf7592f870c31e92768509bb33ca8860c96b8f7e4c/merged 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/45779bf3e4db981d57b3864a08b7670c9fbd0c5d4065846bc92ac3a8d48a0d6f/merged 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/76db39b7ffc4ca582cb3f084a03a56c6b3b9d8ff3d6c03de3ec98480ef05a398/merged 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/af65b4e131a7b36de19d7ddada960d2658e42507841ca586cc224c4871da21b3/merged 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:18:41.353 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:41.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:41.354 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-27T10:18:41.404 INFO:teuthology.orchestra.run.smithi042.stdout:119 2024-05-27T10:18:41.404 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-27T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T10:18:41.406 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-27T10:18:41.407 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-27T10:18:41.408 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-27T10:18:41.472 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:41.095+0000 7f95e5a86200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T10:18:41.568 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.88898.asok 2024-05-27T10:18:41.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T10:18:41.585 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.88898.asok 2024-05-27T10:18:41.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:41.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.88898.asok status 2024-05-27T10:18:41.843 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:41.523+0000 7f95e5a86200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T10:18:41.844 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:41.660+0000 7f95e5a86200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T10:18:41.875 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "a6507388dd5057528934822c0163b0c347ef1d5d", 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)", 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "88898", 2024-05-27T10:18:41.876 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63" 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24925, 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 24925 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1892405527 2024-05-27T10:18:41.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1892405527 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.24925 172.21.15.42:0/1892405527", 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "172.21.15.42:0/1892405527", 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 5, 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 25, 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 81, 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-27T10:18:41.878 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-27T10:18:41.879 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-27T10:18:41.879 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:18:41.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:41.898 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-27T10:18:41.898 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:41.950 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-27T10:18:41.950 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:41.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:18:41.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T10:18:42.026 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T10:18:42.036 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T10:18:42.037 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:18:42.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-27T10:18:42.055 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:41.759+0000 7f6908550200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:18:42.093 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:41.800+0000 7f04f57a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:18:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.301+0000 7f45ad800640 1 -- 172.21.15.42:0/2795395617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a8075a00 msgr2=0x7f45a810b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.301+0000 7f45ad800640 1 --2- 172.21.15.42:0/2795395617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a8075a00 0x7f45a810b680 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4590009900 tx=0x7f459002a410 comp rx=0 tx=0).stop 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 -- 172.21.15.42:0/2795395617 shutdown_connections 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 --2- 172.21.15.42:0/2795395617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a810bbc0 0x7f45a810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 --2- 172.21.15.42:0/2795395617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a8075a00 0x7f45a810b680 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 --2- 172.21.15.42:0/2795395617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45a80750b0 0x7f45a80754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 -- 172.21.15.42:0/2795395617 >> 172.21.15.42:0/2795395617 conn(0x7f45a80fee40 msgr2=0x7f45a81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 -- 172.21.15.42:0/2795395617 shutdown_connections 2024-05-27T10:18:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.302+0000 7f45ad800640 1 -- 172.21.15.42:0/2795395617 wait complete. 2024-05-27T10:18:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 Processor -- start 2024-05-27T10:18:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 -- start start 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a80750b0 0x7f45a819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45a8075a00 0x7f45a819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 0x7f45a81a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111880 con 0x7f45a810bbc0 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111700 con 0x7f45a80750b0 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.303+0000 7f45ad800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111a00 con 0x7f45a8075a00 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 0x7f45a81a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 0x7f45a81a3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36754/0 (socket says 172.21.15.42:36754) 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 -- 172.21.15.42:0/670523361 learned_addr learned my addr 172.21.15.42:0/670523361 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a6a00640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a80750b0 0x7f45a819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f459e000640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45a8075a00 0x7f45a819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 -- 172.21.15.42:0/670523361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45a8075a00 msgr2=0x7f45a819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45a8075a00 0x7f45a819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 -- 172.21.15.42:0/670523361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a80750b0 msgr2=0x7f45a819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a80750b0 0x7f45a819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 -- 172.21.15.42:0/670523361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81a3fc0 con 0x7f45a810bbc0 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.304+0000 7f45a7400640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 0x7f45a81a3950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45a000e9d0 tx=0x7f45a000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.305+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a000cd80 con 0x7f45a810bbc0 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.305+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45a0005020 con 0x7f45a810bbc0 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.305+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a00116d0 con 0x7f45a810bbc0 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.305+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a81a42b0 con 0x7f45a810bbc0 2024-05-27T10:18:42.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.305+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a4800 con 0x7f45a810bbc0 2024-05-27T10:18:42.308 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:18:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z[37482]: 2024-05-27T10:18:42.054+0000 7f6908550200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:18:42.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.306+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a810a490 con 0x7f45a810bbc0 2024-05-27T10:18:42.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: Standby manager daemon z restarted 2024-05-27T10:18:42.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: Standby manager daemon z started 2024-05-27T10:18:42.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T10:18:42.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:18:42.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T10:18:42.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:18:42.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: Standby manager daemon y restarted 2024-05-27T10:18:42.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: Standby manager daemon y started 2024-05-27T10:18:42.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:42 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T10:18:42.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.309+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f45a0004a40 con 0x7f45a810bbc0 2024-05-27T10:18:42.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.310+0000 7f459fe00640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 0x7f458c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:42.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.310+0000 7f45a6a00640 1 -- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 msgr2=0x7f458c0b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2062230330 2024-05-27T10:18:42.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.310+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f45a00e2880 con 0x7f45a810bbc0 2024-05-27T10:18:42.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.310+0000 7f45a6a00640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 0x7f458c0b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T10:18:42.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.313+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45a0016030 con 0x7f45a810bbc0 2024-05-27T10:18:42.448 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:18:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y[36182]: 2024-05-27T10:18:42.092+0000 7f04f57a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:18:42.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: Standby manager daemon z restarted 2024-05-27T10:18:42.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: Standby manager daemon z started 2024-05-27T10:18:42.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T10:18:42.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:18:42.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T10:18:42.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:18:42.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: Standby manager daemon y restarted 2024-05-27T10:18:42.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: Standby manager daemon y started 2024-05-27T10:18:42.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:42 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T10:18:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.511+0000 7f45a6a00640 1 -- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 msgr2=0x7f458c0b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2062230330 2024-05-27T10:18:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.511+0000 7f45a6a00640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 0x7f458c0b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T10:18:42.523 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: Standby manager daemon z restarted 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: Standby manager daemon z started 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.143:0/43792860' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: Standby manager daemon y restarted 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: Standby manager daemon y started 2024-05-27T10:18:42.524 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:42 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T10:18:42.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.567+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f45a802dc20 con 0x7f45a810bbc0 2024-05-27T10:18:42.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:42.522+0000 7f95e5a86200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T10:18:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.872+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f45a0004d30 con 0x7f45a810bbc0 2024-05-27T10:18:42.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.873+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 msgr2=0x7f458c0b4dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T10:18:42.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:42.873+0000 7f459fe00640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 0x7f458c0b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.093 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:42.830+0000 7f95e5a86200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T10:18:43.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.165+0000 7f459fe00640 1 -- 172.21.15.42:0/670523361 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f45a00d8e60 con 0x7f45a810bbc0 2024-05-27T10:18:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 msgr2=0x7f45a81a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 0x7f45a81a3950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45a000e9d0 tx=0x7f45a000ee90 comp rx=0 tx=0).stop 2024-05-27T10:18:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 shutdown_connections 2024-05-27T10:18:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:6800/2062230330,v1:172.21.15.42:6801/2062230330] conn(0x7f458c0b28b0 0x7f458c0b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45a810bbc0 0x7f45a81a3950 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45a8075a00 0x7f45a819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 --2- 172.21.15.42:0/670523361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a80750b0 0x7f45a819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.169+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 >> 172.21.15.42:0/670523361 conn(0x7f45a80fee40 msgr2=0x7f45a8107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.170+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 shutdown_connections 2024-05-27T10:18:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.170+0000 7f45ad800640 1 -- 172.21.15.42:0/670523361 wait complete. 2024-05-27T10:18:43.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/670523361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: Active manager daemon x restarted 2024-05-27T10:18:43.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: Activating manager daemon x 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: mgrmap e21: x(active, starting, since 0.0378968s), standbys: z, y 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T10:18:43.221 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T10:18:43.222 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:18:43.223 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: Manager daemon x is now available 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T10:18:43.224 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:43 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/670523361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: Active manager daemon x restarted 2024-05-27T10:18:43.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: Activating manager daemon x 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: mgrmap e21: x(active, starting, since 0.0378968s), standbys: z, y 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T10:18:43.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:18:43.320 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: Manager daemon x is now available 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:18:43.321 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T10:18:43.322 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:43 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T10:18:43.423 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T10:18:43.423 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.? 172.21.15.104:0/1070865427' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/670523361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: Active manager daemon x restarted 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: Activating manager daemon x 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: mgrmap e21: x(active, starting, since 0.0378968s), standbys: z, y 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T10:18:43.424 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:18:43.425 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T10:18:43.426 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T10:18:43.427 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T10:18:43.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: Manager daemon x is now available 2024-05-27T10:18:43.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:18:43.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T10:18:43.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:43 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.485+0000 7f5d55a00640 1 -- 172.21.15.42:0/1104603357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d50075a00 msgr2=0x7f5d5010fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.485+0000 7f5d55a00640 1 --2- 172.21.15.42:0/1104603357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d50075a00 0x7f5d5010fe90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009900 tx=0x7f5d3c02a410 comp rx=0 tx=0).stop 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 -- 172.21.15.42:0/1104603357 shutdown_connections 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 --2- 172.21.15.42:0/1104603357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d501103d0 0x7f5d501127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 --2- 172.21.15.42:0/1104603357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d50075a00 0x7f5d5010fe90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 --2- 172.21.15.42:0/1104603357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d500754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 -- 172.21.15.42:0/1104603357 >> 172.21.15.42:0/1104603357 conn(0x7f5d500fee40 msgr2=0x7f5d50101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 -- 172.21.15.42:0/1104603357 shutdown_connections 2024-05-27T10:18:43.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.486+0000 7f5d55a00640 1 -- 172.21.15.42:0/1104603357 wait complete. 2024-05-27T10:18:43.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.487+0000 7f5d55a00640 1 Processor -- start 2024-05-27T10:18:43.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 -- start start 2024-05-27T10:18:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d501a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d50075a00 0x7f5d501a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d501103d0 0x7f5d501a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50115cb0 con 0x7f5d501103d0 2024-05-27T10:18:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50115b30 con 0x7f5d500750b0 2024-05-27T10:18:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d55a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50115e30 con 0x7f5d50075a00 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.488+0000 7f5d4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d501a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d501a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:43606/0 (socket says 172.21.15.42:43606) 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 -- 172.21.15.42:0/278357863 learned_addr learned my addr 172.21.15.42:0/278357863 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4e000640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d50075a00 0x7f5d501a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4f400640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d501103d0 0x7f5d501a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 -- 172.21.15.42:0/278357863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d50075a00 msgr2=0x7f5d501a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d50075a00 0x7f5d501a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 -- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d501103d0 msgr2=0x7f5d501a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d501103d0 0x7f5d501a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.489+0000 7f5d4ea00640 1 -- 172.21.15.42:0/278357863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d501a83c0 con 0x7f5d500750b0 2024-05-27T10:18:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.490+0000 7f5d4ea00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d501a0880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000d770 tx=0x7f5d4000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.490+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40017070 con 0x7f5d500750b0 2024-05-27T10:18:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.490+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d501a86b0 con 0x7f5d500750b0 2024-05-27T10:18:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.490+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501a8c20 con 0x7f5d500750b0 2024-05-27T10:18:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.490+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d40009c30 con 0x7f5d500750b0 2024-05-27T10:18:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.490+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000f620 con 0x7f5d500750b0 2024-05-27T10:18:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.492+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d300052c0 con 0x7f5d500750b0 2024-05-27T10:18:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.495+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f5d4001a070 con 0x7f5d500750b0 2024-05-27T10:18:43.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.497+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5d400e1770 con 0x7f5d500750b0 2024-05-27T10:18:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.499+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d40009de0 con 0x7f5d500750b0 2024-05-27T10:18:43.844 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.461+0000 7f953aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.461+0000 7f953aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.461+0000 7f953aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.461+0000 7f953aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.461+0000 7f953aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.463+0000 7f9535a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.463+0000 7f9535a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.463+0000 7f9535a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.463+0000 7f9535a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.845 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.463+0000 7f9535a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.893+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f5d400a4a80 con 0x7f5d500750b0 2024-05-27T10:18:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.893+0000 7f5d47400640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d340b9460 0x7f5d340bb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.894+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5d40013e50 con 0x7f5d340b9460 2024-05-27T10:18:43.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.900+0000 7f5d4e000640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d340b9460 0x7f5d340bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.907+0000 7f5d4e000640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d340b9460 0x7f5d340bb880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005760 tx=0x7f5d3c031400 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:43.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.942+0000 7f5d47400640 1 -- 172.21.15.42:0/278357863 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5d40013e50 con 0x7f5d340b9460 2024-05-27T10:18:43.943 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-05-27T10:18:43.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d340b9460 msgr2=0x7f5d340bb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:43.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d340b9460 0x7f5d340bb880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005760 tx=0x7f5d3c031400 comp rx=0 tx=0).stop 2024-05-27T10:18:43.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 msgr2=0x7f5d501a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d501a0880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000d770 tx=0x7f5d4000dc30 comp rx=0 tx=0).stop 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 shutdown_connections 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d340b9460 0x7f5d340bb880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d501103d0 0x7f5d501a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d50075a00 0x7f5d501a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 --2- 172.21.15.42:0/278357863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d500750b0 0x7f5d501a0880 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 >> 172.21.15.42:0/278357863 conn(0x7f5d500fee40 msgr2=0x7f5d50103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 shutdown_connections 2024-05-27T10:18:43.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:43.945+0000 7f5d55a00640 1 -- 172.21.15.42:0/278357863 wait complete. 2024-05-27T10:18:43.972 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-27T10:18:43.985 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-27T10:18:44.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 -- 172.21.15.42:0/2864353104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f667410bd70 msgr2=0x7f667410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 --2- 172.21.15.42:0/2864353104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f667410bd70 0x7f667410e190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ca0 tx=0x7f666c02a8c0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 -- 172.21.15.42:0/2864353104 shutdown_connections 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 --2- 172.21.15.42:0/2864353104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f667410bd70 0x7f667410e190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 --2- 172.21.15.42:0/2864353104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6674107070 0x7f6674107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 --2- 172.21.15.42:0/2864353104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6674105e30 0x7f6674106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 -- 172.21.15.42:0/2864353104 >> 172.21.15.42:0/2864353104 conn(0x7f6674101040 msgr2=0x7f66741034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 -- 172.21.15.42:0/2864353104 shutdown_connections 2024-05-27T10:18:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.278+0000 7f6679400640 1 -- 172.21.15.42:0/2864353104 wait complete. 2024-05-27T10:18:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.279+0000 7f6679400640 1 Processor -- start 2024-05-27T10:18:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.279+0000 7f6679400640 1 -- start start 2024-05-27T10:18:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6679400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6674105e30 0x7f667419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6679400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6674107070 0x7f667419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6679400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 0x7f66741a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6679400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111860 con 0x7f6674107070 2024-05-27T10:18:44.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6679400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66741116e0 con 0x7f667410bd70 2024-05-27T10:18:44.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6679400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66741119e0 con 0x7f6674105e30 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6673200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 0x7f66741a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f666be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6674107070 0x7f667419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6673200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 0x7f66741a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:43626/0 (socket says 172.21.15.42:43626) 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6673200640 1 -- 172.21.15.42:0/12546092 learned_addr learned my addr 172.21.15.42:0/12546092 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.280+0000 7f6672800640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6674105e30 0x7f667419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f6673200640 1 -- 172.21.15.42:0/12546092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6674105e30 msgr2=0x7f667419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f6673200640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6674105e30 0x7f667419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f6673200640 1 -- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6674107070 msgr2=0x7f667419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f6673200640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6674107070 0x7f667419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f6673200640 1 -- 172.21.15.42:0/12546092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66741a3f80 con 0x7f667410bd70 2024-05-27T10:18:44.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f6673200640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 0x7f66741a3910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f666c02aeb0 tx=0x7f666c003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.281+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c003d60 con 0x7f667410bd70 2024-05-27T10:18:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.282+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f666c0044d0 con 0x7f667410bd70 2024-05-27T10:18:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.282+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66741a4210 con 0x7f667410bd70 2024-05-27T10:18:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.282+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c030b50 con 0x7f667410bd70 2024-05-27T10:18:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.282+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741a47b0 con 0x7f667410bd70 2024-05-27T10:18:44.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.283+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667410ab80 con 0x7f667410bd70 2024-05-27T10:18:44.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.285+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f666c004680 con 0x7f667410bd70 2024-05-27T10:18:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.288+0000 7f666b400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66580b2660 0x7f66580b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.288+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f666c1034e0 con 0x7f667410bd70 2024-05-27T10:18:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.288+0000 7f6672800640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66580b2660 0x7f66580b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.289+0000 7f6672800640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66580b2660 0x7f66580b4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6664005fd0 tx=0x7f666400b390 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:44.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.293+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f666c0cb3e0 con 0x7f667410bd70 2024-05-27T10:18:44.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.918+0000 7f9533200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:44.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.918+0000 7f9533200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:44.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.918+0000 7f9533200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:44.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.918+0000 7f9533200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:44.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:18:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:18:43.918+0000 7f9533200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:18:44.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:44 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/670523361' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T10:18:44.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:44 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:18:44.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:44 smithi042 ceph-mon[31661]: mgrmap e22: x(active, since 1.06209s), standbys: z, y 2024-05-27T10:18:44.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:44 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/670523361' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T10:18:44.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:44 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:18:44.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:44 smithi104 ceph-mon[34812]: mgrmap e22: x(active, since 1.06209s), standbys: z, y 2024-05-27T10:18:44.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.512+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66740638c0 con 0x7f667410bd70 2024-05-27T10:18:44.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.514+0000 7f666b400640 1 -- 172.21.15.42:0/12546092 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22940 (secure 0 0 0) 0x7f666c0cace0 con 0x7f667410bd70 2024-05-27T10:18:44.516 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:44.516 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":82,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:18:42.838286+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1953650070":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2667703980":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2970531755":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/1310078254":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:18:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66580b2660 msgr2=0x7f66580b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66580b2660 0x7f66580b4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6664005fd0 tx=0x7f666400b390 comp rx=0 tx=0).stop 2024-05-27T10:18:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 msgr2=0x7f66741a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 0x7f66741a3910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f666c02aeb0 tx=0x7f666c003970 comp rx=0 tx=0).stop 2024-05-27T10:18:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 shutdown_connections 2024-05-27T10:18:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66580b2660 0x7f66580b4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667410bd70 0x7f66741a3910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6674107070 0x7f667419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 --2- 172.21.15.42:0/12546092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6674105e30 0x7f667419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 >> 172.21.15.42:0/12546092 conn(0x7f6674101040 msgr2=0x7f6674107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 shutdown_connections 2024-05-27T10:18:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.517+0000 7f6679400640 1 -- 172.21.15.42:0/12546092 wait complete. 2024-05-27T10:18:44.540 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T10:07:06.385341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T10:14:07.735206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T10:14:12.919933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T10:18:44.541 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T10:18:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:44 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/670523361' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T10:18:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:44 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:18:44.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:44 smithi143 ceph-mon[35545]: mgrmap e22: x(active, since 1.06209s), standbys: z, y 2024-05-27T10:18:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 -- 172.21.15.42:0/3846017559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 msgr2=0x7f89f810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 --2- 172.21.15.42:0/3846017559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f810bbf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009900 tx=0x7f89f002a410 comp rx=0 tx=0).stop 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 -- 172.21.15.42:0/3846017559 shutdown_connections 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 --2- 172.21.15.42:0/3846017559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f810bbf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 --2- 172.21.15.42:0/3846017559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f8106e70 0x7f89f8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 --2- 172.21.15.42:0/3846017559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 0x7f89f8106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.802+0000 7f89fe000640 1 -- 172.21.15.42:0/3846017559 >> 172.21.15.42:0/3846017559 conn(0x7f89f80fdb10 msgr2=0x7f89f80fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.803+0000 7f89fe000640 1 -- 172.21.15.42:0/3846017559 shutdown_connections 2024-05-27T10:18:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.803+0000 7f89fe000640 1 -- 172.21.15.42:0/3846017559 wait complete. 2024-05-27T10:18:44.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 Processor -- start 2024-05-27T10:18:44.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 -- start start 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 0x7f89f819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f8106e70 0x7f89f819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f81a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89f810f400 con 0x7f89f81097d0 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89f810f280 con 0x7f89f8104510 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fe000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f89f810f580 con 0x7f89f8106e70 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f81a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f81a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53284/0 (socket says 172.21.15.42:53284) 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.804+0000 7f89fd600640 1 -- 172.21.15.42:0/1698890724 learned_addr learned my addr 172.21.15.42:0/1698890724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89efe00640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f8106e70 0x7f89f819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fcc00640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 0x7f89f819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fd600640 1 -- 172.21.15.42:0/1698890724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f8106e70 msgr2=0x7f89f819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fd600640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f8106e70 0x7f89f819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fd600640 1 -- 172.21.15.42:0/1698890724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 msgr2=0x7f89f819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:44.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fd600640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 0x7f89f819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:44.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fd600640 1 -- 172.21.15.42:0/1698890724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f81a3dd0 con 0x7f89f81097d0 2024-05-27T10:18:44.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fcc00640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 0x7f89f819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89fd600640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f81a3760 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f89f0005ab0 tx=0x7f89f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f003f030 con 0x7f89f81097d0 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.805+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89f0003c90 con 0x7f89f81097d0 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.806+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f81a4060 con 0x7f89f81097d0 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.806+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0035070 con 0x7f89f81097d0 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.806+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f81a45a0 con 0x7f89f81097d0 2024-05-27T10:18:44.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.807+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f810b2f0 con 0x7f89f81097d0 2024-05-27T10:18:44.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.809+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f89f0042030 con 0x7f89f81097d0 2024-05-27T10:18:44.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.810+0000 7f89f6000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89dc0b2660 0x7f89dc0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:44.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.811+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f89f01025e0 con 0x7f89f81097d0 2024-05-27T10:18:44.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.811+0000 7f89fcc00640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89dc0b2660 0x7f89dc0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:44.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.812+0000 7f89fcc00640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89dc0b2660 0x7f89dc0b4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89f806a9d0 tx=0x7f89e80093e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:44.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:44.816+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89f00ca4e0 con 0x7f89f81097d0 2024-05-27T10:18:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.021+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89f80638c0 con 0x7f89f81097d0 2024-05-27T10:18:45.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.022+0000 7f89f6000640 1 -- 172.21.15.42:0/1698890724 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f89f00c9de0 con 0x7f89f81097d0 2024-05-27T10:18:45.023 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-27T10:18:45.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.024+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89dc0b2660 msgr2=0x7f89dc0b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.024+0000 7f89fe000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89dc0b2660 0x7f89dc0b4b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89f806a9d0 tx=0x7f89e80093e0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.024+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 msgr2=0x7f89f81a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.024+0000 7f89fe000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f81a3760 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f89f0005ab0 tx=0x7f89f00042f0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 shutdown_connections 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89dc0b2660 0x7f89dc0b4b70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89f81097d0 0x7f89f81a3760 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f8106e70 0x7f89f819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 --2- 172.21.15.42:0/1698890724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f8104510 0x7f89f819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 >> 172.21.15.42:0/1698890724 conn(0x7f89f80fdb10 msgr2=0x7f89f8103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 shutdown_connections 2024-05-27T10:18:45.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.025+0000 7f89fe000640 1 -- 172.21.15.42:0/1698890724 wait complete. 2024-05-27T10:18:45.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T10:18:45.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 -- 172.21.15.42:0/1033981755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 msgr2=0x7fcb9c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 --2- 172.21.15.42:0/1033981755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c0754c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009900 tx=0x7fcb9002a3f0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 -- 172.21.15.42:0/1033981755 shutdown_connections 2024-05-27T10:18:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 --2- 172.21.15.42:0/1033981755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c10b9a0 0x7fcb9c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 --2- 172.21.15.42:0/1033981755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb9c075a00 0x7fcb9c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 --2- 172.21.15.42:0/1033981755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c0754c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 -- 172.21.15.42:0/1033981755 >> 172.21.15.42:0/1033981755 conn(0x7fcb9c0fec40 msgr2=0x7fcb9c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 -- 172.21.15.42:0/1033981755 shutdown_connections 2024-05-27T10:18:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.317+0000 7fcba3000640 1 -- 172.21.15.42:0/1033981755 wait complete. 2024-05-27T10:18:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.318+0000 7fcba3000640 1 Processor -- start 2024-05-27T10:18:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 -- start start 2024-05-27T10:18:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb9c075a00 0x7fcb9c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c10b9a0 0x7fcb9c1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c111610 con 0x7fcb9c0750b0 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c111490 con 0x7fcb9c075a00 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba3000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c111790 con 0x7fcb9c10b9a0 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53320/0 (socket says 172.21.15.42:53320) 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.319+0000 7fcba1c00640 1 -- 172.21.15.42:0/3906063095 learned_addr learned my addr 172.21.15.42:0/3906063095 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1200640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb9c075a00 0x7fcb9c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1c00640 1 -- 172.21.15.42:0/3906063095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c10b9a0 msgr2=0x7fcb9c1a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1c00640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c10b9a0 0x7fcb9c1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1c00640 1 -- 172.21.15.42:0/3906063095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb9c075a00 msgr2=0x7fcb9c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1c00640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb9c075a00 0x7fcb9c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1c00640 1 -- 172.21.15.42:0/3906063095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1a3e20 con 0x7fcb9c0750b0 2024-05-27T10:18:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba1c00640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c19c2e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcb90031460 tx=0x7fcb90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9003f030 con 0x7fcb9c0750b0 2024-05-27T10:18:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb90003a80 con 0x7fcb9c0750b0 2024-05-27T10:18:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1a40b0 con 0x7fcb9c0750b0 2024-05-27T10:18:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.320+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90035070 con 0x7fcb9c0750b0 2024-05-27T10:18:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.321+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1a45a0 con 0x7fcb9c0750b0 2024-05-27T10:18:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.322+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c10ab60 con 0x7fcb9c0750b0 2024-05-27T10:18:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.324+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7fcb90042030 con 0x7fcb9c0750b0 2024-05-27T10:18:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.326+0000 7fcb9a000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcb840b29e0 0x7fcb840b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.326+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcb90103500 con 0x7fcb9c0750b0 2024-05-27T10:18:45.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.326+0000 7fcba1200640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcb840b29e0 0x7fcb840b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.327+0000 7fcba1200640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcb840b29e0 0x7fcb840b4ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005fd0 tx=0x7fcb88005c70 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.330+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb900cad60 con 0x7fcb9c0750b0 2024-05-27T10:18:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:45 smithi042 ceph-mon[31661]: [27/May/2024:10:18:43] ENGINE Bus STARTING 2024-05-27T10:18:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:45 smithi042 ceph-mon[31661]: [27/May/2024:10:18:44] ENGINE Serving on https://172.21.15.42:7150 2024-05-27T10:18:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:45 smithi042 ceph-mon[31661]: [27/May/2024:10:18:44] ENGINE Serving on http://172.21.15.42:8765 2024-05-27T10:18:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:45 smithi042 ceph-mon[31661]: [27/May/2024:10:18:44] ENGINE Bus STARTED 2024-05-27T10:18:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:45 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/12546092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:18:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:45 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1698890724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:18:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:45 smithi143 ceph-mon[35545]: [27/May/2024:10:18:43] ENGINE Bus STARTING 2024-05-27T10:18:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:45 smithi143 ceph-mon[35545]: [27/May/2024:10:18:44] ENGINE Serving on https://172.21.15.42:7150 2024-05-27T10:18:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:45 smithi143 ceph-mon[35545]: [27/May/2024:10:18:44] ENGINE Serving on http://172.21.15.42:8765 2024-05-27T10:18:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:45 smithi143 ceph-mon[35545]: [27/May/2024:10:18:44] ENGINE Bus STARTED 2024-05-27T10:18:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/12546092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:18:45.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:45 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1698890724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:18:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.561+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb9c02ddd0 con 0x7fcb9c0750b0 2024-05-27T10:18:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.561+0000 7fcb9a000640 1 -- 172.21.15.42:0/3906063095 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fcb900ca660 con 0x7fcb9c0750b0 2024-05-27T10:18:45.562 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcb840b29e0 msgr2=0x7fcb840b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcb840b29e0 0x7fcb840b4ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005fd0 tx=0x7fcb88005c70 comp rx=0 tx=0).stop 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 msgr2=0x7fcb9c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c19c2e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcb90031460 tx=0x7fcb90004270 comp rx=0 tx=0).stop 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 shutdown_connections 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcb840b29e0 0x7fcb840b4ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c10b9a0 0x7fcb9c1a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb9c075a00 0x7fcb9c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 --2- 172.21.15.42:0/3906063095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c0750b0 0x7fcb9c19c2e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 >> 172.21.15.42:0/3906063095 conn(0x7fcb9c0fec40 msgr2=0x7fcb9c0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:45.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 shutdown_connections 2024-05-27T10:18:45.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.564+0000 7fcba3000640 1 -- 172.21.15.42:0/3906063095 wait complete. 2024-05-27T10:18:45.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T10:18:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:45 smithi104 ceph-mon[34812]: [27/May/2024:10:18:43] ENGINE Bus STARTING 2024-05-27T10:18:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:45 smithi104 ceph-mon[34812]: [27/May/2024:10:18:44] ENGINE Serving on https://172.21.15.42:7150 2024-05-27T10:18:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:45 smithi104 ceph-mon[34812]: [27/May/2024:10:18:44] ENGINE Serving on http://172.21.15.42:8765 2024-05-27T10:18:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:45 smithi104 ceph-mon[34812]: [27/May/2024:10:18:44] ENGINE Bus STARTED 2024-05-27T10:18:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:45 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/12546092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:18:45.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:45 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1698890724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 -- 172.21.15.42:0/649774590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076140 msgr2=0x7f27f8076550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 --2- 172.21.15.42:0/649774590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076140 0x7f27f8076550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009900 tx=0x7f27e802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 -- 172.21.15.42:0/649774590 shutdown_connections 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 --2- 172.21.15.42:0/649774590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27f810bc00 0x7f27f810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 --2- 172.21.15.42:0/649774590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076a90 0x7f27f810b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 --2- 172.21.15.42:0/649774590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076140 0x7f27f8076550 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 -- 172.21.15.42:0/649774590 >> 172.21.15.42:0/649774590 conn(0x7f27f80fee00 msgr2=0x7f27f81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 -- 172.21.15.42:0/649774590 shutdown_connections 2024-05-27T10:18:45.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.844+0000 7f27fec00640 1 -- 172.21.15.42:0/649774590 wait complete. 2024-05-27T10:18:45.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 Processor -- start 2024-05-27T10:18:45.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 -- start start 2024-05-27T10:18:45.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076140 0x7f27f819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 0x7f27f819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27f810bc00 0x7f27f81a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8111800 con 0x7f27f8076a90 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8111680 con 0x7f27f8076140 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.845+0000 7f27fec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8111980 con 0x7f27f810bc00 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 0x7f27f819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076140 0x7f27f819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 0x7f27f819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53330/0 (socket says 172.21.15.42:53330) 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7400640 1 -- 172.21.15.42:0/1709791256 learned_addr learned my addr 172.21.15.42:0/1709791256 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27fcc00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27f810bc00 0x7f27f81a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7e00640 1 -- 172.21.15.42:0/1709791256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27f810bc00 msgr2=0x7f27f81a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7e00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27f810bc00 0x7f27f81a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7e00640 1 -- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 msgr2=0x7f27f819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7e00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 0x7f27f819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7e00640 1 -- 172.21.15.42:0/1709791256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f81a3fb0 con 0x7f27f8076140 2024-05-27T10:18:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.846+0000 7f27f7400640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 0x7f27f819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:18:45.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.847+0000 7f27f7e00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076140 0x7f27f819c470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009230 tx=0x7f27e8003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.847+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e803f070 con 0x7f27f8076140 2024-05-27T10:18:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.847+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27e8031da0 con 0x7f27f8076140 2024-05-27T10:18:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.847+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f81a4240 con 0x7f27f8076140 2024-05-27T10:18:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.847+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8035030 con 0x7f27f8076140 2024-05-27T10:18:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.847+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f81a4700 con 0x7f27f8076140 2024-05-27T10:18:45.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.849+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f810adc0 con 0x7f27f8076140 2024-05-27T10:18:45.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.851+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7f27e8042070 con 0x7f27f8076140 2024-05-27T10:18:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.853+0000 7f27f4c00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27dc0b29f0 0x7f27dc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.853+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f27e81035d0 con 0x7f27f8076140 2024-05-27T10:18:45.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.853+0000 7f27f7400640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27dc0b29f0 0x7f27dc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:45.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.854+0000 7f27f7400640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27dc0b29f0 0x7f27dc0b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e400b480 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:45.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:45.856+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27e80ca7e0 con 0x7f27f8076140 2024-05-27T10:18:46.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.070+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f27f802ddb0 con 0x7f27f8076140 2024-05-27T10:18:46.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.071+0000 7f27f4c00640 1 -- 172.21.15.42:0/1709791256 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f27f802ddb0 con 0x7f27f8076140 2024-05-27T10:18:46.072 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27dc0b29f0 msgr2=0x7f27dc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27dc0b29f0 0x7f27dc0b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e400b480 comp rx=0 tx=0).stop 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076140 msgr2=0x7f27f819c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076140 0x7f27f819c470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009230 tx=0x7f27e8003c80 comp rx=0 tx=0).stop 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 shutdown_connections 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27dc0b29f0 0x7f27dc0b4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27f810bc00 0x7f27f81a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f8076a90 0x7f27f819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 --2- 172.21.15.42:0/1709791256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27f8076140 0x7f27f819c470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 >> 172.21.15.42:0/1709791256 conn(0x7f27f80fee00 msgr2=0x7f27f80ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 shutdown_connections 2024-05-27T10:18:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:46.074+0000 7f27fec00640 1 -- 172.21.15.42:0/1709791256 wait complete. 2024-05-27T10:18:46.102 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-27T10:18:46.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:18:46.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:46 smithi143 ceph-mon[35545]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:46.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:46 smithi143 ceph-mon[35545]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T10:18:46.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:46 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3906063095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T10:18:46.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:46 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1709791256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T10:18:46.557 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:46.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:46 smithi042 ceph-mon[31661]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:46.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:46 smithi042 ceph-mon[31661]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T10:18:46.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:46 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3906063095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T10:18:46.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:46 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1709791256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T10:18:46.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:46 smithi104 ceph-mon[34812]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:46 smithi104 ceph-mon[34812]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T10:18:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:46 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3906063095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T10:18:46.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:46 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1709791256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T10:18:47.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:47 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:47 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:47 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:47 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:47 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:47 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:47 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:47 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:47 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:47 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:47 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:47 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:47 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:47 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:47 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:47 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:47 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:47.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:47 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:48 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:48 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:48.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:48 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 -- 172.21.15.42:0/2777461539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e410e8f0 msgr2=0x7fb2e4110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:49.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 --2- 172.21.15.42:0/2777461539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e410e8f0 0x7fb2e4110db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00b0a0 tx=0x7fb2dc00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 -- 172.21.15.42:0/2777461539 shutdown_connections 2024-05-27T10:18:49.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 --2- 172.21.15.42:0/2777461539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e410e8f0 0x7fb2e4110db0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 --2- 172.21.15.42:0/2777461539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e4105510 0x7fb2e4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 --2- 172.21.15.42:0/2777461539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2e4105e60 0x7fb2e410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 -- 172.21.15.42:0/2777461539 >> 172.21.15.42:0/2777461539 conn(0x7fb2e40fedb0 msgr2=0x7fb2e41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 -- 172.21.15.42:0/2777461539 shutdown_connections 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.157+0000 7fb2eb400700 1 -- 172.21.15.42:0/2777461539 wait complete. 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 Processor -- start 2024-05-27T10:18:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 -- start start 2024-05-27T10:18:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 0x7fb2e4103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2e4105e60 0x7fb2e4103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e410e8f0 0x7fb2e4103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4114330 con 0x7fb2e4105510 2024-05-27T10:18:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41141b0 con 0x7fb2e4105e60 2024-05-27T10:18:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.159+0000 7fb2eb400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41144b0 con 0x7fb2e410e8f0 2024-05-27T10:18:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 0x7fb2e4103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e3e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2e4105e60 0x7fb2e4103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 0x7fb2e4103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53348/0 (socket says 172.21.15.42:53348) 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 -- 172.21.15.42:0/3326875663 learned_addr learned my addr 172.21.15.42:0/3326875663 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e9400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e410e8f0 0x7fb2e4103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 -- 172.21.15.42:0/3326875663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e410e8f0 msgr2=0x7fb2e4103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e410e8f0 0x7fb2e4103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 -- 172.21.15.42:0/3326875663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2e4105e60 msgr2=0x7fb2e4103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2e4105e60 0x7fb2e4103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.160+0000 7fb2e8a00700 1 -- 172.21.15.42:0/3326875663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e41a9040 con 0x7fb2e4105510 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.161+0000 7fb2e8a00700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 0x7fb2e4103200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800d950 tx=0x7fb2d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.161+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8016070 con 0x7fb2e4105510 2024-05-27T10:18:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.161+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2d800fd20 con 0x7fb2e4105510 2024-05-27T10:18:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.161+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800e3b0 con 0x7fb2e4105510 2024-05-27T10:18:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.161+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e41a92d0 con 0x7fb2e4105510 2024-05-27T10:18:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.161+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a9810 con 0x7fb2e4105510 2024-05-27T10:18:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.163+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e40512b0 con 0x7fb2e4105510 2024-05-27T10:18:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.165+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fb2d8019070 con 0x7fb2e4105510 2024-05-27T10:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.167+0000 7fb2e1600700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb2cc0b2b10 0x7fb2cc0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.167+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb2d80e23a0 con 0x7fb2e4105510 2024-05-27T10:18:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.167+0000 7fb2e3e00700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb2cc0b2b10 0x7fb2cc0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.168+0000 7fb2e3e00700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb2cc0b2b10 0x7fb2cc0b5020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2e41023d0 tx=0x7fb2d00073d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:49.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.172+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2d80a9b20 con 0x7fb2e4105510 2024-05-27T10:18:49.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.548+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2e402e7a0 con 0x7fb2e4105510 2024-05-27T10:18:49.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.549+0000 7fb2e1600700 1 -- 172.21.15.42:0/3326875663 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fb2d80a9420 con 0x7fb2e4105510 2024-05-27T10:18:49.551 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:49.551 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb2cc0b2b10 msgr2=0x7fb2cc0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb2cc0b2b10 0x7fb2cc0b5020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2e41023d0 tx=0x7fb2d00073d0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 msgr2=0x7fb2e4103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 0x7fb2e4103200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800d950 tx=0x7fb2d800dc60 comp rx=0 tx=0).stop 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 shutdown_connections 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb2cc0b2b10 0x7fb2cc0b5020 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e410e8f0 0x7fb2e4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4105510 0x7fb2e4103200 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.552+0000 7fb2eb400700 1 --2- 172.21.15.42:0/3326875663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2e4105e60 0x7fb2e4103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.553+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 >> 172.21.15.42:0/3326875663 conn(0x7fb2e40fedb0 msgr2=0x7fb2e40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.553+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 shutdown_connections 2024-05-27T10:18:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:49.553+0000 7fb2eb400700 1 -- 172.21.15.42:0/3326875663 wait complete. 2024-05-27T10:18:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:18:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi104 to 176.4M 2024-05-27T10:18:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: Unable to set osd_memory_target on smithi104 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:18:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-05-27T10:18:49.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:18:49.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:49 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:18:49.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi104 to 176.4M 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: Unable to set osd_memory_target on smithi104 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:18:49.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:49 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:18:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi104 to 176.4M 2024-05-27T10:18:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: Unable to set osd_memory_target on smithi104 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:18:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-05-27T10:18:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:18:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:49.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:49.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:18:49.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:18:49.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:49 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:18:50.614 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-27T10:18:50.614 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-27T10:18:50.614 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T10:18:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-05-27T10:18:50.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3326875663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:50 smithi143 ceph-mon[35545]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:50.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-05-27T10:18:50.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T10:18:50.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:18:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:18:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:18:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3326875663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:50 smithi042 ceph-mon[31661]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1717275248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc109940 msgr2=0x7fe2bc10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1717275248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc109940 0x7fe2bc10bd60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009ca0 tx=0x7fe2b402a8c0 comp rx=0 tx=0).stop 2024-05-27T10:18:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1717275248 shutdown_connections 2024-05-27T10:18:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1717275248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc109940 0x7fe2bc10bd60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1717275248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc104c40 0x7fe2bc105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1717275248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2bc103a00 0x7fe2bc104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1717275248 >> 172.21.15.42:0/1717275248 conn(0x7fe2bc0febd0 msgr2=0x7fe2bc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1717275248 shutdown_connections 2024-05-27T10:18:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.872+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1717275248 wait complete. 2024-05-27T10:18:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.873+0000 7fe2c0a00640 1 Processor -- start 2024-05-27T10:18:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 -- start start 2024-05-27T10:18:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 0x7fe2bc19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 0x7fe2bc19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2bc109940 0x7fe2bc1a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc10f410 con 0x7fe2bc104c40 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc10f290 con 0x7fe2bc109940 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2c0a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc10f590 con 0x7fe2bc103a00 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2baa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 0x7fe2bc19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.874+0000 7fe2bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 0x7fe2bc19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 0x7fe2bc19c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:43668/0 (socket says 172.21.15.42:43668) 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2bb400640 1 -- 172.21.15.42:0/1087379035 learned_addr learned my addr 172.21.15.42:0/1087379035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2baa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 0x7fe2bc19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53366/0 (socket says 172.21.15.42:53366) 2024-05-27T10:18:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2bbe00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2bc109940 0x7fe2bc1a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2baa00640 1 -- 172.21.15.42:0/1087379035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 msgr2=0x7fe2bc19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2baa00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 0x7fe2bc19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2baa00640 1 -- 172.21.15.42:0/1087379035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2bc109940 msgr2=0x7fe2bc1a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2baa00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2bc109940 0x7fe2bc1a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2baa00640 1 -- 172.21.15.42:0/1087379035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc1a3d80 con 0x7fe2bc104c40 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.875+0000 7fe2bb400640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 0x7fe2bc19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.876+0000 7fe2baa00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 0x7fe2bc19c780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e9f0 tx=0x7fe2a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.876+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800ccc0 con 0x7fe2bc104c40 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.876+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2a80108c0 con 0x7fe2bc104c40 2024-05-27T10:18:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.876+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc1a4070 con 0x7fe2bc104c40 2024-05-27T10:18:50.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.876+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1a4610 con 0x7fe2bc104c40 2024-05-27T10:18:50.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.876+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80187e0 con 0x7fe2bc104c40 2024-05-27T10:18:50.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.878+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c0052c0 con 0x7fe2bc104c40 2024-05-27T10:18:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.881+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fe2a8010430 con 0x7fe2bc104c40 2024-05-27T10:18:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.883+0000 7fe2b3e00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe2a00b2af0 0x7fe2a00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.883+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe2a80e2cf0 con 0x7fe2bc104c40 2024-05-27T10:18:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.884+0000 7fe2bb400640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe2a00b2af0 0x7fe2a00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:50.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.885+0000 7fe2bb400640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe2a00b2af0 0x7fe2a00b5000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009fd0 tx=0x7fe2ac005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:50.885+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2a80a9e20 con 0x7fe2bc104c40 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Updating smithi042:/etc/ceph/ceph.conf 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3326875663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:50.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:50 smithi104 ceph-mon[34812]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T10:18:51.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.132+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe29c000bf0 con 0x7fe2a00b2af0 2024-05-27T10:18:51.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.159+0000 7fe2b3e00640 1 -- 172.21.15.42:0/1087379035 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258021 (secure 0 0 0) 0x7fe29c000bf0 con 0x7fe2a00b2af0 2024-05-27T10:18:51.164 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:51.165 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:18:51.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.165+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe2a00b2af0 msgr2=0x7fe2a00b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.165+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe2a00b2af0 0x7fe2a00b5000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009fd0 tx=0x7fe2ac005f90 comp rx=0 tx=0).stop 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.165+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 msgr2=0x7fe2bc19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.165+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 0x7fe2bc19c780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e9f0 tx=0x7fe2a800eeb0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 shutdown_connections 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe2a00b2af0 0x7fe2a00b5000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2bc109940 0x7fe2bc1a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2bc104c40 0x7fe2bc19c780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 --2- 172.21.15.42:0/1087379035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc103a00 0x7fe2bc19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 >> 172.21.15.42:0/1087379035 conn(0x7fe2bc0febd0 msgr2=0x7fe2bc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:51.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 shutdown_connections 2024-05-27T10:18:51.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.166+0000 7fe2c0a00640 1 -- 172.21.15.42:0/1087379035 wait complete. 2024-05-27T10:18:51.196 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T10:18:50.894144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190047,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1757,"num_read_kb":1167,"num_write":442,"num_write_kb":3599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360232,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581592,"statfs":{"total":1151940427776,"available":1151571550208,"internally_reserved":0,"allocated":11317248,"data_stored":8806965,"data_compressed":219078,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3302,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":97,"num_read_kb":206,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.977203"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866630+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:18:42.866630+0000","last_peered":"2024-05-27T10:18:42.866630+0000","last_clean":"2024-05-27T10:18:42.866630+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:18:42.866630+0000","last_undegraded":"2024-05-27T10:18:42.866630+0000","last_fullsized":"2024-05-27T10:18:42.866630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035777+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:18:43.035777+0000","last_peered":"2024-05-27T10:18:43.035777+0000","last_clean":"2024-05-27T10:18:43.035777+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:18:43.035777+0000","last_undegraded":"2024-05-27T10:18:43.035777+0000","last_fullsized":"2024-05-27T10:18:43.035777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036368+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:18:43.036368+0000","last_peered":"2024-05-27T10:18:43.036368+0000","last_clean":"2024-05-27T10:18:43.036368+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:18:43.036368+0000","last_undegraded":"2024-05-27T10:18:43.036368+0000","last_fullsized":"2024-05-27T10:18:43.036368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011005+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:18:43.011005+0000","last_peered":"2024-05-27T10:18:43.011005+0000","last_clean":"2024-05-27T10:18:43.011005+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:18:43.011005+0000","last_undegraded":"2024-05-27T10:18:43.011005+0000","last_fullsized":"2024-05-27T10:18:43.011005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149389+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:18:43.149389+0000","last_peered":"2024-05-27T10:18:43.149389+0000","last_clean":"2024-05-27T10:18:43.149389+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:18:43.149389+0000","last_undegraded":"2024-05-27T10:18:43.149389+0000","last_fullsized":"2024-05-27T10:18:43.149389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864338+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:18:42.864338+0000","last_peered":"2024-05-27T10:18:42.864338+0000","last_clean":"2024-05-27T10:18:42.864338+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:18:42.864338+0000","last_undegraded":"2024-05-27T10:18:42.864338+0000","last_fullsized":"2024-05-27T10:18:42.864338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914557+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:18:42.914557+0000","last_peered":"2024-05-27T10:18:42.914557+0000","last_clean":"2024-05-27T10:18:42.914557+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:18:42.914557+0000","last_undegraded":"2024-05-27T10:18:42.914557+0000","last_fullsized":"2024-05-27T10:18:42.914557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013138+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:18:43.013138+0000","last_peered":"2024-05-27T10:18:43.013138+0000","last_clean":"2024-05-27T10:18:43.013138+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:18:43.013138+0000","last_undegraded":"2024-05-27T10:18:43.013138+0000","last_fullsized":"2024-05-27T10:18:43.013138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913466+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:18:42.913466+0000","last_peered":"2024-05-27T10:18:42.913466+0000","last_clean":"2024-05-27T10:18:42.913466+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:18:42.913466+0000","last_undegraded":"2024-05-27T10:18:42.913466+0000","last_fullsized":"2024-05-27T10:18:42.913466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864950+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:18:42.864950+0000","last_peered":"2024-05-27T10:18:42.864950+0000","last_clean":"2024-05-27T10:18:42.864950+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:18:42.864950+0000","last_undegraded":"2024-05-27T10:18:42.864950+0000","last_fullsized":"2024-05-27T10:18:42.864950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915378+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:18:42.915378+0000","last_peered":"2024-05-27T10:18:42.915378+0000","last_clean":"2024-05-27T10:18:42.915378+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:18:42.915378+0000","last_undegraded":"2024-05-27T10:18:42.915378+0000","last_fullsized":"2024-05-27T10:18:42.915378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036117+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:18:43.036117+0000","last_peered":"2024-05-27T10:18:43.036117+0000","last_clean":"2024-05-27T10:18:43.036117+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:18:43.036117+0000","last_undegraded":"2024-05-27T10:18:43.036117+0000","last_fullsized":"2024-05-27T10:18:43.036117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858317+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:18:42.858317+0000","last_peered":"2024-05-27T10:18:42.858317+0000","last_clean":"2024-05-27T10:18:42.858317+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:18:42.858317+0000","last_undegraded":"2024-05-27T10:18:42.858317+0000","last_fullsized":"2024-05-27T10:18:42.858317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035965+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:18:43.035965+0000","last_peered":"2024-05-27T10:18:43.035965+0000","last_clean":"2024-05-27T10:18:43.035965+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:18:43.035965+0000","last_undegraded":"2024-05-27T10:18:43.035965+0000","last_fullsized":"2024-05-27T10:18:43.035965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035665+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:18:43.035665+0000","last_peered":"2024-05-27T10:18:43.035665+0000","last_clean":"2024-05-27T10:18:43.035665+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:18:43.035665+0000","last_undegraded":"2024-05-27T10:18:43.035665+0000","last_fullsized":"2024-05-27T10:18:43.035665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910287+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:18:42.910287+0000","last_peered":"2024-05-27T10:18:42.910287+0000","last_clean":"2024-05-27T10:18:42.910287+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:18:42.910287+0000","last_undegraded":"2024-05-27T10:18:42.910287+0000","last_fullsized":"2024-05-27T10:18:42.910287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.867088+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:18:42.867088+0000","last_peered":"2024-05-27T10:18:42.867088+0000","last_clean":"2024-05-27T10:18:42.867088+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:18:42.867088+0000","last_undegraded":"2024-05-27T10:18:42.867088+0000","last_fullsized":"2024-05-27T10:18:42.867088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011077+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:18:43.011077+0000","last_peered":"2024-05-27T10:18:43.011077+0000","last_clean":"2024-05-27T10:18:43.011077+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:18:43.011077+0000","last_undegraded":"2024-05-27T10:18:43.011077+0000","last_fullsized":"2024-05-27T10:18:43.011077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892271+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:18:42.892271+0000","last_peered":"2024-05-27T10:18:42.892271+0000","last_clean":"2024-05-27T10:18:42.892271+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:18:42.892271+0000","last_undegraded":"2024-05-27T10:18:42.892271+0000","last_fullsized":"2024-05-27T10:18:42.892271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010950+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:18:43.010950+0000","last_peered":"2024-05-27T10:18:43.010950+0000","last_clean":"2024-05-27T10:18:43.010950+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:18:43.010950+0000","last_undegraded":"2024-05-27T10:18:43.010950+0000","last_fullsized":"2024-05-27T10:18:43.010950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013844+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:18:43.013844+0000","last_peered":"2024-05-27T10:18:43.013844+0000","last_clean":"2024-05-27T10:18:43.013844+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:18:43.013844+0000","last_undegraded":"2024-05-27T10:18:43.013844+0000","last_fullsized":"2024-05-27T10:18:43.013844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013898+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:18:43.013898+0000","last_peered":"2024-05-27T10:18:43.013898+0000","last_clean":"2024-05-27T10:18:43.013898+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:18:43.013898+0000","last_undegraded":"2024-05-27T10:18:43.013898+0000","last_fullsized":"2024-05-27T10:18:43.013898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036523+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:18:43.036523+0000","last_peered":"2024-05-27T10:18:43.036523+0000","last_clean":"2024-05-27T10:18:43.036523+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:18:43.036523+0000","last_undegraded":"2024-05-27T10:18:43.036523+0000","last_fullsized":"2024-05-27T10:18:43.036523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150318+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:18:43.150318+0000","last_peered":"2024-05-27T10:18:43.150318+0000","last_clean":"2024-05-27T10:18:43.150318+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:18:43.150318+0000","last_undegraded":"2024-05-27T10:18:43.150318+0000","last_fullsized":"2024-05-27T10:18:43.150318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912126+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:18:42.912126+0000","last_peered":"2024-05-27T10:18:42.912126+0000","last_clean":"2024-05-27T10:18:42.912126+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:18:42.912126+0000","last_undegraded":"2024-05-27T10:18:42.912126+0000","last_fullsized":"2024-05-27T10:18:42.912126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150197+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:18:43.150197+0000","last_peered":"2024-05-27T10:18:43.150197+0000","last_clean":"2024-05-27T10:18:43.150197+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:18:43.150197+0000","last_undegraded":"2024-05-27T10:18:43.150197+0000","last_fullsized":"2024-05-27T10:18:43.150197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029108+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:18:43.029108+0000","last_peered":"2024-05-27T10:18:43.029108+0000","last_clean":"2024-05-27T10:18:43.029108+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:18:43.029108+0000","last_undegraded":"2024-05-27T10:18:43.029108+0000","last_fullsized":"2024-05-27T10:18:43.029108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'11","reported_seq":293,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.446077+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:18:47.446077+0000","last_peered":"2024-05-27T10:18:47.446077+0000","last_clean":"2024-05-27T10:18:47.446077+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:18:47.446077+0000","last_undegraded":"2024-05-27T10:18:47.446077+0000","last_fullsized":"2024-05-27T10:18:47.446077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":253,"num_read_kb":7,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149127+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:18:43.149127+0000","last_peered":"2024-05-27T10:18:43.149127+0000","last_clean":"2024-05-27T10:18:43.149127+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:18:43.149127+0000","last_undegraded":"2024-05-27T10:18:43.149127+0000","last_fullsized":"2024-05-27T10:18:43.149127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864000+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:18:42.864000+0000","last_peered":"2024-05-27T10:18:42.864000+0000","last_clean":"2024-05-27T10:18:42.864000+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:18:42.864000+0000","last_undegraded":"2024-05-27T10:18:42.864000+0000","last_fullsized":"2024-05-27T10:18:42.864000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028881+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:18:43.028881+0000","last_peered":"2024-05-27T10:18:43.028881+0000","last_clean":"2024-05-27T10:18:43.028881+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:18:43.028881+0000","last_undegraded":"2024-05-27T10:18:43.028881+0000","last_fullsized":"2024-05-27T10:18:43.028881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857767+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:18:42.857767+0000","last_peered":"2024-05-27T10:18:42.857767+0000","last_clean":"2024-05-27T10:18:42.857767+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:18:42.857767+0000","last_undegraded":"2024-05-27T10:18:42.857767+0000","last_fullsized":"2024-05-27T10:18:42.857767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013048+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:18:43.013048+0000","last_peered":"2024-05-27T10:18:43.013048+0000","last_clean":"2024-05-27T10:18:43.013048+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:18:43.013048+0000","last_undegraded":"2024-05-27T10:18:43.013048+0000","last_fullsized":"2024-05-27T10:18:43.013048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.027680+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:18:43.027680+0000","last_peered":"2024-05-27T10:18:43.027680+0000","last_clean":"2024-05-27T10:18:43.027680+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:18:43.027680+0000","last_undegraded":"2024-05-27T10:18:43.027680+0000","last_fullsized":"2024-05-27T10:18:43.027680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010824+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:18:43.010824+0000","last_peered":"2024-05-27T10:18:43.010824+0000","last_clean":"2024-05-27T10:18:43.010824+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:18:43.010824+0000","last_undegraded":"2024-05-27T10:18:43.010824+0000","last_fullsized":"2024-05-27T10:18:43.010824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":281,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.444884+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:18:47.444884+0000","last_peered":"2024-05-27T10:18:47.444884+0000","last_clean":"2024-05-27T10:18:47.444884+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:18:47.444884+0000","last_undegraded":"2024-05-27T10:18:47.444884+0000","last_fullsized":"2024-05-27T10:18:47.444884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913390+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:18:42.913390+0000","last_peered":"2024-05-27T10:18:42.913390+0000","last_clean":"2024-05-27T10:18:42.913390+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:18:42.913390+0000","last_undegraded":"2024-05-27T10:18:42.913390+0000","last_fullsized":"2024-05-27T10:18:42.913390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915631+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:18:42.915631+0000","last_peered":"2024-05-27T10:18:42.915631+0000","last_clean":"2024-05-27T10:18:42.915631+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:18:42.915631+0000","last_undegraded":"2024-05-27T10:18:42.915631+0000","last_fullsized":"2024-05-27T10:18:42.915631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036560+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:18:43.036560+0000","last_peered":"2024-05-27T10:18:43.036560+0000","last_clean":"2024-05-27T10:18:43.036560+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:18:43.036560+0000","last_undegraded":"2024-05-27T10:18:43.036560+0000","last_fullsized":"2024-05-27T10:18:43.036560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029425+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:18:43.029425+0000","last_peered":"2024-05-27T10:18:43.029425+0000","last_clean":"2024-05-27T10:18:43.029425+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:18:43.029425+0000","last_undegraded":"2024-05-27T10:18:43.029425+0000","last_fullsized":"2024-05-27T10:18:43.029425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912507+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:18:42.912507+0000","last_peered":"2024-05-27T10:18:42.912507+0000","last_clean":"2024-05-27T10:18:42.912507+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:18:42.912507+0000","last_undegraded":"2024-05-27T10:18:42.912507+0000","last_fullsized":"2024-05-27T10:18:42.912507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":573,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:48.399408+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:18:48.399408+0000","last_peered":"2024-05-27T10:18:48.399408+0000","last_clean":"2024-05-27T10:18:48.399408+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:18:48.399408+0000","last_undegraded":"2024-05-27T10:18:48.399408+0000","last_fullsized":"2024-05-27T10:18:48.399408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":507,"num_read_kb":319,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149956+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:18:43.149956+0000","last_peered":"2024-05-27T10:18:43.149956+0000","last_clean":"2024-05-27T10:18:43.149956+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:18:43.149956+0000","last_undegraded":"2024-05-27T10:18:43.149956+0000","last_fullsized":"2024-05-27T10:18:43.149956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892232+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:18:42.892232+0000","last_peered":"2024-05-27T10:18:42.892232+0000","last_clean":"2024-05-27T10:18:42.892232+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:18:42.892232+0000","last_undegraded":"2024-05-27T10:18:42.892232+0000","last_fullsized":"2024-05-27T10:18:42.892232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857993+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:18:42.857993+0000","last_peered":"2024-05-27T10:18:42.857993+0000","last_clean":"2024-05-27T10:18:42.857993+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:18:42.857993+0000","last_undegraded":"2024-05-27T10:18:42.857993+0000","last_fullsized":"2024-05-27T10:18:42.857993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028152+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:18:43.028152+0000","last_peered":"2024-05-27T10:18:43.028152+0000","last_clean":"2024-05-27T10:18:43.028152+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:18:43.028152+0000","last_undegraded":"2024-05-27T10:18:43.028152+0000","last_fullsized":"2024-05-27T10:18:43.028152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915250+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:18:42.915250+0000","last_peered":"2024-05-27T10:18:42.915250+0000","last_clean":"2024-05-27T10:18:42.915250+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:18:42.915250+0000","last_undegraded":"2024-05-27T10:18:42.915250+0000","last_fullsized":"2024-05-27T10:18:42.915250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":290,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:48.434451+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:18:48.434451+0000","last_peered":"2024-05-27T10:18:48.434451+0000","last_clean":"2024-05-27T10:18:48.434451+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:18:48.434451+0000","last_undegraded":"2024-05-27T10:18:48.434451+0000","last_fullsized":"2024-05-27T10:18:48.434451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914298+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:18:42.914298+0000","last_peered":"2024-05-27T10:18:42.914298+0000","last_clean":"2024-05-27T10:18:42.914298+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:18:42.914298+0000","last_undegraded":"2024-05-27T10:18:42.914298+0000","last_fullsized":"2024-05-27T10:18:42.914298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148905+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:18:43.148905+0000","last_peered":"2024-05-27T10:18:43.148905+0000","last_clean":"2024-05-27T10:18:43.148905+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:18:43.148905+0000","last_undegraded":"2024-05-27T10:18:43.148905+0000","last_fullsized":"2024-05-27T10:18:43.148905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028231+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:18:43.028231+0000","last_peered":"2024-05-27T10:18:43.028231+0000","last_clean":"2024-05-27T10:18:43.028231+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:18:43.028231+0000","last_undegraded":"2024-05-27T10:18:43.028231+0000","last_fullsized":"2024-05-27T10:18:43.028231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.641820+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:18:43.641820+0000","last_peered":"2024-05-27T10:18:43.641820+0000","last_clean":"2024-05-27T10:18:43.641820+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:18:43.641820+0000","last_undegraded":"2024-05-27T10:18:43.641820+0000","last_fullsized":"2024-05-27T10:18:43.641820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865929+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:18:42.865929+0000","last_peered":"2024-05-27T10:18:42.865929+0000","last_clean":"2024-05-27T10:18:42.865929+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:18:42.865929+0000","last_undegraded":"2024-05-27T10:18:42.865929+0000","last_fullsized":"2024-05-27T10:18:42.865929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010814+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:18:43.010814+0000","last_peered":"2024-05-27T10:18:43.010814+0000","last_clean":"2024-05-27T10:18:43.010814+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:18:43.010814+0000","last_undegraded":"2024-05-27T10:18:43.010814+0000","last_fullsized":"2024-05-27T10:18:43.010814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864668+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:18:42.864668+0000","last_peered":"2024-05-27T10:18:42.864668+0000","last_clean":"2024-05-27T10:18:42.864668+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:18:42.864668+0000","last_undegraded":"2024-05-27T10:18:42.864668+0000","last_fullsized":"2024-05-27T10:18:42.864668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914915+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:18:42.914915+0000","last_peered":"2024-05-27T10:18:42.914915+0000","last_clean":"2024-05-27T10:18:42.914915+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:18:42.914915+0000","last_undegraded":"2024-05-27T10:18:42.914915+0000","last_fullsized":"2024-05-27T10:18:42.914915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865657+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:18:42.865657+0000","last_peered":"2024-05-27T10:18:42.865657+0000","last_clean":"2024-05-27T10:18:42.865657+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:18:42.865657+0000","last_undegraded":"2024-05-27T10:18:42.865657+0000","last_fullsized":"2024-05-27T10:18:42.865657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028265+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:18:43.028265+0000","last_peered":"2024-05-27T10:18:43.028265+0000","last_clean":"2024-05-27T10:18:43.028265+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:18:43.028265+0000","last_undegraded":"2024-05-27T10:18:43.028265+0000","last_fullsized":"2024-05-27T10:18:43.028265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":283,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.430751+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:18:47.430751+0000","last_peered":"2024-05-27T10:18:47.430751+0000","last_clean":"2024-05-27T10:18:47.430751+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:18:47.430751+0000","last_undegraded":"2024-05-27T10:18:47.430751+0000","last_fullsized":"2024-05-27T10:18:47.430751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013516+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:18:43.013516+0000","last_peered":"2024-05-27T10:18:43.013516+0000","last_clean":"2024-05-27T10:18:43.013516+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:18:43.013516+0000","last_undegraded":"2024-05-27T10:18:43.013516+0000","last_fullsized":"2024-05-27T10:18:43.013516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036179+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:18:43.036179+0000","last_peered":"2024-05-27T10:18:43.036179+0000","last_clean":"2024-05-27T10:18:43.036179+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:18:43.036179+0000","last_undegraded":"2024-05-27T10:18:43.036179+0000","last_fullsized":"2024-05-27T10:18:43.036179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"82'99","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:45.052019+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:18:45.052019+0000","last_peered":"2024-05-27T10:18:45.052019+0000","last_clean":"2024-05-27T10:18:45.052019+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:18:45.052019+0000","last_undegraded":"2024-05-27T10:18:45.052019+0000","last_fullsized":"2024-05-27T10:18:45.052019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857424+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:18:42.857424+0000","last_peered":"2024-05-27T10:18:42.857424+0000","last_clean":"2024-05-27T10:18:42.857424+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:18:42.857424+0000","last_undegraded":"2024-05-27T10:18:42.857424+0000","last_fullsized":"2024-05-27T10:18:42.857424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914885+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:18:42.914885+0000","last_peered":"2024-05-27T10:18:42.914885+0000","last_clean":"2024-05-27T10:18:42.914885+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:18:42.914885+0000","last_undegraded":"2024-05-27T10:18:42.914885+0000","last_fullsized":"2024-05-27T10:18:42.914885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149983+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:18:43.149983+0000","last_peered":"2024-05-27T10:18:43.149983+0000","last_clean":"2024-05-27T10:18:43.149983+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:18:43.149983+0000","last_undegraded":"2024-05-27T10:18:43.149983+0000","last_fullsized":"2024-05-27T10:18:43.149983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858159+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:18:42.858159+0000","last_peered":"2024-05-27T10:18:42.858159+0000","last_clean":"2024-05-27T10:18:42.858159+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:18:42.858159+0000","last_undegraded":"2024-05-27T10:18:42.858159+0000","last_fullsized":"2024-05-27T10:18:42.858159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'49","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.228670+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:18:43.228670+0000","last_peered":"2024-05-27T10:18:43.228670+0000","last_clean":"2024-05-27T10:18:43.228670+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:18:43.228670+0000","last_undegraded":"2024-05-27T10:18:43.228670+0000","last_fullsized":"2024-05-27T10:18:43.228670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.014095+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:18:43.014095+0000","last_peered":"2024-05-27T10:18:43.014095+0000","last_clean":"2024-05-27T10:18:43.014095+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:18:43.014095+0000","last_undegraded":"2024-05-27T10:18:43.014095+0000","last_fullsized":"2024-05-27T10:18:43.014095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036143+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:18:43.036143+0000","last_peered":"2024-05-27T10:18:43.036143+0000","last_clean":"2024-05-27T10:18:43.036143+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:18:43.036143+0000","last_undegraded":"2024-05-27T10:18:43.036143+0000","last_fullsized":"2024-05-27T10:18:43.036143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035987+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:18:43.035987+0000","last_peered":"2024-05-27T10:18:43.035987+0000","last_clean":"2024-05-27T10:18:43.035987+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:18:43.035987+0000","last_undegraded":"2024-05-27T10:18:43.035987+0000","last_fullsized":"2024-05-27T10:18:43.035987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910322+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:18:42.910322+0000","last_peered":"2024-05-27T10:18:42.910322+0000","last_clean":"2024-05-27T10:18:42.910322+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:18:42.910322+0000","last_undegraded":"2024-05-27T10:18:42.910322+0000","last_fullsized":"2024-05-27T10:18:42.910322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148975+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:18:43.148975+0000","last_peered":"2024-05-27T10:18:43.148975+0000","last_clean":"2024-05-27T10:18:43.148975+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:18:43.148975+0000","last_undegraded":"2024-05-27T10:18:43.148975+0000","last_fullsized":"2024-05-27T10:18:43.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013173+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:18:43.013173+0000","last_peered":"2024-05-27T10:18:43.013173+0000","last_clean":"2024-05-27T10:18:43.013173+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:18:43.013173+0000","last_undegraded":"2024-05-27T10:18:43.013173+0000","last_fullsized":"2024-05-27T10:18:43.013173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865624+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:18:42.865624+0000","last_peered":"2024-05-27T10:18:42.865624+0000","last_clean":"2024-05-27T10:18:42.865624+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:18:42.865624+0000","last_undegraded":"2024-05-27T10:18:42.865624+0000","last_fullsized":"2024-05-27T10:18:42.865624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865382+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:18:42.865382+0000","last_peered":"2024-05-27T10:18:42.865382+0000","last_clean":"2024-05-27T10:18:42.865382+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:18:42.865382+0000","last_undegraded":"2024-05-27T10:18:42.865382+0000","last_fullsized":"2024-05-27T10:18:42.865382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013727+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:18:43.013727+0000","last_peered":"2024-05-27T10:18:43.013727+0000","last_clean":"2024-05-27T10:18:43.013727+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:18:43.013727+0000","last_undegraded":"2024-05-27T10:18:43.013727+0000","last_fullsized":"2024-05-27T10:18:43.013727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150814+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:18:43.150814+0000","last_peered":"2024-05-27T10:18:43.150814+0000","last_clean":"2024-05-27T10:18:43.150814+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:18:43.150814+0000","last_undegraded":"2024-05-27T10:18:43.150814+0000","last_fullsized":"2024-05-27T10:18:43.150814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915235+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:18:42.915235+0000","last_peered":"2024-05-27T10:18:42.915235+0000","last_clean":"2024-05-27T10:18:42.915235+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:18:42.915235+0000","last_undegraded":"2024-05-27T10:18:42.915235+0000","last_fullsized":"2024-05-27T10:18:42.915235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892674+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:18:42.892674+0000","last_peered":"2024-05-27T10:18:42.892674+0000","last_clean":"2024-05-27T10:18:42.892674+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:18:42.892674+0000","last_undegraded":"2024-05-27T10:18:42.892674+0000","last_fullsized":"2024-05-27T10:18:42.892674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916054+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:18:42.916054+0000","last_peered":"2024-05-27T10:18:42.916054+0000","last_clean":"2024-05-27T10:18:42.916054+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:18:42.916054+0000","last_undegraded":"2024-05-27T10:18:42.916054+0000","last_fullsized":"2024-05-27T10:18:42.916054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'39","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.226174+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:18:43.226174+0000","last_peered":"2024-05-27T10:18:43.226174+0000","last_clean":"2024-05-27T10:18:43.226174+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:18:43.226174+0000","last_undegraded":"2024-05-27T10:18:43.226174+0000","last_fullsized":"2024-05-27T10:18:43.226174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915511+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:18:42.915511+0000","last_peered":"2024-05-27T10:18:42.915511+0000","last_clean":"2024-05-27T10:18:42.915511+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:18:42.915511+0000","last_undegraded":"2024-05-27T10:18:42.915511+0000","last_fullsized":"2024-05-27T10:18:42.915511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858229+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:18:42.858229+0000","last_peered":"2024-05-27T10:18:42.858229+0000","last_clean":"2024-05-27T10:18:42.858229+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:18:42.858229+0000","last_undegraded":"2024-05-27T10:18:42.858229+0000","last_fullsized":"2024-05-27T10:18:42.858229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916095+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:18:42.916095+0000","last_peered":"2024-05-27T10:18:42.916095+0000","last_clean":"2024-05-27T10:18:42.916095+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:18:42.916095+0000","last_undegraded":"2024-05-27T10:18:42.916095+0000","last_fullsized":"2024-05-27T10:18:42.916095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865960+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:18:42.865960+0000","last_peered":"2024-05-27T10:18:42.865960+0000","last_clean":"2024-05-27T10:18:42.865960+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:18:42.865960+0000","last_undegraded":"2024-05-27T10:18:42.865960+0000","last_fullsized":"2024-05-27T10:18:42.865960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892084+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:18:42.892084+0000","last_peered":"2024-05-27T10:18:42.892084+0000","last_clean":"2024-05-27T10:18:42.892084+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:18:42.892084+0000","last_undegraded":"2024-05-27T10:18:42.892084+0000","last_fullsized":"2024-05-27T10:18:42.892084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.891669+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:18:42.891669+0000","last_peered":"2024-05-27T10:18:42.891669+0000","last_clean":"2024-05-27T10:18:42.891669+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:18:42.891669+0000","last_undegraded":"2024-05-27T10:18:42.891669+0000","last_fullsized":"2024-05-27T10:18:42.891669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036165+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:18:43.036165+0000","last_peered":"2024-05-27T10:18:43.036165+0000","last_clean":"2024-05-27T10:18:43.036165+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:18:43.036165+0000","last_undegraded":"2024-05-27T10:18:43.036165+0000","last_fullsized":"2024-05-27T10:18:43.036165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858813+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:18:42.858813+0000","last_peered":"2024-05-27T10:18:42.858813+0000","last_clean":"2024-05-27T10:18:42.858813+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:18:42.858813+0000","last_undegraded":"2024-05-27T10:18:42.858813+0000","last_fullsized":"2024-05-27T10:18:42.858813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865913+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:18:42.865913+0000","last_peered":"2024-05-27T10:18:42.865913+0000","last_clean":"2024-05-27T10:18:42.865913+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:18:42.865913+0000","last_undegraded":"2024-05-27T10:18:42.865913+0000","last_fullsized":"2024-05-27T10:18:42.865913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892758+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:18:42.892758+0000","last_peered":"2024-05-27T10:18:42.892758+0000","last_clean":"2024-05-27T10:18:42.892758+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:18:42.892758+0000","last_undegraded":"2024-05-27T10:18:42.892758+0000","last_fullsized":"2024-05-27T10:18:42.892758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010907+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:18:43.010907+0000","last_peered":"2024-05-27T10:18:43.010907+0000","last_clean":"2024-05-27T10:18:43.010907+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:18:43.010907+0000","last_undegraded":"2024-05-27T10:18:43.010907+0000","last_fullsized":"2024-05-27T10:18:43.010907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028084+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:18:43.028084+0000","last_peered":"2024-05-27T10:18:43.028084+0000","last_clean":"2024-05-27T10:18:43.028084+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:18:43.028084+0000","last_undegraded":"2024-05-27T10:18:43.028084+0000","last_fullsized":"2024-05-27T10:18:43.028084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013207+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:18:43.013207+0000","last_peered":"2024-05-27T10:18:43.013207+0000","last_clean":"2024-05-27T10:18:43.013207+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:18:43.013207+0000","last_undegraded":"2024-05-27T10:18:43.013207+0000","last_fullsized":"2024-05-27T10:18:43.013207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913669+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:18:42.913669+0000","last_peered":"2024-05-27T10:18:42.913669+0000","last_clean":"2024-05-27T10:18:42.913669+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:18:42.913669+0000","last_undegraded":"2024-05-27T10:18:42.913669+0000","last_fullsized":"2024-05-27T10:18:42.913669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858143+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:18:42.858143+0000","last_peered":"2024-05-27T10:18:42.858143+0000","last_clean":"2024-05-27T10:18:42.858143+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:18:42.858143+0000","last_undegraded":"2024-05-27T10:18:42.858143+0000","last_fullsized":"2024-05-27T10:18:42.858143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866758+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:18:42.866758+0000","last_peered":"2024-05-27T10:18:42.866758+0000","last_clean":"2024-05-27T10:18:42.866758+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:18:42.866758+0000","last_undegraded":"2024-05-27T10:18:42.866758+0000","last_fullsized":"2024-05-27T10:18:42.866758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599445,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1613,"num_read_kb":919,"num_write":305,"num_write_kb":1447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1798335,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":830200,"data_compressed":22039,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874321,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1228886,"data_compressed":45201,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037849,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":830436,"data_compressed":22039,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234081,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":437140,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430311,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1032293,"data_compressed":39045,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593839,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":888832,"data_stored":436609,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":503807,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":437880,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":507355,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149830,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":961736,"data_compressed":28195,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953600,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":438462,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1162161,"data_compressed":45201,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":135274,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:18:51.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T10:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 -- 172.21.15.42:0/4282741204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc114880 msgr2=0x7fbecc116ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 --2- 172.21.15.42:0/4282741204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc114880 0x7fbecc116ca0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009ca0 tx=0x7fbec402a8c0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 -- 172.21.15.42:0/4282741204 shutdown_connections 2024-05-27T10:18:51.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 --2- 172.21.15.42:0/4282741204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc114880 0x7fbecc116ca0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 --2- 172.21.15.42:0/4282741204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbecc111f20 0x7fbecc114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 --2- 172.21.15.42:0/4282741204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbecc077610 0x7fbecc077a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 -- 172.21.15.42:0/4282741204 >> 172.21.15.42:0/4282741204 conn(0x7fbecc1012d0 msgr2=0x7fbecc103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 -- 172.21.15.42:0/4282741204 shutdown_connections 2024-05-27T10:18:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.493+0000 7fbed1600640 1 -- 172.21.15.42:0/4282741204 wait complete. 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.494+0000 7fbed1600640 1 Processor -- start 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.494+0000 7fbed1600640 1 -- start start 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbed1600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbecc077610 0x7fbecc1a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbed1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 0x7fbecc1a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbed1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbecc114880 0x7fbecc1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbed1600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc11a0f0 con 0x7fbecc111f20 2024-05-27T10:18:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbed1600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc119f70 con 0x7fbecc077610 2024-05-27T10:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbed1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc11a270 con 0x7fbecc114880 2024-05-27T10:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbecb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 0x7fbecc1a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbecb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 0x7fbecc1a6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53374/0 (socket says 172.21.15.42:53374) 2024-05-27T10:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbecbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbecc077610 0x7fbecc1a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.495+0000 7fbecb400640 1 -- 172.21.15.42:0/2703898080 learned_addr learned my addr 172.21.15.42:0/2703898080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:51.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbed0c00640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbecc114880 0x7fbecc1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbecb400640 1 -- 172.21.15.42:0/2703898080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbecc114880 msgr2=0x7fbecc1a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbecb400640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbecc114880 0x7fbecc1a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbecb400640 1 -- 172.21.15.42:0/2703898080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbecc077610 msgr2=0x7fbecc1a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbecb400640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbecc077610 0x7fbecc1a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbecb400640 1 -- 172.21.15.42:0/2703898080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc1a0f90 con 0x7fbecc111f20 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.496+0000 7fbecb400640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 0x7fbecc1a6c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8005fd0 tx=0x7fbeb800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:51.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.497+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8017070 con 0x7fbecc111f20 2024-05-27T10:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.497+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeb8009c30 con 0x7fbecc111f20 2024-05-27T10:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.497+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc1a1280 con 0x7fbecc111f20 2024-05-27T10:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.497+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8003c70 con 0x7fbecc111f20 2024-05-27T10:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.497+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc1ad560 con 0x7fbecc111f20 2024-05-27T10:18:51.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.498+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc06a150 con 0x7fbecc111f20 2024-05-27T10:18:51.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.501+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fbeb8003e10 con 0x7fbecc111f20 2024-05-27T10:18:51.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.503+0000 7fbec8c00640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbeb00b2a70 0x7fbeb00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.503+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbeb80e2c30 con 0x7fbecc111f20 2024-05-27T10:18:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.503+0000 7fbecbe00640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbeb00b2a70 0x7fbeb00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.504+0000 7fbecbe00640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbeb00b2a70 0x7fbeb00b4f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbecc1a1a30 tx=0x7fbebc00b390 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:51.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.506+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbeb8020030 con 0x7fbecc111f20 2024-05-27T10:18:51.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.728+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbecc063100 con 0x7fbeb00b2a70 2024-05-27T10:18:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.744+0000 7fbec8c00640 1 -- 172.21.15.42:0/2703898080 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258021 (secure 0 0 0) 0x7fbecc063100 con 0x7fbeb00b2a70 2024-05-27T10:18:51.749 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:18:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbeb00b2a70 msgr2=0x7fbeb00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbeb00b2a70 0x7fbeb00b4f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbecc1a1a30 tx=0x7fbebc00b390 comp rx=0 tx=0).stop 2024-05-27T10:18:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 msgr2=0x7fbecc1a6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 0x7fbecc1a6c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8005fd0 tx=0x7fbeb800dde0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 shutdown_connections 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbeb00b2a70 0x7fbeb00b4f80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbecc114880 0x7fbecc1a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbecc111f20 0x7fbecc1a6c20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 --2- 172.21.15.42:0/2703898080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbecc077610 0x7fbecc1a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 >> 172.21.15.42:0/2703898080 conn(0x7fbecc1012d0 msgr2=0x7fbecc07a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 shutdown_connections 2024-05-27T10:18:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:51.750+0000 7fbed1600640 1 -- 172.21.15.42:0/2703898080 wait complete. 2024-05-27T10:18:51.781 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T10:18:50.894144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190047,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1757,"num_read_kb":1167,"num_write":442,"num_write_kb":3599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360232,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581592,"statfs":{"total":1151940427776,"available":1151571550208,"internally_reserved":0,"allocated":11317248,"data_stored":8806965,"data_compressed":219078,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3302,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":97,"num_read_kb":206,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.977203"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866630+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:18:42.866630+0000","last_peered":"2024-05-27T10:18:42.866630+0000","last_clean":"2024-05-27T10:18:42.866630+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:18:42.866630+0000","last_undegraded":"2024-05-27T10:18:42.866630+0000","last_fullsized":"2024-05-27T10:18:42.866630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035777+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:18:43.035777+0000","last_peered":"2024-05-27T10:18:43.035777+0000","last_clean":"2024-05-27T10:18:43.035777+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:18:43.035777+0000","last_undegraded":"2024-05-27T10:18:43.035777+0000","last_fullsized":"2024-05-27T10:18:43.035777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036368+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:18:43.036368+0000","last_peered":"2024-05-27T10:18:43.036368+0000","last_clean":"2024-05-27T10:18:43.036368+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:18:43.036368+0000","last_undegraded":"2024-05-27T10:18:43.036368+0000","last_fullsized":"2024-05-27T10:18:43.036368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011005+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:18:43.011005+0000","last_peered":"2024-05-27T10:18:43.011005+0000","last_clean":"2024-05-27T10:18:43.011005+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:18:43.011005+0000","last_undegraded":"2024-05-27T10:18:43.011005+0000","last_fullsized":"2024-05-27T10:18:43.011005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149389+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:18:43.149389+0000","last_peered":"2024-05-27T10:18:43.149389+0000","last_clean":"2024-05-27T10:18:43.149389+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:18:43.149389+0000","last_undegraded":"2024-05-27T10:18:43.149389+0000","last_fullsized":"2024-05-27T10:18:43.149389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864338+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:18:42.864338+0000","last_peered":"2024-05-27T10:18:42.864338+0000","last_clean":"2024-05-27T10:18:42.864338+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:18:42.864338+0000","last_undegraded":"2024-05-27T10:18:42.864338+0000","last_fullsized":"2024-05-27T10:18:42.864338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914557+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:18:42.914557+0000","last_peered":"2024-05-27T10:18:42.914557+0000","last_clean":"2024-05-27T10:18:42.914557+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:18:42.914557+0000","last_undegraded":"2024-05-27T10:18:42.914557+0000","last_fullsized":"2024-05-27T10:18:42.914557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013138+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:18:43.013138+0000","last_peered":"2024-05-27T10:18:43.013138+0000","last_clean":"2024-05-27T10:18:43.013138+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:18:43.013138+0000","last_undegraded":"2024-05-27T10:18:43.013138+0000","last_fullsized":"2024-05-27T10:18:43.013138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913466+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:18:42.913466+0000","last_peered":"2024-05-27T10:18:42.913466+0000","last_clean":"2024-05-27T10:18:42.913466+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:18:42.913466+0000","last_undegraded":"2024-05-27T10:18:42.913466+0000","last_fullsized":"2024-05-27T10:18:42.913466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864950+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:18:42.864950+0000","last_peered":"2024-05-27T10:18:42.864950+0000","last_clean":"2024-05-27T10:18:42.864950+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:18:42.864950+0000","last_undegraded":"2024-05-27T10:18:42.864950+0000","last_fullsized":"2024-05-27T10:18:42.864950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915378+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:18:42.915378+0000","last_peered":"2024-05-27T10:18:42.915378+0000","last_clean":"2024-05-27T10:18:42.915378+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:18:42.915378+0000","last_undegraded":"2024-05-27T10:18:42.915378+0000","last_fullsized":"2024-05-27T10:18:42.915378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036117+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:18:43.036117+0000","last_peered":"2024-05-27T10:18:43.036117+0000","last_clean":"2024-05-27T10:18:43.036117+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:18:43.036117+0000","last_undegraded":"2024-05-27T10:18:43.036117+0000","last_fullsized":"2024-05-27T10:18:43.036117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858317+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:18:42.858317+0000","last_peered":"2024-05-27T10:18:42.858317+0000","last_clean":"2024-05-27T10:18:42.858317+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:18:42.858317+0000","last_undegraded":"2024-05-27T10:18:42.858317+0000","last_fullsized":"2024-05-27T10:18:42.858317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035965+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:18:43.035965+0000","last_peered":"2024-05-27T10:18:43.035965+0000","last_clean":"2024-05-27T10:18:43.035965+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:18:43.035965+0000","last_undegraded":"2024-05-27T10:18:43.035965+0000","last_fullsized":"2024-05-27T10:18:43.035965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035665+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:18:43.035665+0000","last_peered":"2024-05-27T10:18:43.035665+0000","last_clean":"2024-05-27T10:18:43.035665+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:18:43.035665+0000","last_undegraded":"2024-05-27T10:18:43.035665+0000","last_fullsized":"2024-05-27T10:18:43.035665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910287+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:18:42.910287+0000","last_peered":"2024-05-27T10:18:42.910287+0000","last_clean":"2024-05-27T10:18:42.910287+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:18:42.910287+0000","last_undegraded":"2024-05-27T10:18:42.910287+0000","last_fullsized":"2024-05-27T10:18:42.910287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.867088+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:18:42.867088+0000","last_peered":"2024-05-27T10:18:42.867088+0000","last_clean":"2024-05-27T10:18:42.867088+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:18:42.867088+0000","last_undegraded":"2024-05-27T10:18:42.867088+0000","last_fullsized":"2024-05-27T10:18:42.867088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011077+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:18:43.011077+0000","last_peered":"2024-05-27T10:18:43.011077+0000","last_clean":"2024-05-27T10:18:43.011077+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:18:43.011077+0000","last_undegraded":"2024-05-27T10:18:43.011077+0000","last_fullsized":"2024-05-27T10:18:43.011077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892271+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:18:42.892271+0000","last_peered":"2024-05-27T10:18:42.892271+0000","last_clean":"2024-05-27T10:18:42.892271+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:18:42.892271+0000","last_undegraded":"2024-05-27T10:18:42.892271+0000","last_fullsized":"2024-05-27T10:18:42.892271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010950+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:18:43.010950+0000","last_peered":"2024-05-27T10:18:43.010950+0000","last_clean":"2024-05-27T10:18:43.010950+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:18:43.010950+0000","last_undegraded":"2024-05-27T10:18:43.010950+0000","last_fullsized":"2024-05-27T10:18:43.010950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013844+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:18:43.013844+0000","last_peered":"2024-05-27T10:18:43.013844+0000","last_clean":"2024-05-27T10:18:43.013844+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:18:43.013844+0000","last_undegraded":"2024-05-27T10:18:43.013844+0000","last_fullsized":"2024-05-27T10:18:43.013844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013898+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:18:43.013898+0000","last_peered":"2024-05-27T10:18:43.013898+0000","last_clean":"2024-05-27T10:18:43.013898+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:18:43.013898+0000","last_undegraded":"2024-05-27T10:18:43.013898+0000","last_fullsized":"2024-05-27T10:18:43.013898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036523+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:18:43.036523+0000","last_peered":"2024-05-27T10:18:43.036523+0000","last_clean":"2024-05-27T10:18:43.036523+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:18:43.036523+0000","last_undegraded":"2024-05-27T10:18:43.036523+0000","last_fullsized":"2024-05-27T10:18:43.036523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150318+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:18:43.150318+0000","last_peered":"2024-05-27T10:18:43.150318+0000","last_clean":"2024-05-27T10:18:43.150318+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:18:43.150318+0000","last_undegraded":"2024-05-27T10:18:43.150318+0000","last_fullsized":"2024-05-27T10:18:43.150318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912126+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:18:42.912126+0000","last_peered":"2024-05-27T10:18:42.912126+0000","last_clean":"2024-05-27T10:18:42.912126+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:18:42.912126+0000","last_undegraded":"2024-05-27T10:18:42.912126+0000","last_fullsized":"2024-05-27T10:18:42.912126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150197+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:18:43.150197+0000","last_peered":"2024-05-27T10:18:43.150197+0000","last_clean":"2024-05-27T10:18:43.150197+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:18:43.150197+0000","last_undegraded":"2024-05-27T10:18:43.150197+0000","last_fullsized":"2024-05-27T10:18:43.150197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029108+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:18:43.029108+0000","last_peered":"2024-05-27T10:18:43.029108+0000","last_clean":"2024-05-27T10:18:43.029108+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:18:43.029108+0000","last_undegraded":"2024-05-27T10:18:43.029108+0000","last_fullsized":"2024-05-27T10:18:43.029108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'11","reported_seq":293,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.446077+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:18:47.446077+0000","last_peered":"2024-05-27T10:18:47.446077+0000","last_clean":"2024-05-27T10:18:47.446077+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:18:47.446077+0000","last_undegraded":"2024-05-27T10:18:47.446077+0000","last_fullsized":"2024-05-27T10:18:47.446077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":253,"num_read_kb":7,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149127+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:18:43.149127+0000","last_peered":"2024-05-27T10:18:43.149127+0000","last_clean":"2024-05-27T10:18:43.149127+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:18:43.149127+0000","last_undegraded":"2024-05-27T10:18:43.149127+0000","last_fullsized":"2024-05-27T10:18:43.149127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864000+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:18:42.864000+0000","last_peered":"2024-05-27T10:18:42.864000+0000","last_clean":"2024-05-27T10:18:42.864000+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:18:42.864000+0000","last_undegraded":"2024-05-27T10:18:42.864000+0000","last_fullsized":"2024-05-27T10:18:42.864000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028881+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:18:43.028881+0000","last_peered":"2024-05-27T10:18:43.028881+0000","last_clean":"2024-05-27T10:18:43.028881+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:18:43.028881+0000","last_undegraded":"2024-05-27T10:18:43.028881+0000","last_fullsized":"2024-05-27T10:18:43.028881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857767+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:18:42.857767+0000","last_peered":"2024-05-27T10:18:42.857767+0000","last_clean":"2024-05-27T10:18:42.857767+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:18:42.857767+0000","last_undegraded":"2024-05-27T10:18:42.857767+0000","last_fullsized":"2024-05-27T10:18:42.857767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013048+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:18:43.013048+0000","last_peered":"2024-05-27T10:18:43.013048+0000","last_clean":"2024-05-27T10:18:43.013048+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:18:43.013048+0000","last_undegraded":"2024-05-27T10:18:43.013048+0000","last_fullsized":"2024-05-27T10:18:43.013048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.027680+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:18:43.027680+0000","last_peered":"2024-05-27T10:18:43.027680+0000","last_clean":"2024-05-27T10:18:43.027680+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:18:43.027680+0000","last_undegraded":"2024-05-27T10:18:43.027680+0000","last_fullsized":"2024-05-27T10:18:43.027680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010824+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:18:43.010824+0000","last_peered":"2024-05-27T10:18:43.010824+0000","last_clean":"2024-05-27T10:18:43.010824+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:18:43.010824+0000","last_undegraded":"2024-05-27T10:18:43.010824+0000","last_fullsized":"2024-05-27T10:18:43.010824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":281,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.444884+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:18:47.444884+0000","last_peered":"2024-05-27T10:18:47.444884+0000","last_clean":"2024-05-27T10:18:47.444884+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:18:47.444884+0000","last_undegraded":"2024-05-27T10:18:47.444884+0000","last_fullsized":"2024-05-27T10:18:47.444884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913390+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:18:42.913390+0000","last_peered":"2024-05-27T10:18:42.913390+0000","last_clean":"2024-05-27T10:18:42.913390+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:18:42.913390+0000","last_undegraded":"2024-05-27T10:18:42.913390+0000","last_fullsized":"2024-05-27T10:18:42.913390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915631+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:18:42.915631+0000","last_peered":"2024-05-27T10:18:42.915631+0000","last_clean":"2024-05-27T10:18:42.915631+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:18:42.915631+0000","last_undegraded":"2024-05-27T10:18:42.915631+0000","last_fullsized":"2024-05-27T10:18:42.915631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036560+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:18:43.036560+0000","last_peered":"2024-05-27T10:18:43.036560+0000","last_clean":"2024-05-27T10:18:43.036560+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:18:43.036560+0000","last_undegraded":"2024-05-27T10:18:43.036560+0000","last_fullsized":"2024-05-27T10:18:43.036560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029425+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:18:43.029425+0000","last_peered":"2024-05-27T10:18:43.029425+0000","last_clean":"2024-05-27T10:18:43.029425+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:18:43.029425+0000","last_undegraded":"2024-05-27T10:18:43.029425+0000","last_fullsized":"2024-05-27T10:18:43.029425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912507+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:18:42.912507+0000","last_peered":"2024-05-27T10:18:42.912507+0000","last_clean":"2024-05-27T10:18:42.912507+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:18:42.912507+0000","last_undegraded":"2024-05-27T10:18:42.912507+0000","last_fullsized":"2024-05-27T10:18:42.912507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":573,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:48.399408+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:18:48.399408+0000","last_peered":"2024-05-27T10:18:48.399408+0000","last_clean":"2024-05-27T10:18:48.399408+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:18:48.399408+0000","last_undegraded":"2024-05-27T10:18:48.399408+0000","last_fullsized":"2024-05-27T10:18:48.399408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":507,"num_read_kb":319,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149956+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:18:43.149956+0000","last_peered":"2024-05-27T10:18:43.149956+0000","last_clean":"2024-05-27T10:18:43.149956+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:18:43.149956+0000","last_undegraded":"2024-05-27T10:18:43.149956+0000","last_fullsized":"2024-05-27T10:18:43.149956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892232+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:18:42.892232+0000","last_peered":"2024-05-27T10:18:42.892232+0000","last_clean":"2024-05-27T10:18:42.892232+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:18:42.892232+0000","last_undegraded":"2024-05-27T10:18:42.892232+0000","last_fullsized":"2024-05-27T10:18:42.892232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857993+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:18:42.857993+0000","last_peered":"2024-05-27T10:18:42.857993+0000","last_clean":"2024-05-27T10:18:42.857993+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:18:42.857993+0000","last_undegraded":"2024-05-27T10:18:42.857993+0000","last_fullsized":"2024-05-27T10:18:42.857993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028152+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:18:43.028152+0000","last_peered":"2024-05-27T10:18:43.028152+0000","last_clean":"2024-05-27T10:18:43.028152+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:18:43.028152+0000","last_undegraded":"2024-05-27T10:18:43.028152+0000","last_fullsized":"2024-05-27T10:18:43.028152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915250+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:18:42.915250+0000","last_peered":"2024-05-27T10:18:42.915250+0000","last_clean":"2024-05-27T10:18:42.915250+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:18:42.915250+0000","last_undegraded":"2024-05-27T10:18:42.915250+0000","last_fullsized":"2024-05-27T10:18:42.915250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":290,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:48.434451+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:18:48.434451+0000","last_peered":"2024-05-27T10:18:48.434451+0000","last_clean":"2024-05-27T10:18:48.434451+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:18:48.434451+0000","last_undegraded":"2024-05-27T10:18:48.434451+0000","last_fullsized":"2024-05-27T10:18:48.434451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914298+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:18:42.914298+0000","last_peered":"2024-05-27T10:18:42.914298+0000","last_clean":"2024-05-27T10:18:42.914298+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:18:42.914298+0000","last_undegraded":"2024-05-27T10:18:42.914298+0000","last_fullsized":"2024-05-27T10:18:42.914298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148905+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:18:43.148905+0000","last_peered":"2024-05-27T10:18:43.148905+0000","last_clean":"2024-05-27T10:18:43.148905+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:18:43.148905+0000","last_undegraded":"2024-05-27T10:18:43.148905+0000","last_fullsized":"2024-05-27T10:18:43.148905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028231+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:18:43.028231+0000","last_peered":"2024-05-27T10:18:43.028231+0000","last_clean":"2024-05-27T10:18:43.028231+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:18:43.028231+0000","last_undegraded":"2024-05-27T10:18:43.028231+0000","last_fullsized":"2024-05-27T10:18:43.028231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.641820+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:18:43.641820+0000","last_peered":"2024-05-27T10:18:43.641820+0000","last_clean":"2024-05-27T10:18:43.641820+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:18:43.641820+0000","last_undegraded":"2024-05-27T10:18:43.641820+0000","last_fullsized":"2024-05-27T10:18:43.641820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865929+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:18:42.865929+0000","last_peered":"2024-05-27T10:18:42.865929+0000","last_clean":"2024-05-27T10:18:42.865929+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:18:42.865929+0000","last_undegraded":"2024-05-27T10:18:42.865929+0000","last_fullsized":"2024-05-27T10:18:42.865929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010814+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:18:43.010814+0000","last_peered":"2024-05-27T10:18:43.010814+0000","last_clean":"2024-05-27T10:18:43.010814+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:18:43.010814+0000","last_undegraded":"2024-05-27T10:18:43.010814+0000","last_fullsized":"2024-05-27T10:18:43.010814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864668+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:18:42.864668+0000","last_peered":"2024-05-27T10:18:42.864668+0000","last_clean":"2024-05-27T10:18:42.864668+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:18:42.864668+0000","last_undegraded":"2024-05-27T10:18:42.864668+0000","last_fullsized":"2024-05-27T10:18:42.864668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914915+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:18:42.914915+0000","last_peered":"2024-05-27T10:18:42.914915+0000","last_clean":"2024-05-27T10:18:42.914915+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:18:42.914915+0000","last_undegraded":"2024-05-27T10:18:42.914915+0000","last_fullsized":"2024-05-27T10:18:42.914915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865657+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:18:42.865657+0000","last_peered":"2024-05-27T10:18:42.865657+0000","last_clean":"2024-05-27T10:18:42.865657+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:18:42.865657+0000","last_undegraded":"2024-05-27T10:18:42.865657+0000","last_fullsized":"2024-05-27T10:18:42.865657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028265+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:18:43.028265+0000","last_peered":"2024-05-27T10:18:43.028265+0000","last_clean":"2024-05-27T10:18:43.028265+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:18:43.028265+0000","last_undegraded":"2024-05-27T10:18:43.028265+0000","last_fullsized":"2024-05-27T10:18:43.028265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":283,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.430751+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:18:47.430751+0000","last_peered":"2024-05-27T10:18:47.430751+0000","last_clean":"2024-05-27T10:18:47.430751+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:18:47.430751+0000","last_undegraded":"2024-05-27T10:18:47.430751+0000","last_fullsized":"2024-05-27T10:18:47.430751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013516+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:18:43.013516+0000","last_peered":"2024-05-27T10:18:43.013516+0000","last_clean":"2024-05-27T10:18:43.013516+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:18:43.013516+0000","last_undegraded":"2024-05-27T10:18:43.013516+0000","last_fullsized":"2024-05-27T10:18:43.013516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036179+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:18:43.036179+0000","last_peered":"2024-05-27T10:18:43.036179+0000","last_clean":"2024-05-27T10:18:43.036179+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:18:43.036179+0000","last_undegraded":"2024-05-27T10:18:43.036179+0000","last_fullsized":"2024-05-27T10:18:43.036179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"82'99","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:45.052019+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:18:45.052019+0000","last_peered":"2024-05-27T10:18:45.052019+0000","last_clean":"2024-05-27T10:18:45.052019+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:18:45.052019+0000","last_undegraded":"2024-05-27T10:18:45.052019+0000","last_fullsized":"2024-05-27T10:18:45.052019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857424+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:18:42.857424+0000","last_peered":"2024-05-27T10:18:42.857424+0000","last_clean":"2024-05-27T10:18:42.857424+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:18:42.857424+0000","last_undegraded":"2024-05-27T10:18:42.857424+0000","last_fullsized":"2024-05-27T10:18:42.857424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914885+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:18:42.914885+0000","last_peered":"2024-05-27T10:18:42.914885+0000","last_clean":"2024-05-27T10:18:42.914885+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:18:42.914885+0000","last_undegraded":"2024-05-27T10:18:42.914885+0000","last_fullsized":"2024-05-27T10:18:42.914885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149983+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:18:43.149983+0000","last_peered":"2024-05-27T10:18:43.149983+0000","last_clean":"2024-05-27T10:18:43.149983+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:18:43.149983+0000","last_undegraded":"2024-05-27T10:18:43.149983+0000","last_fullsized":"2024-05-27T10:18:43.149983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858159+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:18:42.858159+0000","last_peered":"2024-05-27T10:18:42.858159+0000","last_clean":"2024-05-27T10:18:42.858159+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:18:42.858159+0000","last_undegraded":"2024-05-27T10:18:42.858159+0000","last_fullsized":"2024-05-27T10:18:42.858159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'49","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.228670+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:18:43.228670+0000","last_peered":"2024-05-27T10:18:43.228670+0000","last_clean":"2024-05-27T10:18:43.228670+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:18:43.228670+0000","last_undegraded":"2024-05-27T10:18:43.228670+0000","last_fullsized":"2024-05-27T10:18:43.228670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.014095+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:18:43.014095+0000","last_peered":"2024-05-27T10:18:43.014095+0000","last_clean":"2024-05-27T10:18:43.014095+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:18:43.014095+0000","last_undegraded":"2024-05-27T10:18:43.014095+0000","last_fullsized":"2024-05-27T10:18:43.014095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036143+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:18:43.036143+0000","last_peered":"2024-05-27T10:18:43.036143+0000","last_clean":"2024-05-27T10:18:43.036143+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:18:43.036143+0000","last_undegraded":"2024-05-27T10:18:43.036143+0000","last_fullsized":"2024-05-27T10:18:43.036143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035987+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:18:43.035987+0000","last_peered":"2024-05-27T10:18:43.035987+0000","last_clean":"2024-05-27T10:18:43.035987+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:18:43.035987+0000","last_undegraded":"2024-05-27T10:18:43.035987+0000","last_fullsized":"2024-05-27T10:18:43.035987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910322+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:18:42.910322+0000","last_peered":"2024-05-27T10:18:42.910322+0000","last_clean":"2024-05-27T10:18:42.910322+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:18:42.910322+0000","last_undegraded":"2024-05-27T10:18:42.910322+0000","last_fullsized":"2024-05-27T10:18:42.910322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148975+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:18:43.148975+0000","last_peered":"2024-05-27T10:18:43.148975+0000","last_clean":"2024-05-27T10:18:43.148975+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:18:43.148975+0000","last_undegraded":"2024-05-27T10:18:43.148975+0000","last_fullsized":"2024-05-27T10:18:43.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013173+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:18:43.013173+0000","last_peered":"2024-05-27T10:18:43.013173+0000","last_clean":"2024-05-27T10:18:43.013173+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:18:43.013173+0000","last_undegraded":"2024-05-27T10:18:43.013173+0000","last_fullsized":"2024-05-27T10:18:43.013173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865624+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:18:42.865624+0000","last_peered":"2024-05-27T10:18:42.865624+0000","last_clean":"2024-05-27T10:18:42.865624+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:18:42.865624+0000","last_undegraded":"2024-05-27T10:18:42.865624+0000","last_fullsized":"2024-05-27T10:18:42.865624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865382+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:18:42.865382+0000","last_peered":"2024-05-27T10:18:42.865382+0000","last_clean":"2024-05-27T10:18:42.865382+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:18:42.865382+0000","last_undegraded":"2024-05-27T10:18:42.865382+0000","last_fullsized":"2024-05-27T10:18:42.865382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013727+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:18:43.013727+0000","last_peered":"2024-05-27T10:18:43.013727+0000","last_clean":"2024-05-27T10:18:43.013727+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:18:43.013727+0000","last_undegraded":"2024-05-27T10:18:43.013727+0000","last_fullsized":"2024-05-27T10:18:43.013727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150814+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:18:43.150814+0000","last_peered":"2024-05-27T10:18:43.150814+0000","last_clean":"2024-05-27T10:18:43.150814+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:18:43.150814+0000","last_undegraded":"2024-05-27T10:18:43.150814+0000","last_fullsized":"2024-05-27T10:18:43.150814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915235+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:18:42.915235+0000","last_peered":"2024-05-27T10:18:42.915235+0000","last_clean":"2024-05-27T10:18:42.915235+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:18:42.915235+0000","last_undegraded":"2024-05-27T10:18:42.915235+0000","last_fullsized":"2024-05-27T10:18:42.915235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892674+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:18:42.892674+0000","last_peered":"2024-05-27T10:18:42.892674+0000","last_clean":"2024-05-27T10:18:42.892674+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:18:42.892674+0000","last_undegraded":"2024-05-27T10:18:42.892674+0000","last_fullsized":"2024-05-27T10:18:42.892674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916054+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:18:42.916054+0000","last_peered":"2024-05-27T10:18:42.916054+0000","last_clean":"2024-05-27T10:18:42.916054+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:18:42.916054+0000","last_undegraded":"2024-05-27T10:18:42.916054+0000","last_fullsized":"2024-05-27T10:18:42.916054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'39","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.226174+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:18:43.226174+0000","last_peered":"2024-05-27T10:18:43.226174+0000","last_clean":"2024-05-27T10:18:43.226174+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:18:43.226174+0000","last_undegraded":"2024-05-27T10:18:43.226174+0000","last_fullsized":"2024-05-27T10:18:43.226174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915511+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:18:42.915511+0000","last_peered":"2024-05-27T10:18:42.915511+0000","last_clean":"2024-05-27T10:18:42.915511+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:18:42.915511+0000","last_undegraded":"2024-05-27T10:18:42.915511+0000","last_fullsized":"2024-05-27T10:18:42.915511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858229+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:18:42.858229+0000","last_peered":"2024-05-27T10:18:42.858229+0000","last_clean":"2024-05-27T10:18:42.858229+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:18:42.858229+0000","last_undegraded":"2024-05-27T10:18:42.858229+0000","last_fullsized":"2024-05-27T10:18:42.858229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916095+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:18:42.916095+0000","last_peered":"2024-05-27T10:18:42.916095+0000","last_clean":"2024-05-27T10:18:42.916095+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:18:42.916095+0000","last_undegraded":"2024-05-27T10:18:42.916095+0000","last_fullsized":"2024-05-27T10:18:42.916095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865960+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:18:42.865960+0000","last_peered":"2024-05-27T10:18:42.865960+0000","last_clean":"2024-05-27T10:18:42.865960+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:18:42.865960+0000","last_undegraded":"2024-05-27T10:18:42.865960+0000","last_fullsized":"2024-05-27T10:18:42.865960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892084+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:18:42.892084+0000","last_peered":"2024-05-27T10:18:42.892084+0000","last_clean":"2024-05-27T10:18:42.892084+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:18:42.892084+0000","last_undegraded":"2024-05-27T10:18:42.892084+0000","last_fullsized":"2024-05-27T10:18:42.892084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.891669+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:18:42.891669+0000","last_peered":"2024-05-27T10:18:42.891669+0000","last_clean":"2024-05-27T10:18:42.891669+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:18:42.891669+0000","last_undegraded":"2024-05-27T10:18:42.891669+0000","last_fullsized":"2024-05-27T10:18:42.891669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036165+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:18:43.036165+0000","last_peered":"2024-05-27T10:18:43.036165+0000","last_clean":"2024-05-27T10:18:43.036165+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:18:43.036165+0000","last_undegraded":"2024-05-27T10:18:43.036165+0000","last_fullsized":"2024-05-27T10:18:43.036165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858813+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:18:42.858813+0000","last_peered":"2024-05-27T10:18:42.858813+0000","last_clean":"2024-05-27T10:18:42.858813+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:18:42.858813+0000","last_undegraded":"2024-05-27T10:18:42.858813+0000","last_fullsized":"2024-05-27T10:18:42.858813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865913+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:18:42.865913+0000","last_peered":"2024-05-27T10:18:42.865913+0000","last_clean":"2024-05-27T10:18:42.865913+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:18:42.865913+0000","last_undegraded":"2024-05-27T10:18:42.865913+0000","last_fullsized":"2024-05-27T10:18:42.865913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892758+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:18:42.892758+0000","last_peered":"2024-05-27T10:18:42.892758+0000","last_clean":"2024-05-27T10:18:42.892758+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:18:42.892758+0000","last_undegraded":"2024-05-27T10:18:42.892758+0000","last_fullsized":"2024-05-27T10:18:42.892758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010907+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:18:43.010907+0000","last_peered":"2024-05-27T10:18:43.010907+0000","last_clean":"2024-05-27T10:18:43.010907+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:18:43.010907+0000","last_undegraded":"2024-05-27T10:18:43.010907+0000","last_fullsized":"2024-05-27T10:18:43.010907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028084+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:18:43.028084+0000","last_peered":"2024-05-27T10:18:43.028084+0000","last_clean":"2024-05-27T10:18:43.028084+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:18:43.028084+0000","last_undegraded":"2024-05-27T10:18:43.028084+0000","last_fullsized":"2024-05-27T10:18:43.028084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013207+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:18:43.013207+0000","last_peered":"2024-05-27T10:18:43.013207+0000","last_clean":"2024-05-27T10:18:43.013207+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:18:43.013207+0000","last_undegraded":"2024-05-27T10:18:43.013207+0000","last_fullsized":"2024-05-27T10:18:43.013207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913669+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:18:42.913669+0000","last_peered":"2024-05-27T10:18:42.913669+0000","last_clean":"2024-05-27T10:18:42.913669+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:18:42.913669+0000","last_undegraded":"2024-05-27T10:18:42.913669+0000","last_fullsized":"2024-05-27T10:18:42.913669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858143+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:18:42.858143+0000","last_peered":"2024-05-27T10:18:42.858143+0000","last_clean":"2024-05-27T10:18:42.858143+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:18:42.858143+0000","last_undegraded":"2024-05-27T10:18:42.858143+0000","last_fullsized":"2024-05-27T10:18:42.858143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866758+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:18:42.866758+0000","last_peered":"2024-05-27T10:18:42.866758+0000","last_clean":"2024-05-27T10:18:42.866758+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:18:42.866758+0000","last_undegraded":"2024-05-27T10:18:42.866758+0000","last_fullsized":"2024-05-27T10:18:42.866758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599445,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1613,"num_read_kb":919,"num_write":305,"num_write_kb":1447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1798335,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":830200,"data_compressed":22039,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874321,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1228886,"data_compressed":45201,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037849,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":830436,"data_compressed":22039,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234081,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":437140,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430311,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1032293,"data_compressed":39045,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593839,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":888832,"data_stored":436609,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":503807,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":437880,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":507355,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149830,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":961736,"data_compressed":28195,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953600,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":438462,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1162161,"data_compressed":45201,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":135274,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:18:51.785 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-27T10:18:51.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T10:18:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:51 smithi143 ceph-mon[35545]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:51 smithi143 ceph-mon[35545]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:51 smithi143 ceph-mon[35545]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:51 smithi042 ceph-mon[31661]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:51 smithi042 ceph-mon[31661]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:51 smithi042 ceph-mon[31661]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:51 smithi104 ceph-mon[34812]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:51 smithi104 ceph-mon[34812]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:51.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:51 smithi104 ceph-mon[34812]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-27T10:18:52.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 -- 172.21.15.42:0/4067362804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10dfa0 msgr2=0x7f821c110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 --2- 172.21.15.42:0/4067362804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10dfa0 0x7f821c110490 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8214009ca0 tx=0x7f821402a8c0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 -- 172.21.15.42:0/4067362804 shutdown_connections 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 --2- 172.21.15.42:0/4067362804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10dfa0 0x7f821c110490 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 --2- 172.21.15.42:0/4067362804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c10b5b0 0x7f821c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 --2- 172.21.15.42:0/4067362804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f821c104b50 0x7f821c104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.101+0000 7f821b200640 1 -- 172.21.15.42:0/4067362804 >> 172.21.15.42:0/4067362804 conn(0x7f821c0780b0 msgr2=0x7f821c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.102+0000 7f821b200640 1 -- 172.21.15.42:0/4067362804 shutdown_connections 2024-05-27T10:18:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.102+0000 7f821b200640 1 -- 172.21.15.42:0/4067362804 wait complete. 2024-05-27T10:18:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 Processor -- start 2024-05-27T10:18:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 -- start start 2024-05-27T10:18:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c104b50 0x7f821c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 0x7f821c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f821c10dfa0 0x7f821c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f821c113b00 con 0x7f821c10b5b0 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f821c113980 con 0x7f821c104b50 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.103+0000 7f821b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f821c113c80 con 0x7f821c10dfa0 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 0x7f821c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 0x7f821c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53392/0 (socket says 172.21.15.42:53392) 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 -- 172.21.15.42:0/3723087713 learned_addr learned my addr 172.21.15.42:0/3723087713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219e00640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c104b50 0x7f821c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f821a800640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f821c10dfa0 0x7f821c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:52.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 -- 172.21.15.42:0/3723087713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f821c10dfa0 msgr2=0x7f821c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:52.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f821c10dfa0 0x7f821c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 -- 172.21.15.42:0/3723087713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c104b50 msgr2=0x7f821c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c104b50 0x7f821c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219400640 1 -- 172.21.15.42:0/3723087713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c19f380 con 0x7f821c10b5b0 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.104+0000 7f8219e00640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c104b50 0x7f821c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.105+0000 7f8219400640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 0x7f821c19aa80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f820800e9f0 tx=0x7f820800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.105+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820800cd80 con 0x7f821c10b5b0 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.105+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82080108c0 con 0x7f821c10b5b0 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.105+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c19f670 con 0x7f821c10b5b0 2024-05-27T10:18:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.105+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82080187c0 con 0x7f821c10b5b0 2024-05-27T10:18:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.105+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c1a6ff0 con 0x7f821c10b5b0 2024-05-27T10:18:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.106+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821c06a890 con 0x7f821c10b5b0 2024-05-27T10:18:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.109+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f8208010430 con 0x7f821c10b5b0 2024-05-27T10:18:52.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.111+0000 7f8212a00640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f82000b2a70 0x7f82000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.111+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f82080e3090 con 0x7f821c10b5b0 2024-05-27T10:18:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.111+0000 7f8219e00640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f82000b2a70 0x7f82000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.112+0000 7f8219e00640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f82000b2a70 0x7f82000b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f820c009fc0 tx=0x7f820c005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.113+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82080aa1c0 con 0x7f821c10b5b0 2024-05-27T10:18:52.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.334+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f821c101780 con 0x7f82000b2a70 2024-05-27T10:18:52.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.345+0000 7f8212a00640 1 -- 172.21.15.42:0/3723087713 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258021 (secure 0 0 0) 0x7f821c101780 con 0x7f82000b2a70 2024-05-27T10:18:52.349 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:52.350 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f82000b2a70 msgr2=0x7f82000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f82000b2a70 0x7f82000b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f820c009fc0 tx=0x7f820c005f90 comp rx=0 tx=0).stop 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 msgr2=0x7f821c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 0x7f821c19aa80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f820800e9f0 tx=0x7f820800eeb0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 shutdown_connections 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f82000b2a70 0x7f82000b4f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f821c10dfa0 0x7f821c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f821c10b5b0 0x7f821c19aa80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 --2- 172.21.15.42:0/3723087713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f821c104b50 0x7f821c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 >> 172.21.15.42:0/3723087713 conn(0x7f821c0780b0 msgr2=0x7f821c101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:52.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 shutdown_connections 2024-05-27T10:18:52.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:52.350+0000 7f821b200640 1 -- 172.21.15.42:0/3723087713 wait complete. 2024-05-27T10:18:52.377 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T10:18:50.894144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190047,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1757,"num_read_kb":1167,"num_write":442,"num_write_kb":3599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360232,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581592,"statfs":{"total":1151940427776,"available":1151571550208,"internally_reserved":0,"allocated":11317248,"data_stored":8806965,"data_compressed":219078,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3302,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":97,"num_read_kb":206,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.977203"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866630+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:18:42.866630+0000","last_peered":"2024-05-27T10:18:42.866630+0000","last_clean":"2024-05-27T10:18:42.866630+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:18:42.866630+0000","last_undegraded":"2024-05-27T10:18:42.866630+0000","last_fullsized":"2024-05-27T10:18:42.866630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035777+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:18:43.035777+0000","last_peered":"2024-05-27T10:18:43.035777+0000","last_clean":"2024-05-27T10:18:43.035777+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:18:43.035777+0000","last_undegraded":"2024-05-27T10:18:43.035777+0000","last_fullsized":"2024-05-27T10:18:43.035777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036368+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:18:43.036368+0000","last_peered":"2024-05-27T10:18:43.036368+0000","last_clean":"2024-05-27T10:18:43.036368+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:18:43.036368+0000","last_undegraded":"2024-05-27T10:18:43.036368+0000","last_fullsized":"2024-05-27T10:18:43.036368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011005+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:18:43.011005+0000","last_peered":"2024-05-27T10:18:43.011005+0000","last_clean":"2024-05-27T10:18:43.011005+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:18:43.011005+0000","last_undegraded":"2024-05-27T10:18:43.011005+0000","last_fullsized":"2024-05-27T10:18:43.011005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149389+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:18:43.149389+0000","last_peered":"2024-05-27T10:18:43.149389+0000","last_clean":"2024-05-27T10:18:43.149389+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:18:43.149389+0000","last_undegraded":"2024-05-27T10:18:43.149389+0000","last_fullsized":"2024-05-27T10:18:43.149389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864338+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:18:42.864338+0000","last_peered":"2024-05-27T10:18:42.864338+0000","last_clean":"2024-05-27T10:18:42.864338+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:18:42.864338+0000","last_undegraded":"2024-05-27T10:18:42.864338+0000","last_fullsized":"2024-05-27T10:18:42.864338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914557+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:18:42.914557+0000","last_peered":"2024-05-27T10:18:42.914557+0000","last_clean":"2024-05-27T10:18:42.914557+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:18:42.914557+0000","last_undegraded":"2024-05-27T10:18:42.914557+0000","last_fullsized":"2024-05-27T10:18:42.914557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013138+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:18:43.013138+0000","last_peered":"2024-05-27T10:18:43.013138+0000","last_clean":"2024-05-27T10:18:43.013138+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:18:43.013138+0000","last_undegraded":"2024-05-27T10:18:43.013138+0000","last_fullsized":"2024-05-27T10:18:43.013138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913466+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:18:42.913466+0000","last_peered":"2024-05-27T10:18:42.913466+0000","last_clean":"2024-05-27T10:18:42.913466+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:18:42.913466+0000","last_undegraded":"2024-05-27T10:18:42.913466+0000","last_fullsized":"2024-05-27T10:18:42.913466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864950+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:18:42.864950+0000","last_peered":"2024-05-27T10:18:42.864950+0000","last_clean":"2024-05-27T10:18:42.864950+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:18:42.864950+0000","last_undegraded":"2024-05-27T10:18:42.864950+0000","last_fullsized":"2024-05-27T10:18:42.864950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915378+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:18:42.915378+0000","last_peered":"2024-05-27T10:18:42.915378+0000","last_clean":"2024-05-27T10:18:42.915378+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:18:42.915378+0000","last_undegraded":"2024-05-27T10:18:42.915378+0000","last_fullsized":"2024-05-27T10:18:42.915378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036117+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:18:43.036117+0000","last_peered":"2024-05-27T10:18:43.036117+0000","last_clean":"2024-05-27T10:18:43.036117+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:18:43.036117+0000","last_undegraded":"2024-05-27T10:18:43.036117+0000","last_fullsized":"2024-05-27T10:18:43.036117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858317+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:18:42.858317+0000","last_peered":"2024-05-27T10:18:42.858317+0000","last_clean":"2024-05-27T10:18:42.858317+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:18:42.858317+0000","last_undegraded":"2024-05-27T10:18:42.858317+0000","last_fullsized":"2024-05-27T10:18:42.858317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035965+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:18:43.035965+0000","last_peered":"2024-05-27T10:18:43.035965+0000","last_clean":"2024-05-27T10:18:43.035965+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:18:43.035965+0000","last_undegraded":"2024-05-27T10:18:43.035965+0000","last_fullsized":"2024-05-27T10:18:43.035965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035665+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:18:43.035665+0000","last_peered":"2024-05-27T10:18:43.035665+0000","last_clean":"2024-05-27T10:18:43.035665+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:18:43.035665+0000","last_undegraded":"2024-05-27T10:18:43.035665+0000","last_fullsized":"2024-05-27T10:18:43.035665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910287+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:18:42.910287+0000","last_peered":"2024-05-27T10:18:42.910287+0000","last_clean":"2024-05-27T10:18:42.910287+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:18:42.910287+0000","last_undegraded":"2024-05-27T10:18:42.910287+0000","last_fullsized":"2024-05-27T10:18:42.910287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.867088+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:18:42.867088+0000","last_peered":"2024-05-27T10:18:42.867088+0000","last_clean":"2024-05-27T10:18:42.867088+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:18:42.867088+0000","last_undegraded":"2024-05-27T10:18:42.867088+0000","last_fullsized":"2024-05-27T10:18:42.867088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011077+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:18:43.011077+0000","last_peered":"2024-05-27T10:18:43.011077+0000","last_clean":"2024-05-27T10:18:43.011077+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:18:43.011077+0000","last_undegraded":"2024-05-27T10:18:43.011077+0000","last_fullsized":"2024-05-27T10:18:43.011077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892271+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:18:42.892271+0000","last_peered":"2024-05-27T10:18:42.892271+0000","last_clean":"2024-05-27T10:18:42.892271+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:18:42.892271+0000","last_undegraded":"2024-05-27T10:18:42.892271+0000","last_fullsized":"2024-05-27T10:18:42.892271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010950+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:18:43.010950+0000","last_peered":"2024-05-27T10:18:43.010950+0000","last_clean":"2024-05-27T10:18:43.010950+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:18:43.010950+0000","last_undegraded":"2024-05-27T10:18:43.010950+0000","last_fullsized":"2024-05-27T10:18:43.010950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013844+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:18:43.013844+0000","last_peered":"2024-05-27T10:18:43.013844+0000","last_clean":"2024-05-27T10:18:43.013844+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:18:43.013844+0000","last_undegraded":"2024-05-27T10:18:43.013844+0000","last_fullsized":"2024-05-27T10:18:43.013844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013898+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:18:43.013898+0000","last_peered":"2024-05-27T10:18:43.013898+0000","last_clean":"2024-05-27T10:18:43.013898+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:18:43.013898+0000","last_undegraded":"2024-05-27T10:18:43.013898+0000","last_fullsized":"2024-05-27T10:18:43.013898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036523+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:18:43.036523+0000","last_peered":"2024-05-27T10:18:43.036523+0000","last_clean":"2024-05-27T10:18:43.036523+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:18:43.036523+0000","last_undegraded":"2024-05-27T10:18:43.036523+0000","last_fullsized":"2024-05-27T10:18:43.036523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150318+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:18:43.150318+0000","last_peered":"2024-05-27T10:18:43.150318+0000","last_clean":"2024-05-27T10:18:43.150318+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:18:43.150318+0000","last_undegraded":"2024-05-27T10:18:43.150318+0000","last_fullsized":"2024-05-27T10:18:43.150318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912126+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:18:42.912126+0000","last_peered":"2024-05-27T10:18:42.912126+0000","last_clean":"2024-05-27T10:18:42.912126+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:18:42.912126+0000","last_undegraded":"2024-05-27T10:18:42.912126+0000","last_fullsized":"2024-05-27T10:18:42.912126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150197+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:18:43.150197+0000","last_peered":"2024-05-27T10:18:43.150197+0000","last_clean":"2024-05-27T10:18:43.150197+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:18:43.150197+0000","last_undegraded":"2024-05-27T10:18:43.150197+0000","last_fullsized":"2024-05-27T10:18:43.150197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029108+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:18:43.029108+0000","last_peered":"2024-05-27T10:18:43.029108+0000","last_clean":"2024-05-27T10:18:43.029108+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:18:43.029108+0000","last_undegraded":"2024-05-27T10:18:43.029108+0000","last_fullsized":"2024-05-27T10:18:43.029108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'11","reported_seq":293,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.446077+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:18:47.446077+0000","last_peered":"2024-05-27T10:18:47.446077+0000","last_clean":"2024-05-27T10:18:47.446077+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:18:47.446077+0000","last_undegraded":"2024-05-27T10:18:47.446077+0000","last_fullsized":"2024-05-27T10:18:47.446077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":253,"num_read_kb":7,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149127+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:18:43.149127+0000","last_peered":"2024-05-27T10:18:43.149127+0000","last_clean":"2024-05-27T10:18:43.149127+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:18:43.149127+0000","last_undegraded":"2024-05-27T10:18:43.149127+0000","last_fullsized":"2024-05-27T10:18:43.149127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864000+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:18:42.864000+0000","last_peered":"2024-05-27T10:18:42.864000+0000","last_clean":"2024-05-27T10:18:42.864000+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:18:42.864000+0000","last_undegraded":"2024-05-27T10:18:42.864000+0000","last_fullsized":"2024-05-27T10:18:42.864000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028881+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:18:43.028881+0000","last_peered":"2024-05-27T10:18:43.028881+0000","last_clean":"2024-05-27T10:18:43.028881+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:18:43.028881+0000","last_undegraded":"2024-05-27T10:18:43.028881+0000","last_fullsized":"2024-05-27T10:18:43.028881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857767+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:18:42.857767+0000","last_peered":"2024-05-27T10:18:42.857767+0000","last_clean":"2024-05-27T10:18:42.857767+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:18:42.857767+0000","last_undegraded":"2024-05-27T10:18:42.857767+0000","last_fullsized":"2024-05-27T10:18:42.857767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013048+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:18:43.013048+0000","last_peered":"2024-05-27T10:18:43.013048+0000","last_clean":"2024-05-27T10:18:43.013048+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:18:43.013048+0000","last_undegraded":"2024-05-27T10:18:43.013048+0000","last_fullsized":"2024-05-27T10:18:43.013048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.027680+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:18:43.027680+0000","last_peered":"2024-05-27T10:18:43.027680+0000","last_clean":"2024-05-27T10:18:43.027680+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:18:43.027680+0000","last_undegraded":"2024-05-27T10:18:43.027680+0000","last_fullsized":"2024-05-27T10:18:43.027680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010824+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:18:43.010824+0000","last_peered":"2024-05-27T10:18:43.010824+0000","last_clean":"2024-05-27T10:18:43.010824+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:18:43.010824+0000","last_undegraded":"2024-05-27T10:18:43.010824+0000","last_fullsized":"2024-05-27T10:18:43.010824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":281,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.444884+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:18:47.444884+0000","last_peered":"2024-05-27T10:18:47.444884+0000","last_clean":"2024-05-27T10:18:47.444884+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:18:47.444884+0000","last_undegraded":"2024-05-27T10:18:47.444884+0000","last_fullsized":"2024-05-27T10:18:47.444884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913390+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:18:42.913390+0000","last_peered":"2024-05-27T10:18:42.913390+0000","last_clean":"2024-05-27T10:18:42.913390+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:18:42.913390+0000","last_undegraded":"2024-05-27T10:18:42.913390+0000","last_fullsized":"2024-05-27T10:18:42.913390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915631+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:18:42.915631+0000","last_peered":"2024-05-27T10:18:42.915631+0000","last_clean":"2024-05-27T10:18:42.915631+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:18:42.915631+0000","last_undegraded":"2024-05-27T10:18:42.915631+0000","last_fullsized":"2024-05-27T10:18:42.915631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036560+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:18:43.036560+0000","last_peered":"2024-05-27T10:18:43.036560+0000","last_clean":"2024-05-27T10:18:43.036560+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:18:43.036560+0000","last_undegraded":"2024-05-27T10:18:43.036560+0000","last_fullsized":"2024-05-27T10:18:43.036560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029425+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:18:43.029425+0000","last_peered":"2024-05-27T10:18:43.029425+0000","last_clean":"2024-05-27T10:18:43.029425+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:18:43.029425+0000","last_undegraded":"2024-05-27T10:18:43.029425+0000","last_fullsized":"2024-05-27T10:18:43.029425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912507+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:18:42.912507+0000","last_peered":"2024-05-27T10:18:42.912507+0000","last_clean":"2024-05-27T10:18:42.912507+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:18:42.912507+0000","last_undegraded":"2024-05-27T10:18:42.912507+0000","last_fullsized":"2024-05-27T10:18:42.912507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":573,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:48.399408+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:18:48.399408+0000","last_peered":"2024-05-27T10:18:48.399408+0000","last_clean":"2024-05-27T10:18:48.399408+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:18:48.399408+0000","last_undegraded":"2024-05-27T10:18:48.399408+0000","last_fullsized":"2024-05-27T10:18:48.399408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":507,"num_read_kb":319,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149956+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:18:43.149956+0000","last_peered":"2024-05-27T10:18:43.149956+0000","last_clean":"2024-05-27T10:18:43.149956+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:18:43.149956+0000","last_undegraded":"2024-05-27T10:18:43.149956+0000","last_fullsized":"2024-05-27T10:18:43.149956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892232+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:18:42.892232+0000","last_peered":"2024-05-27T10:18:42.892232+0000","last_clean":"2024-05-27T10:18:42.892232+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:18:42.892232+0000","last_undegraded":"2024-05-27T10:18:42.892232+0000","last_fullsized":"2024-05-27T10:18:42.892232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857993+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:18:42.857993+0000","last_peered":"2024-05-27T10:18:42.857993+0000","last_clean":"2024-05-27T10:18:42.857993+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:18:42.857993+0000","last_undegraded":"2024-05-27T10:18:42.857993+0000","last_fullsized":"2024-05-27T10:18:42.857993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028152+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:18:43.028152+0000","last_peered":"2024-05-27T10:18:43.028152+0000","last_clean":"2024-05-27T10:18:43.028152+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:18:43.028152+0000","last_undegraded":"2024-05-27T10:18:43.028152+0000","last_fullsized":"2024-05-27T10:18:43.028152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915250+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:18:42.915250+0000","last_peered":"2024-05-27T10:18:42.915250+0000","last_clean":"2024-05-27T10:18:42.915250+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:18:42.915250+0000","last_undegraded":"2024-05-27T10:18:42.915250+0000","last_fullsized":"2024-05-27T10:18:42.915250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":290,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:48.434451+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:18:48.434451+0000","last_peered":"2024-05-27T10:18:48.434451+0000","last_clean":"2024-05-27T10:18:48.434451+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:18:48.434451+0000","last_undegraded":"2024-05-27T10:18:48.434451+0000","last_fullsized":"2024-05-27T10:18:48.434451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914298+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:18:42.914298+0000","last_peered":"2024-05-27T10:18:42.914298+0000","last_clean":"2024-05-27T10:18:42.914298+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:18:42.914298+0000","last_undegraded":"2024-05-27T10:18:42.914298+0000","last_fullsized":"2024-05-27T10:18:42.914298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148905+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:18:43.148905+0000","last_peered":"2024-05-27T10:18:43.148905+0000","last_clean":"2024-05-27T10:18:43.148905+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:18:43.148905+0000","last_undegraded":"2024-05-27T10:18:43.148905+0000","last_fullsized":"2024-05-27T10:18:43.148905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028231+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:18:43.028231+0000","last_peered":"2024-05-27T10:18:43.028231+0000","last_clean":"2024-05-27T10:18:43.028231+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:18:43.028231+0000","last_undegraded":"2024-05-27T10:18:43.028231+0000","last_fullsized":"2024-05-27T10:18:43.028231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.641820+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:18:43.641820+0000","last_peered":"2024-05-27T10:18:43.641820+0000","last_clean":"2024-05-27T10:18:43.641820+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:18:43.641820+0000","last_undegraded":"2024-05-27T10:18:43.641820+0000","last_fullsized":"2024-05-27T10:18:43.641820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865929+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:18:42.865929+0000","last_peered":"2024-05-27T10:18:42.865929+0000","last_clean":"2024-05-27T10:18:42.865929+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:18:42.865929+0000","last_undegraded":"2024-05-27T10:18:42.865929+0000","last_fullsized":"2024-05-27T10:18:42.865929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010814+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:18:43.010814+0000","last_peered":"2024-05-27T10:18:43.010814+0000","last_clean":"2024-05-27T10:18:43.010814+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:18:43.010814+0000","last_undegraded":"2024-05-27T10:18:43.010814+0000","last_fullsized":"2024-05-27T10:18:43.010814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864668+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:18:42.864668+0000","last_peered":"2024-05-27T10:18:42.864668+0000","last_clean":"2024-05-27T10:18:42.864668+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:18:42.864668+0000","last_undegraded":"2024-05-27T10:18:42.864668+0000","last_fullsized":"2024-05-27T10:18:42.864668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914915+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:18:42.914915+0000","last_peered":"2024-05-27T10:18:42.914915+0000","last_clean":"2024-05-27T10:18:42.914915+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:18:42.914915+0000","last_undegraded":"2024-05-27T10:18:42.914915+0000","last_fullsized":"2024-05-27T10:18:42.914915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865657+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:18:42.865657+0000","last_peered":"2024-05-27T10:18:42.865657+0000","last_clean":"2024-05-27T10:18:42.865657+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:18:42.865657+0000","last_undegraded":"2024-05-27T10:18:42.865657+0000","last_fullsized":"2024-05-27T10:18:42.865657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028265+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:18:43.028265+0000","last_peered":"2024-05-27T10:18:43.028265+0000","last_clean":"2024-05-27T10:18:43.028265+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:18:43.028265+0000","last_undegraded":"2024-05-27T10:18:43.028265+0000","last_fullsized":"2024-05-27T10:18:43.028265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":283,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:47.430751+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:18:47.430751+0000","last_peered":"2024-05-27T10:18:47.430751+0000","last_clean":"2024-05-27T10:18:47.430751+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:18:47.430751+0000","last_undegraded":"2024-05-27T10:18:47.430751+0000","last_fullsized":"2024-05-27T10:18:47.430751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013516+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:18:43.013516+0000","last_peered":"2024-05-27T10:18:43.013516+0000","last_clean":"2024-05-27T10:18:43.013516+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:18:43.013516+0000","last_undegraded":"2024-05-27T10:18:43.013516+0000","last_fullsized":"2024-05-27T10:18:43.013516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036179+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:18:43.036179+0000","last_peered":"2024-05-27T10:18:43.036179+0000","last_clean":"2024-05-27T10:18:43.036179+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:18:43.036179+0000","last_undegraded":"2024-05-27T10:18:43.036179+0000","last_fullsized":"2024-05-27T10:18:43.036179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"82'99","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:45.052019+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:18:45.052019+0000","last_peered":"2024-05-27T10:18:45.052019+0000","last_clean":"2024-05-27T10:18:45.052019+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:18:45.052019+0000","last_undegraded":"2024-05-27T10:18:45.052019+0000","last_fullsized":"2024-05-27T10:18:45.052019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857424+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:18:42.857424+0000","last_peered":"2024-05-27T10:18:42.857424+0000","last_clean":"2024-05-27T10:18:42.857424+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:18:42.857424+0000","last_undegraded":"2024-05-27T10:18:42.857424+0000","last_fullsized":"2024-05-27T10:18:42.857424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914885+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:18:42.914885+0000","last_peered":"2024-05-27T10:18:42.914885+0000","last_clean":"2024-05-27T10:18:42.914885+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:18:42.914885+0000","last_undegraded":"2024-05-27T10:18:42.914885+0000","last_fullsized":"2024-05-27T10:18:42.914885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149983+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:18:43.149983+0000","last_peered":"2024-05-27T10:18:43.149983+0000","last_clean":"2024-05-27T10:18:43.149983+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:18:43.149983+0000","last_undegraded":"2024-05-27T10:18:43.149983+0000","last_fullsized":"2024-05-27T10:18:43.149983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858159+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:18:42.858159+0000","last_peered":"2024-05-27T10:18:42.858159+0000","last_clean":"2024-05-27T10:18:42.858159+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:18:42.858159+0000","last_undegraded":"2024-05-27T10:18:42.858159+0000","last_fullsized":"2024-05-27T10:18:42.858159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'49","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.228670+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:18:43.228670+0000","last_peered":"2024-05-27T10:18:43.228670+0000","last_clean":"2024-05-27T10:18:43.228670+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:18:43.228670+0000","last_undegraded":"2024-05-27T10:18:43.228670+0000","last_fullsized":"2024-05-27T10:18:43.228670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.014095+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:18:43.014095+0000","last_peered":"2024-05-27T10:18:43.014095+0000","last_clean":"2024-05-27T10:18:43.014095+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:18:43.014095+0000","last_undegraded":"2024-05-27T10:18:43.014095+0000","last_fullsized":"2024-05-27T10:18:43.014095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036143+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:18:43.036143+0000","last_peered":"2024-05-27T10:18:43.036143+0000","last_clean":"2024-05-27T10:18:43.036143+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:18:43.036143+0000","last_undegraded":"2024-05-27T10:18:43.036143+0000","last_fullsized":"2024-05-27T10:18:43.036143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035987+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:18:43.035987+0000","last_peered":"2024-05-27T10:18:43.035987+0000","last_clean":"2024-05-27T10:18:43.035987+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:18:43.035987+0000","last_undegraded":"2024-05-27T10:18:43.035987+0000","last_fullsized":"2024-05-27T10:18:43.035987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910322+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:18:42.910322+0000","last_peered":"2024-05-27T10:18:42.910322+0000","last_clean":"2024-05-27T10:18:42.910322+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:18:42.910322+0000","last_undegraded":"2024-05-27T10:18:42.910322+0000","last_fullsized":"2024-05-27T10:18:42.910322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148975+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:18:43.148975+0000","last_peered":"2024-05-27T10:18:43.148975+0000","last_clean":"2024-05-27T10:18:43.148975+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:18:43.148975+0000","last_undegraded":"2024-05-27T10:18:43.148975+0000","last_fullsized":"2024-05-27T10:18:43.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013173+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:18:43.013173+0000","last_peered":"2024-05-27T10:18:43.013173+0000","last_clean":"2024-05-27T10:18:43.013173+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:18:43.013173+0000","last_undegraded":"2024-05-27T10:18:43.013173+0000","last_fullsized":"2024-05-27T10:18:43.013173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865624+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:18:42.865624+0000","last_peered":"2024-05-27T10:18:42.865624+0000","last_clean":"2024-05-27T10:18:42.865624+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:18:42.865624+0000","last_undegraded":"2024-05-27T10:18:42.865624+0000","last_fullsized":"2024-05-27T10:18:42.865624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865382+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:18:42.865382+0000","last_peered":"2024-05-27T10:18:42.865382+0000","last_clean":"2024-05-27T10:18:42.865382+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:18:42.865382+0000","last_undegraded":"2024-05-27T10:18:42.865382+0000","last_fullsized":"2024-05-27T10:18:42.865382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013727+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:18:43.013727+0000","last_peered":"2024-05-27T10:18:43.013727+0000","last_clean":"2024-05-27T10:18:43.013727+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:18:43.013727+0000","last_undegraded":"2024-05-27T10:18:43.013727+0000","last_fullsized":"2024-05-27T10:18:43.013727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150814+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:18:43.150814+0000","last_peered":"2024-05-27T10:18:43.150814+0000","last_clean":"2024-05-27T10:18:43.150814+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:18:43.150814+0000","last_undegraded":"2024-05-27T10:18:43.150814+0000","last_fullsized":"2024-05-27T10:18:43.150814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915235+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:18:42.915235+0000","last_peered":"2024-05-27T10:18:42.915235+0000","last_clean":"2024-05-27T10:18:42.915235+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:18:42.915235+0000","last_undegraded":"2024-05-27T10:18:42.915235+0000","last_fullsized":"2024-05-27T10:18:42.915235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892674+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:18:42.892674+0000","last_peered":"2024-05-27T10:18:42.892674+0000","last_clean":"2024-05-27T10:18:42.892674+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:18:42.892674+0000","last_undegraded":"2024-05-27T10:18:42.892674+0000","last_fullsized":"2024-05-27T10:18:42.892674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916054+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:18:42.916054+0000","last_peered":"2024-05-27T10:18:42.916054+0000","last_clean":"2024-05-27T10:18:42.916054+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:18:42.916054+0000","last_undegraded":"2024-05-27T10:18:42.916054+0000","last_fullsized":"2024-05-27T10:18:42.916054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'39","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.226174+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:18:43.226174+0000","last_peered":"2024-05-27T10:18:43.226174+0000","last_clean":"2024-05-27T10:18:43.226174+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:18:43.226174+0000","last_undegraded":"2024-05-27T10:18:43.226174+0000","last_fullsized":"2024-05-27T10:18:43.226174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915511+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:18:42.915511+0000","last_peered":"2024-05-27T10:18:42.915511+0000","last_clean":"2024-05-27T10:18:42.915511+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:18:42.915511+0000","last_undegraded":"2024-05-27T10:18:42.915511+0000","last_fullsized":"2024-05-27T10:18:42.915511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858229+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:18:42.858229+0000","last_peered":"2024-05-27T10:18:42.858229+0000","last_clean":"2024-05-27T10:18:42.858229+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:18:42.858229+0000","last_undegraded":"2024-05-27T10:18:42.858229+0000","last_fullsized":"2024-05-27T10:18:42.858229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916095+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:18:42.916095+0000","last_peered":"2024-05-27T10:18:42.916095+0000","last_clean":"2024-05-27T10:18:42.916095+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:18:42.916095+0000","last_undegraded":"2024-05-27T10:18:42.916095+0000","last_fullsized":"2024-05-27T10:18:42.916095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865960+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:18:42.865960+0000","last_peered":"2024-05-27T10:18:42.865960+0000","last_clean":"2024-05-27T10:18:42.865960+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:18:42.865960+0000","last_undegraded":"2024-05-27T10:18:42.865960+0000","last_fullsized":"2024-05-27T10:18:42.865960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892084+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:18:42.892084+0000","last_peered":"2024-05-27T10:18:42.892084+0000","last_clean":"2024-05-27T10:18:42.892084+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:18:42.892084+0000","last_undegraded":"2024-05-27T10:18:42.892084+0000","last_fullsized":"2024-05-27T10:18:42.892084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.891669+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:18:42.891669+0000","last_peered":"2024-05-27T10:18:42.891669+0000","last_clean":"2024-05-27T10:18:42.891669+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:18:42.891669+0000","last_undegraded":"2024-05-27T10:18:42.891669+0000","last_fullsized":"2024-05-27T10:18:42.891669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036165+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:18:43.036165+0000","last_peered":"2024-05-27T10:18:43.036165+0000","last_clean":"2024-05-27T10:18:43.036165+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:18:43.036165+0000","last_undegraded":"2024-05-27T10:18:43.036165+0000","last_fullsized":"2024-05-27T10:18:43.036165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858813+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:18:42.858813+0000","last_peered":"2024-05-27T10:18:42.858813+0000","last_clean":"2024-05-27T10:18:42.858813+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:18:42.858813+0000","last_undegraded":"2024-05-27T10:18:42.858813+0000","last_fullsized":"2024-05-27T10:18:42.858813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865913+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:18:42.865913+0000","last_peered":"2024-05-27T10:18:42.865913+0000","last_clean":"2024-05-27T10:18:42.865913+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:18:42.865913+0000","last_undegraded":"2024-05-27T10:18:42.865913+0000","last_fullsized":"2024-05-27T10:18:42.865913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892758+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:18:42.892758+0000","last_peered":"2024-05-27T10:18:42.892758+0000","last_clean":"2024-05-27T10:18:42.892758+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:18:42.892758+0000","last_undegraded":"2024-05-27T10:18:42.892758+0000","last_fullsized":"2024-05-27T10:18:42.892758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010907+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:18:43.010907+0000","last_peered":"2024-05-27T10:18:43.010907+0000","last_clean":"2024-05-27T10:18:43.010907+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:18:43.010907+0000","last_undegraded":"2024-05-27T10:18:43.010907+0000","last_fullsized":"2024-05-27T10:18:43.010907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028084+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:18:43.028084+0000","last_peered":"2024-05-27T10:18:43.028084+0000","last_clean":"2024-05-27T10:18:43.028084+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:18:43.028084+0000","last_undegraded":"2024-05-27T10:18:43.028084+0000","last_fullsized":"2024-05-27T10:18:43.028084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013207+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:18:43.013207+0000","last_peered":"2024-05-27T10:18:43.013207+0000","last_clean":"2024-05-27T10:18:43.013207+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:18:43.013207+0000","last_undegraded":"2024-05-27T10:18:43.013207+0000","last_fullsized":"2024-05-27T10:18:43.013207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913669+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:18:42.913669+0000","last_peered":"2024-05-27T10:18:42.913669+0000","last_clean":"2024-05-27T10:18:42.913669+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:18:42.913669+0000","last_undegraded":"2024-05-27T10:18:42.913669+0000","last_fullsized":"2024-05-27T10:18:42.913669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858143+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:18:42.858143+0000","last_peered":"2024-05-27T10:18:42.858143+0000","last_clean":"2024-05-27T10:18:42.858143+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:18:42.858143+0000","last_undegraded":"2024-05-27T10:18:42.858143+0000","last_fullsized":"2024-05-27T10:18:42.858143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866758+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:18:42.866758+0000","last_peered":"2024-05-27T10:18:42.866758+0000","last_clean":"2024-05-27T10:18:42.866758+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:18:42.866758+0000","last_undegraded":"2024-05-27T10:18:42.866758+0000","last_fullsized":"2024-05-27T10:18:42.866758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599445,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1613,"num_read_kb":919,"num_write":305,"num_write_kb":1447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1798335,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":830200,"data_compressed":22039,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874321,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1228886,"data_compressed":45201,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037849,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":830436,"data_compressed":22039,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234081,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":437140,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430311,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1032293,"data_compressed":39045,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593839,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":888832,"data_stored":436609,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":503807,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":437880,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":507355,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149830,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":961736,"data_compressed":28195,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953600,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":438462,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1162161,"data_compressed":45201,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":135274,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:18:52.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:18:52.858 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:53.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 880 B/s wr, 16 op/s 2024-05-27T10:18:53.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:53.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:18:53.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:52 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 880 B/s wr, 16 op/s 2024-05-27T10:18:53.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:53.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:18:53.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:52 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 880 B/s wr, 16 op/s 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: Updating smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: Updating smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: Updating smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:53.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:18:53.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:18:53.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:18:53.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:52 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:18:54.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:53 smithi143 ceph-mon[35545]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:54.060 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:53 smithi042 ceph-mon[31661]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:54.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:53 smithi104 ceph-mon[34812]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:18:55.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:54 smithi042 ceph-mon[31661]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 684 B/s wr, 12 op/s 2024-05-27T10:18:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:54 smithi143 ceph-mon[35545]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 684 B/s wr, 12 op/s 2024-05-27T10:18:55.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:54 smithi104 ceph-mon[34812]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 684 B/s wr, 12 op/s 2024-05-27T10:18:55.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.376+0000 7f349fc00700 1 -- 172.21.15.42:0/87218078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498106560 msgr2=0x7f3498106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:55.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.376+0000 7f349fc00700 1 --2- 172.21.15.42:0/87218078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498106560 0x7f3498106970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3480009990 tx=0x7f3480009ca0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 -- 172.21.15.42:0/87218078 shutdown_connections 2024-05-27T10:18:55.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 --2- 172.21.15.42:0/87218078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106eb0 0x7f3498113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 --2- 172.21.15.42:0/87218078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349810c5a0 0x7f349810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 --2- 172.21.15.42:0/87218078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498106560 0x7f3498106970 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 -- 172.21.15.42:0/87218078 >> 172.21.15.42:0/87218078 conn(0x7f3498102040 msgr2=0x7f3498104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:55.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 -- 172.21.15.42:0/87218078 shutdown_connections 2024-05-27T10:18:55.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.377+0000 7f349fc00700 1 -- 172.21.15.42:0/87218078 wait complete. 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.378+0000 7f349fc00700 1 Processor -- start 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.378+0000 7f349fc00700 1 -- start start 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349fc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106560 0x7f349819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349fc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3498106eb0 0x7f349819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349fc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 0x7f34981a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349fc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3498116580 con 0x7f349810c5a0 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349fc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3498116400 con 0x7f3498106560 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349fc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3498116700 con 0x7f3498106eb0 2024-05-27T10:18:55.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 0x7f34981a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:55.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 0x7f34981a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46900/0 (socket says 172.21.15.42:46900) 2024-05-27T10:18:55.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349dc00700 1 -- 172.21.15.42:0/371120013 learned_addr learned my addr 172.21.15.42:0/371120013 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:55.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f349d200700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106560 0x7f349819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:55.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.379+0000 7f348fe00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3498106eb0 0x7f349819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349dc00700 1 -- 172.21.15.42:0/371120013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3498106eb0 msgr2=0x7f349819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349dc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3498106eb0 0x7f349819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349dc00700 1 -- 172.21.15.42:0/371120013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106560 msgr2=0x7f349819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349dc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106560 0x7f349819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349dc00700 1 -- 172.21.15.42:0/371120013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34981a1d50 con 0x7f349810c5a0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349d200700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106560 0x7f349819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349dc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 0x7f34981a1760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3490007ee0 tx=0x7f349000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349000dbd0 con 0x7f349810c5a0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f349000dd70 con 0x7f349810c5a0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34981a2040 con 0x7f349810c5a0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900058d0 con 0x7f349810c5a0 2024-05-27T10:18:55.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.380+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34981a2530 con 0x7f349810c5a0 2024-05-27T10:18:55.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.382+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34980512b0 con 0x7f349810c5a0 2024-05-27T10:18:55.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.385+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f349001a020 con 0x7f349810c5a0 2024-05-27T10:18:55.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.386+0000 7f3496000700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f347c0b2b10 0x7f347c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:55.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.386+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f34900e2780 con 0x7f349810c5a0 2024-05-27T10:18:55.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.386+0000 7f349d200700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f347c0b2b10 0x7f347c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:55.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.387+0000 7f349d200700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f347c0b2b10 0x7f347c0b5020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3488005fd0 tx=0x7f3488005e30 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:55.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.389+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3490010070 con 0x7f349810c5a0 2024-05-27T10:18:55.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.678+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f349802e5e0 con 0x7f349810c5a0 2024-05-27T10:18:55.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.681+0000 7f3496000700 1 -- 172.21.15.42:0/371120013 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f34900a9f60 con 0x7f349810c5a0 2024-05-27T10:18:55.682 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:18:55.682 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:18:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.684+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f347c0b2b10 msgr2=0x7f347c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.684+0000 7f349fc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f347c0b2b10 0x7f347c0b5020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3488005fd0 tx=0x7f3488005e30 comp rx=0 tx=0).stop 2024-05-27T10:18:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.684+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 msgr2=0x7f34981a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.684+0000 7f349fc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 0x7f34981a1760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3490007ee0 tx=0x7f349000d3b0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 shutdown_connections 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f347c0b2b10 0x7f347c0b5020 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349810c5a0 0x7f34981a1760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3498106560 0x7f349819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 --2- 172.21.15.42:0/371120013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3498106eb0 0x7f349819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 >> 172.21.15.42:0/371120013 conn(0x7f3498102040 msgr2=0x7f34981038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 shutdown_connections 2024-05-27T10:18:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:55.685+0000 7f349fc00700 1 -- 172.21.15.42:0/371120013 wait complete. 2024-05-27T10:18:55.687 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:18:56.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:55 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/371120013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:56.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:55 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/371120013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:56.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:55 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/371120013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:18:56.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:18:57.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:56 smithi143 ceph-mon[35545]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 559 B/s wr, 13 op/s 2024-05-27T10:18:57.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:56 smithi042 ceph-mon[31661]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 559 B/s wr, 13 op/s 2024-05-27T10:18:57.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:56 smithi104 ceph-mon[34812]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 559 B/s wr, 13 op/s 2024-05-27T10:18:57.200 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:18:59.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:18:58 smithi143 ceph-mon[35545]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T10:18:59.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:18:58 smithi042 ceph-mon[31661]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T10:18:59.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:18:58 smithi104 ceph-mon[34812]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T10:18:59.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 -- 172.21.15.42:0/1524312740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 msgr2=0x7fbef810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 --2- 172.21.15.42:0/1524312740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef810abc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009990 tx=0x7fbee4009ca0 comp rx=0 tx=0).stop 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 -- 172.21.15.42:0/1524312740 shutdown_connections 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 --2- 172.21.15.42:0/1524312740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef810b100 0x7fbef8117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 --2- 172.21.15.42:0/1524312740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810f6b0 0x7fbef810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 --2- 172.21.15.42:0/1524312740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef810abc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 -- 172.21.15.42:0/1524312740 >> 172.21.15.42:0/1524312740 conn(0x7fbef8076af0 msgr2=0x7fbef8076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 -- 172.21.15.42:0/1524312740 shutdown_connections 2024-05-27T10:18:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.844+0000 7fbef7200700 1 -- 172.21.15.42:0/1524312740 wait complete. 2024-05-27T10:18:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 Processor -- start 2024-05-27T10:18:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 -- start start 2024-05-27T10:18:59.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef81a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:59.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810b100 0x7fbef81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:59.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef810f6b0 0x7fbef81a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:59.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbef811a7a0 con 0x7fbef810a7b0 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbef811a620 con 0x7fbef810f6b0 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.846+0000 7fbef7200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbef811a920 con 0x7fbef810b100 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef81a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef81a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46916/0 (socket says 172.21.15.42:46916) 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 -- 172.21.15.42:0/3396319625 learned_addr learned my addr 172.21.15.42:0/3396319625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5400700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810b100 0x7fbef81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef6800700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef810f6b0 0x7fbef81a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 -- 172.21.15.42:0/3396319625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810b100 msgr2=0x7fbef81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810b100 0x7fbef81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 -- 172.21.15.42:0/3396319625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef810f6b0 msgr2=0x7fbef81a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef810f6b0 0x7fbef81a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5e00700 1 -- 172.21.15.42:0/3396319625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef81a6040 con 0x7fbef810a7b0 2024-05-27T10:18:59.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.847+0000 7fbef5400700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810b100 0x7fbef81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:18:59.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.848+0000 7fbef5e00700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef81a1170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbee800d950 tx=0x7fbee800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.848+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8016070 con 0x7fbef810a7b0 2024-05-27T10:18:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.848+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbee8010460 con 0x7fbef810a7b0 2024-05-27T10:18:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.848+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee800f620 con 0x7fbef810a7b0 2024-05-27T10:18:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.848+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef81a6330 con 0x7fbef810a7b0 2024-05-27T10:18:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.848+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef81adc50 con 0x7fbef810a7b0 2024-05-27T10:18:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.850+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef80512b0 con 0x7fbef810a7b0 2024-05-27T10:18:59.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.852+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fbee8019070 con 0x7fbef810a7b0 2024-05-27T10:18:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.854+0000 7fbeeea00700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbedc0b2b90 0x7fbedc0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:18:59.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.854+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbee80e2410 con 0x7fbef810a7b0 2024-05-27T10:18:59.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.854+0000 7fbef5400700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbedc0b2b90 0x7fbedc0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:18:59.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.855+0000 7fbef5400700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbedc0b2b90 0x7fbedc0b50a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbee4003760 tx=0x7fbee4005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:18:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:18:59.857+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbee80a9b90 con 0x7fbef810a7b0 2024-05-27T10:19:00.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.096+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbef802e5e0 con 0x7fbef810a7b0 2024-05-27T10:19:00.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.103+0000 7fbeeea00700 1 -- 172.21.15.42:0/3396319625 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22940 (secure 0 0 0) 0x7fbee80a9490 con 0x7fbef810a7b0 2024-05-27T10:19:00.105 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:00.105 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":82,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:18:42.838286+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1953650070":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2667703980":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2970531755":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/1310078254":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:19:00.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbedc0b2b90 msgr2=0x7fbedc0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:00.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbedc0b2b90 0x7fbedc0b50a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbee4003760 tx=0x7fbee4005fd0 comp rx=0 tx=0).stop 2024-05-27T10:19:00.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 msgr2=0x7fbef81a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:00.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef81a1170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbee800d950 tx=0x7fbee800dd10 comp rx=0 tx=0).stop 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 shutdown_connections 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbedc0b2b90 0x7fbedc0b50a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef810f6b0 0x7fbef81a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef810a7b0 0x7fbef81a1170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 --2- 172.21.15.42:0/3396319625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef810b100 0x7fbef81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 >> 172.21.15.42:0/3396319625 conn(0x7fbef8076af0 msgr2=0x7fbef81088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 shutdown_connections 2024-05-27T10:19:00.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:00.107+0000 7fbef7200700 1 -- 172.21.15.42:0/3396319625 wait complete. 2024-05-27T10:19:01.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:00 smithi042 ceph-mon[31661]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T10:19:01.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:00 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3396319625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:19:01.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:00 smithi143 ceph-mon[35545]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T10:19:01.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:00 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3396319625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:19:01.119 DEBUG:tasks.fwd_scrub:Yielding 2024-05-27T10:19:01.119 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T10:19:01.129 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T10:19:01.129 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:19:01.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-27T10:19:01.130 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-27T10:19:01.130 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T10:19:01.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T10:19:01.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:00 smithi104 ceph-mon[34812]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T10:19:01.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:00 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3396319625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 -- 172.21.15.42:0/2783866991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3010a140 msgr2=0x7f3f30103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 --2- 172.21.15.42:0/2783866991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3010a140 0x7f3f30103d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009900 tx=0x7f3f1802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 -- 172.21.15.42:0/2783866991 shutdown_connections 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 --2- 172.21.15.42:0/2783866991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f301043f0 0x7f3f301107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 --2- 172.21.15.42:0/2783866991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3010a140 0x7f3f30103d80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 --2- 172.21.15.42:0/2783866991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f30109ca0 0x7f3f30103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 -- 172.21.15.42:0/2783866991 >> 172.21.15.42:0/2783866991 conn(0x7f3f30079390 msgr2=0x7f3f300797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 -- 172.21.15.42:0/2783866991 shutdown_connections 2024-05-27T10:19:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.411+0000 7f3f35e00640 1 -- 172.21.15.42:0/2783866991 wait complete. 2024-05-27T10:19:01.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.412+0000 7f3f35e00640 1 Processor -- start 2024-05-27T10:19:01.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 -- start start 2024-05-27T10:19:01.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f301043f0 0x7f3f30074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 0x7f3f30075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3010a140 0x7f3f30075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113d40 con 0x7f3f30109ca0 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113bc0 con 0x7f3f301043f0 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.413+0000 7f3f35e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113ec0 con 0x7f3f3010a140 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 0x7f3f30075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 0x7f3f30075500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46926/0 (socket says 172.21.15.42:46926) 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2e000640 1 -- 172.21.15.42:0/893045462 learned_addr learned my addr 172.21.15.42:0/893045462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2ea00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f301043f0 0x7f3f30074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2f400640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3010a140 0x7f3f30075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2e000640 1 -- 172.21.15.42:0/893045462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3010a140 msgr2=0x7f3f30075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2e000640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3010a140 0x7f3f30075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.414+0000 7f3f2e000640 1 -- 172.21.15.42:0/893045462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f301043f0 msgr2=0x7f3f30074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f2e000640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f301043f0 0x7f3f30074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f2e000640 1 -- 172.21.15.42:0/893045462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f301a8a30 con 0x7f3f30109ca0 2024-05-27T10:19:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f2f400640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3010a140 0x7f3f30075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f2e000640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 0x7f3f30075500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f18005940 tx=0x7f3f180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1803f030 con 0x7f3f30109ca0 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f18003c90 con 0x7f3f30109ca0 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18035070 con 0x7f3f30109ca0 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f301a8cc0 con 0x7f3f30109ca0 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.415+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a91b0 con 0x7f3f30109ca0 2024-05-27T10:19:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.417+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3006a230 con 0x7f3f30109ca0 2024-05-27T10:19:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.419+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f3f18042030 con 0x7f3f30109ca0 2024-05-27T10:19:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.422+0000 7f3f34a00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3f100b2a70 0x7f3f100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.422+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3f181032d0 con 0x7f3f30109ca0 2024-05-27T10:19:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.422+0000 7f3f2ea00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3f100b2a70 0x7f3f100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.423+0000 7f3f2ea00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3f100b2a70 0x7f3f100b4f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f30104980 tx=0x7f3f2000a2d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:01.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.426+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f180cab20 con 0x7f3f30109ca0 2024-05-27T10:19:01.611 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:01.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.647+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f3f301a93f0 con 0x7f3f30109ca0 2024-05-27T10:19:01.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.653+0000 7f3f34a00640 1 -- 172.21.15.42:0/893045462 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f3f180ca420 con 0x7f3f30109ca0 2024-05-27T10:19:01.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3f100b2a70 msgr2=0x7f3f100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3f100b2a70 0x7f3f100b4f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f30104980 tx=0x7f3f2000a2d0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 msgr2=0x7f3f30075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 0x7f3f30075500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f18005940 tx=0x7f3f180042f0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 shutdown_connections 2024-05-27T10:19:01.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3f100b2a70 0x7f3f100b4f80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3010a140 0x7f3f30075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f30109ca0 0x7f3f30075500 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 --2- 172.21.15.42:0/893045462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f301043f0 0x7f3f30074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 >> 172.21.15.42:0/893045462 conn(0x7f3f30079390 msgr2=0x7f3f30102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:01.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 shutdown_connections 2024-05-27T10:19:01.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.657+0000 7f3f35e00640 1 -- 172.21.15.42:0/893045462 wait complete. 2024-05-27T10:19:01.684 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-27T10:19:01.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 -- 172.21.15.42:0/1885272127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 msgr2=0x7f8080117db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 --2- 172.21.15.42:0/1885272127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f8080117db0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8078009900 tx=0x7f807802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 -- 172.21.15.42:0/1885272127 shutdown_connections 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 --2- 172.21.15.42:0/1885272127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f8080117db0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 --2- 172.21.15.42:0/1885272127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f808010fb50 0x7f808010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 --2- 172.21.15.42:0/1885272127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808010b5c0 0x7f808010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 -- 172.21.15.42:0/1885272127 >> 172.21.15.42:0/1885272127 conn(0x7f808006e080 msgr2=0x7f80800704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 -- 172.21.15.42:0/1885272127 shutdown_connections 2024-05-27T10:19:01.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.949+0000 7f8085400640 1 -- 172.21.15.42:0/1885272127 wait complete. 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.950+0000 7f8085400640 1 Processor -- start 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 -- start start 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808010b5c0 0x7f80801a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f808010fb50 0x7f80801a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f80801a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f808011c4f0 con 0x7f8080110610 2024-05-27T10:19:01.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f808011c370 con 0x7f808010fb50 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f8085400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f808011c670 con 0x7f808010b5c0 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f807f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f80801a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f807f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f80801a77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46950/0 (socket says 172.21.15.42:46950) 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.951+0000 7f807f200640 1 -- 172.21.15.42:0/1246600761 learned_addr learned my addr 172.21.15.42:0/1246600761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807e800640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808010b5c0 0x7f80801a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807de00640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f808010fb50 0x7f80801a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807f200640 1 -- 172.21.15.42:0/1246600761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808010b5c0 msgr2=0x7f80801a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807f200640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808010b5c0 0x7f80801a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807f200640 1 -- 172.21.15.42:0/1246600761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f808010fb50 msgr2=0x7f80801a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807f200640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f808010fb50 0x7f80801a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807f200640 1 -- 172.21.15.42:0/1246600761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80801a7e10 con 0x7f8080110610 2024-05-27T10:19:01.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.952+0000 7f807f200640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f80801a77d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8078004770 tx=0x7f8078004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:01.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.953+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807803f030 con 0x7f8080110610 2024-05-27T10:19:01.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.953+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8078004440 con 0x7f8080110610 2024-05-27T10:19:01.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.953+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078035070 con 0x7f8080110610 2024-05-27T10:19:01.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.953+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a80a0 con 0x7f8080110610 2024-05-27T10:19:01.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.953+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a8590 con 0x7f8080110610 2024-05-27T10:19:01.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.954+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8080050860 con 0x7f8080110610 2024-05-27T10:19:01.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.957+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f8078042030 con 0x7f8080110610 2024-05-27T10:19:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.959+0000 7f8077400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f80640b2a70 0x7f80640b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:01.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.959+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8078103090 con 0x7f8080110610 2024-05-27T10:19:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.959+0000 7f807e800640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f80640b2a70 0x7f80640b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:01.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.960+0000 7f807e800640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f80640b2a70 0x7f80640b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8070005fd0 tx=0x7f8070009680 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:01.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:01.962+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80780ca8e0 con 0x7f8080110610 2024-05-27T10:19:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.191+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f80801133d0 con 0x7f8080110610 2024-05-27T10:19:02.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.199+0000 7f8077400640 1 -- 172.21.15.42:0/1246600761 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f80780ca1e0 con 0x7f8080110610 2024-05-27T10:19:02.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f80640b2a70 msgr2=0x7f80640b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f80640b2a70 0x7f80640b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8070005fd0 tx=0x7f8070009680 comp rx=0 tx=0).stop 2024-05-27T10:19:02.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 msgr2=0x7f80801a77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f80801a77d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8078004770 tx=0x7f8078004040 comp rx=0 tx=0).stop 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 shutdown_connections 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f80640b2a70 0x7f80640b4f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8080110610 0x7f80801a77d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f808010fb50 0x7f80801a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 --2- 172.21.15.42:0/1246600761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808010b5c0 0x7f80801a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 >> 172.21.15.42:0/1246600761 conn(0x7f808006e080 msgr2=0x7f808006e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 shutdown_connections 2024-05-27T10:19:02.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.202+0000 7f8085400640 1 -- 172.21.15.42:0/1246600761 wait complete. 2024-05-27T10:19:02.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 -- 172.21.15.42:0/1481616720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c110110 msgr2=0x7f0a3c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 --2- 172.21.15.42:0/1481616720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c110110 0x7f0a3c109ca0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009900 tx=0x7f0a2402a480 comp rx=0 tx=0).stop 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 -- 172.21.15.42:0/1481616720 shutdown_connections 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 --2- 172.21.15.42:0/1481616720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c10a890 0x7f0a3c116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 --2- 172.21.15.42:0/1481616720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a3c110520 0x7f0a3c10a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 --2- 172.21.15.42:0/1481616720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c110110 0x7f0a3c109ca0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 -- 172.21.15.42:0/1481616720 >> 172.21.15.42:0/1481616720 conn(0x7f0a3c076070 msgr2=0x7f0a3c078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 -- 172.21.15.42:0/1481616720 shutdown_connections 2024-05-27T10:19:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.507+0000 7f0a3be00640 1 -- 172.21.15.42:0/1481616720 wait complete. 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.508+0000 7f0a3be00640 1 Processor -- start 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.508+0000 7f0a3be00640 1 -- start start 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a3c110110 0x7f0a3c1a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c110520 0x7f0a3c1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 0x7f0a3c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c11a160 con 0x7f0a3c1a0690 2024-05-27T10:19:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c119fe0 con 0x7f0a3c110110 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c11a2e0 con 0x7f0a3c110520 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 0x7f0a3c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 0x7f0a3c1a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46968/0 (socket says 172.21.15.42:46968) 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3b400640 1 -- 172.21.15.42:0/659538885 learned_addr learned my addr 172.21.15.42:0/659538885 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3aa00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a3c110110 0x7f0a3c1a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.509+0000 7f0a3a000640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c110520 0x7f0a3c1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3b400640 1 -- 172.21.15.42:0/659538885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c110520 msgr2=0x7f0a3c1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3b400640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c110520 0x7f0a3c1a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3b400640 1 -- 172.21.15.42:0/659538885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a3c110110 msgr2=0x7f0a3c1a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3b400640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a3c110110 0x7f0a3c1a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3b400640 1 -- 172.21.15.42:0/659538885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a1180 con 0x7f0a3c1a0690 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3b400640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 0x7f0a3c1a0aa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400eaa0 tx=0x7f0a3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3400cdb0 con 0x7f0a3c1a0690 2024-05-27T10:19:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a34005020 con 0x7f0a3c1a0690 2024-05-27T10:19:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340116d0 con 0x7f0a3c1a0690 2024-05-27T10:19:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.510+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a1470 con 0x7f0a3c1a0690 2024-05-27T10:19:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.511+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1ad550 con 0x7f0a3c1a0690 2024-05-27T10:19:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.512+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c10b390 con 0x7f0a3c1a0690 2024-05-27T10:19:02.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.515+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0a340051d0 con 0x7f0a3c1a0690 2024-05-27T10:19:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.517+0000 7f0a40a00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0a200b2af0 0x7f0a200b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.518+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0a340e2e80 con 0x7f0a3c1a0690 2024-05-27T10:19:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.518+0000 7f0a3aa00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0a200b2af0 0x7f0a200b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.519+0000 7f0a3aa00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0a200b2af0 0x7f0a200b5000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006010 tx=0x7f0a24032ba0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.520+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a34016030 con 0x7f0a3c1a0690 2024-05-27T10:19:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.744+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f0a3c050860 con 0x7f0a3c1a0690 2024-05-27T10:19:02.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.753+0000 7f0a40a00640 1 -- 172.21.15.42:0/659538885 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f0a3c050860 con 0x7f0a3c1a0690 2024-05-27T10:19:02.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.756+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0a200b2af0 msgr2=0x7f0a200b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.756+0000 7f0a3be00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0a200b2af0 0x7f0a200b5000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006010 tx=0x7f0a24032ba0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.756+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 msgr2=0x7f0a3c1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.756+0000 7f0a3be00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 0x7f0a3c1a0aa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400eaa0 tx=0x7f0a3400ef60 comp rx=0 tx=0).stop 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 shutdown_connections 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0a200b2af0 0x7f0a200b5000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a3c1a0690 0x7f0a3c1a0aa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c110520 0x7f0a3c1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 --2- 172.21.15.42:0/659538885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a3c110110 0x7f0a3c1a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 >> 172.21.15.42:0/659538885 conn(0x7f0a3c076070 msgr2=0x7f0a3c1158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 shutdown_connections 2024-05-27T10:19:02.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:02.757+0000 7f0a3be00640 1 -- 172.21.15.42:0/659538885 wait complete. 2024-05-27T10:19:02.789 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-27T10:19:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: pgmap v12: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-27T10:19:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/893045462' entity='client.admin' 2024-05-27T10:19:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:02.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1246600761' entity='client.admin' 2024-05-27T10:19:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:02.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:02 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: pgmap v12: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/893045462' entity='client.admin' 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1246600761' entity='client.admin' 2024-05-27T10:19:03.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:03.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:03.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:03.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:02 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:03.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 -- 172.21.15.42:0/2150362445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d410a890 msgr2=0x7ff9d4116b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 --2- 172.21.15.42:0/2150362445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d410a890 0x7ff9d4116b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009ca0 tx=0x7ff9cc02a950 comp rx=0 tx=0).stop 2024-05-27T10:19:03.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 -- 172.21.15.42:0/2150362445 shutdown_connections 2024-05-27T10:19:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 --2- 172.21.15.42:0/2150362445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d410a890 0x7ff9d4116b50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 --2- 172.21.15.42:0/2150362445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d4110520 0x7ff9d410a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 --2- 172.21.15.42:0/2150362445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d4110110 0x7ff9d4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 -- 172.21.15.42:0/2150362445 >> 172.21.15.42:0/2150362445 conn(0x7ff9d4076070 msgr2=0x7ff9d4078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.068+0000 7ff9da400640 1 -- 172.21.15.42:0/2150362445 shutdown_connections 2024-05-27T10:19:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.069+0000 7ff9da400640 1 -- 172.21.15.42:0/2150362445 wait complete. 2024-05-27T10:19:03.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 Processor -- start 2024-05-27T10:19:03.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 -- start start 2024-05-27T10:19:03.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 0x7ff9d41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d4110520 0x7ff9d41a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d41a4f30 0x7ff9d41a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d411a1d0 con 0x7ff9d4110110 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d411a050 con 0x7ff9d4110520 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.070+0000 7ff9da400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d411a350 con 0x7ff9d41a4f30 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 0x7ff9d41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 0x7ff9d41a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42792/0 (socket says 172.21.15.42:42792) 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9000640 1 -- 172.21.15.42:0/1624022881 learned_addr learned my addr 172.21.15.42:0/1624022881 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9a00640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d41a4f30 0x7ff9d41a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d3e00640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d4110520 0x7ff9d41a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9a00640 1 -- 172.21.15.42:0/1624022881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d4110520 msgr2=0x7ff9d41a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9a00640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d4110520 0x7ff9d41a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9a00640 1 -- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 msgr2=0x7ff9d41a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9a00640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 0x7ff9d41a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9a00640 1 -- 172.21.15.42:0/1624022881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d41a5980 con 0x7ff9d41a4f30 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.071+0000 7ff9d9000640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 0x7ff9d41a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.072+0000 7ff9d9a00640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d41a4f30 0x7ff9d41a5340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005e30 tx=0x7ff9cc003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.072+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc034040 con 0x7ff9d41a4f30 2024-05-27T10:19:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.072+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d41a5c10 con 0x7ff9d41a4f30 2024-05-27T10:19:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.072+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9cc030850 con 0x7ff9d41a4f30 2024-05-27T10:19:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.072+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0045a0 con 0x7ff9d41a4f30 2024-05-27T10:19:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.072+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d41ad410 con 0x7ff9d41a4f30 2024-05-27T10:19:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.074+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d410b390 con 0x7ff9d41a4f30 2024-05-27T10:19:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.076+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7ff9cc037020 con 0x7ff9d41a4f30 2024-05-27T10:19:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.078+0000 7ff9d1600640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff9bc0b2a70 0x7ff9bc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.078+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff9cc103870 con 0x7ff9d41a4f30 2024-05-27T10:19:03.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.078+0000 7ff9d9000640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff9bc0b2a70 0x7ff9bc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.079+0000 7ff9d9000640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff9bc0b2a70 0x7ff9bc0b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80058c0 tx=0x7ff9c8005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:03.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.084+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9cc0cb100 con 0x7ff9d41a4f30 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: pgmap v12: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/893045462' entity='client.admin' 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1246600761' entity='client.admin' 2024-05-27T10:19:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:03.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:03.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:03.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:02 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.328+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff9d410ef10 con 0x7ff9bc0b2a70 2024-05-27T10:19:03.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.517+0000 7ff9d1600640 1 -- 172.21.15.42:0/1624022881 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff9d410ef10 con 0x7ff9bc0b2a70 2024-05-27T10:19:03.518 INFO:teuthology.orchestra.run.smithi042.stdout:Schedule set for path / 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff9bc0b2a70 msgr2=0x7ff9bc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff9bc0b2a70 0x7ff9bc0b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80058c0 tx=0x7ff9c8005f90 comp rx=0 tx=0).stop 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d41a4f30 msgr2=0x7ff9d41a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d41a4f30 0x7ff9d41a5340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005e30 tx=0x7ff9cc003ce0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 shutdown_connections 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff9bc0b2a70 0x7ff9bc0b4f80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9d41a4f30 0x7ff9d41a5340 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d4110520 0x7ff9d41a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 --2- 172.21.15.42:0/1624022881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9d4110110 0x7ff9d41a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 >> 172.21.15.42:0/1624022881 conn(0x7ff9d4076070 msgr2=0x7ff9d410d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:03.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 shutdown_connections 2024-05-27T10:19:03.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.520+0000 7ff9da400640 1 -- 172.21.15.42:0/1624022881 wait complete. 2024-05-27T10:19:03.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-27T10:19:03.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.818+0000 7f0bed800640 1 -- 172.21.15.42:0/1394180528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be81102f0 msgr2=0x7f0be8112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.818+0000 7f0bed800640 1 --2- 172.21.15.42:0/1394180528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be81102f0 0x7f0be8112710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009900 tx=0x7f0be002a480 comp rx=0 tx=0).stop 2024-05-27T10:19:03.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 -- 172.21.15.42:0/1394180528 shutdown_connections 2024-05-27T10:19:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 --2- 172.21.15.42:0/1394180528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be81102f0 0x7f0be8112710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 --2- 172.21.15.42:0/1394180528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0be810d990 0x7f0be810fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 --2- 172.21.15.42:0/1394180528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be8075480 0x7f0be8075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 -- 172.21.15.42:0/1394180528 >> 172.21.15.42:0/1394180528 conn(0x7f0be8079240 msgr2=0x7f0be8079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 -- 172.21.15.42:0/1394180528 shutdown_connections 2024-05-27T10:19:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.819+0000 7f0bed800640 1 -- 172.21.15.42:0/1394180528 wait complete. 2024-05-27T10:19:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.820+0000 7f0bed800640 1 Processor -- start 2024-05-27T10:19:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.820+0000 7f0bed800640 1 -- start start 2024-05-27T10:19:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.820+0000 7f0bed800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 0x7f0be819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.820+0000 7f0bed800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0be810d990 0x7f0be819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0bed800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be81102f0 0x7f0be81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0bed800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8115d30 con 0x7f0be8075480 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0bed800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8115bb0 con 0x7f0be81102f0 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0bed800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8115eb0 con 0x7f0be810d990 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 0x7f0be819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 0x7f0be819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42816/0 (socket says 172.21.15.42:42816) 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 -- 172.21.15.42:0/1076809131 learned_addr learned my addr 172.21.15.42:0/1076809131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7400640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0be810d990 0x7f0be819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0bece00640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be81102f0 0x7f0be81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 -- 172.21.15.42:0/1076809131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0be810d990 msgr2=0x7f0be819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0be810d990 0x7f0be819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 -- 172.21.15.42:0/1076809131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be81102f0 msgr2=0x7f0be81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be81102f0 0x7f0be81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0be7e00640 1 -- 172.21.15.42:0/1076809131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be81a1650 con 0x7f0be8075480 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.821+0000 7f0bece00640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be81102f0 0x7f0be81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.822+0000 7f0be7e00640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 0x7f0be819c7f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ea80 tx=0x7f0bd800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.822+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800ce40 con 0x7f0be8075480 2024-05-27T10:19:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.822+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd8011900 con 0x7f0be8075480 2024-05-27T10:19:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.822+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be81a1940 con 0x7f0be8075480 2024-05-27T10:19:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.822+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd801a9e0 con 0x7f0be8075480 2024-05-27T10:19:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.822+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81a92c0 con 0x7f0be8075480 2024-05-27T10:19:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.825+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be806a230 con 0x7f0be8075480 2024-05-27T10:19:03.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.829+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0bd801ab80 con 0x7f0be8075480 2024-05-27T10:19:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.830+0000 7f0be4c00640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0bcc0b2a70 0x7f0bcc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.830+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0bd80e30f0 con 0x7f0be8075480 2024-05-27T10:19:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.830+0000 7f0be7400640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0bcc0b2a70 0x7f0bcc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.831+0000 7f0be7400640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0bcc0b2a70 0x7f0bcc0b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005fd0 tx=0x7f0bd000b510 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:03.832+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0bd8056df0 con 0x7f0be8075480 2024-05-27T10:19:03.909 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:03 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/659538885' entity='client.admin' 2024-05-27T10:19:03.909 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:03 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:03.909 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:03 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:03.910 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:03 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:03.910 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:03 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:04.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:03 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/659538885' entity='client.admin' 2024-05-27T10:19:04.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:03 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:04.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:03 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:04.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:03 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:04.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:03 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:04.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.086+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f0be8102930 con 0x7f0bcc0b2a70 2024-05-27T10:19:04.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:03 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/659538885' entity='client.admin' 2024-05-27T10:19:04.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:03 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:19:04.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:03 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:19:04.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:03 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:19:04.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:03 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:19:04.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.259+0000 7f0be4c00640 1 -- 172.21.15.42:0/1076809131 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f0be8102930 con 0x7f0bcc0b2a70 2024-05-27T10:19:04.260 INFO:teuthology.orchestra.run.smithi042.stdout:Retention added to path / 2024-05-27T10:19:04.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0bcc0b2a70 msgr2=0x7f0bcc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0bcc0b2a70 0x7f0bcc0b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005fd0 tx=0x7f0bd000b510 comp rx=0 tx=0).stop 2024-05-27T10:19:04.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 msgr2=0x7f0be819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 0x7f0be819c7f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ea80 tx=0x7f0bd800ef40 comp rx=0 tx=0).stop 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 shutdown_connections 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0bcc0b2a70 0x7f0bcc0b4f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0be81102f0 0x7f0be81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0be810d990 0x7f0be819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 --2- 172.21.15.42:0/1076809131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0be8075480 0x7f0be819c7f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 >> 172.21.15.42:0/1076809131 conn(0x7f0be8079240 msgr2=0x7f0be8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 shutdown_connections 2024-05-27T10:19:04.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.262+0000 7f0bed800640 1 -- 172.21.15.42:0/1076809131 wait complete. 2024-05-27T10:19:04.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T10:19:04.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 -- 172.21.15.42:0/2192214853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8075d20 msgr2=0x7f00c8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 --2- 172.21.15.42:0/2192214853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8075d20 0x7f00c8076130 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009990 tx=0x7f00bc009ca0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 -- 172.21.15.42:0/2192214853 shutdown_connections 2024-05-27T10:19:04.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 --2- 172.21.15.42:0/2192214853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c8110b90 0x7f00c8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 --2- 172.21.15.42:0/2192214853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8075d20 0x7f00c8076130 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 --2- 172.21.15.42:0/2192214853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8076670 0x7f00c8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 -- 172.21.15.42:0/2192214853 >> 172.21.15.42:0/2192214853 conn(0x7f00c8100030 msgr2=0x7f00c8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 -- 172.21.15.42:0/2192214853 shutdown_connections 2024-05-27T10:19:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.335+0000 7f00d0800700 1 -- 172.21.15.42:0/2192214853 wait complete. 2024-05-27T10:19:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.336+0000 7f00d0800700 1 Processor -- start 2024-05-27T10:19:04.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 -- start start 2024-05-27T10:19:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8075d20 0x7f00c819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c8076670 0x7f00c819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 0x7f00c81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8116580 con 0x7f00c8110b90 2024-05-27T10:19:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8116400 con 0x7f00c8075d20 2024-05-27T10:19:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00d0800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8116700 con 0x7f00c8076670 2024-05-27T10:19:04.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00ce800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 0x7f00c81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.337+0000 7f00ce800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 0x7f00c81a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42828/0 (socket says 172.21.15.42:42828) 2024-05-27T10:19:04.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 -- 172.21.15.42:0/3743768153 learned_addr learned my addr 172.21.15.42:0/3743768153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:04.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00cde00700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8075d20 0x7f00c819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00cd400700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c8076670 0x7f00c819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 -- 172.21.15.42:0/3743768153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c8076670 msgr2=0x7f00c819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c8076670 0x7f00c819d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 -- 172.21.15.42:0/3743768153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8075d20 msgr2=0x7f00c819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8075d20 0x7f00c819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 -- 172.21.15.42:0/3743768153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c81a1e40 con 0x7f00c8110b90 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00cde00700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8075d20 0x7f00c819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.338+0000 7f00ce800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 0x7f00c81a17a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f00c40094f0 tx=0x7f00c4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.339+0000 7f00c2a00700 1 -- 172.21.15.42:0/3743768153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4004db0 con 0x7f00c8110b90 2024-05-27T10:19:04.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.339+0000 7f00c2a00700 1 -- 172.21.15.42:0/3743768153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c4005020 con 0x7f00c8110b90 2024-05-27T10:19:04.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.339+0000 7f00c2a00700 1 -- 172.21.15.42:0/3743768153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4015660 con 0x7f00c8110b90 2024-05-27T10:19:04.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.339+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c81a2130 con 0x7f00c8110b90 2024-05-27T10:19:04.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.339+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c81a9a50 con 0x7f00c8110b90 2024-05-27T10:19:04.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.343+0000 7f00c2a00700 1 -- 172.21.15.42:0/3743768153 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f00c40051d0 con 0x7f00c8110b90 2024-05-27T10:19:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.344+0000 7f00df2e3b80 1 Processor -- start 2024-05-27T10:19:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.344+0000 7f00df2e3b80 1 -- start start 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdc8a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdc8aef0 0x5589fdc97190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5589fdbb8ec0 con 0x5589fdb0bec0 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589fdba8a60 con 0x5589fdbf7a70 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589fdbacaa0 con 0x5589fdc8aef0 2024-05-27T10:19:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00cde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdc8a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00cde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdc8a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42836/0 (socket says 172.21.15.42:42836) 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00cde00700 1 -- 172.21.15.42:0/2455417501 learned_addr learned my addr 172.21.15.42:0/2455417501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00cd400700 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00cd400700 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdc8aef0 0x5589fdc97190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.345+0000 7f00c2a00700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f00b00b2b10 0x7f00b00b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00cde00700 1 -- 172.21.15.42:0/2455417501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdc8aef0 msgr2=0x5589fdc97190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00cde00700 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdc8aef0 0x5589fdc97190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00cde00700 1 -- 172.21.15.42:0/2455417501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 msgr2=0x5589fdc8a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00cde00700 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00cde00700 1 -- 172.21.15.42:0/2455417501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fdb714a0 con 0x5589fdb0bec0 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00c2a00700 1 -- 172.21.15.42:0/3743768153 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f00c40e23a0 con 0x7f00c8110b90 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00ce800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f00b00b2b10 0x7f00b00b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.346+0000 7f00cde00700 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdc8a8f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00b5c0 tx=0x7f00bc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8684808b30169eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.347+0000 7f00b7400700 1 -- 172.21.15.42:0/2455417501 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc026070 con 0x5589fdb0bec0 2024-05-27T10:19:04.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.347+0000 7f00b7400700 1 -- 172.21.15.42:0/2455417501 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00bc003c50 con 0x5589fdb0bec0 2024-05-27T10:19:04.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.347+0000 7f00b7400700 1 -- 172.21.15.42:0/2455417501 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00b7d0 con 0x5589fdb0bec0 2024-05-27T10:19:04.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 -- 172.21.15.42:0/2455417501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 msgr2=0x5589fdc8a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00ce800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f00b00b2b10 0x7f00b00b5020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00c40df9c0 tx=0x7f00c400b040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdc8a8f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00b5c0 tx=0x7f00bc0096c0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 -- 172.21.15.42:0/2455417501 shutdown_connections 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdc8aef0 0x5589fdc97190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/2455417501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdc8a8f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 -- 172.21.15.42:0/2455417501 >> 172.21.15.42:0/2455417501 conn(0x5589fdba5410 msgr2=0x5589fdc86540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 -- 172.21.15.42:0/2455417501 shutdown_connections 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.348+0000 7f00df2e3b80 1 -- 172.21.15.42:0/2455417501 wait complete. 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.349+0000 7f00df2e3b80 1 -- start start 2024-05-27T10:19:04.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.349+0000 7f00df2e3b80 1 Processor -- start 2024-05-27T10:19:04.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc94750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 0x5589fdc94c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdc951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5589fdaaf020 con 0x5589fdbf7a70 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589fdb497e0 con 0x5589fdc8aef0 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589fdc97ba0 con 0x5589fdb0bec0 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00df2e3b80 10 client.0 _ll_register_callbacks cb 0x5589fdb52cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00cde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 0x5589fdc94c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00cde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 0x5589fdc94c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42838/0 (socket says 172.21.15.42:42838) 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00cde00700 1 -- 172.21.15.42:0/3505441688 learned_addr learned my addr 172.21.15.42:0/3505441688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00cd400700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc94750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.350+0000 7f00cd400700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdc951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cde00700 1 -- 172.21.15.42:0/3505441688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 msgr2=0x5589fdc94750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cde00700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc94750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cde00700 1 -- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 msgr2=0x5589fdc951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cde00700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdc951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cde00700 1 -- 172.21.15.42:0/3505441688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589fdaf8950 con 0x5589fdbf7a70 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cd400700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdc951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00cde00700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 0x5589fdc94c90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00bc024040 tx=0x7f00bc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c8f8942f4c3ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00abe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:04.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00abe00700 1 -- 172.21.15.42:0/3505441688 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc026070 con 0x5589fdbf7a70 2024-05-27T10:19:04.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00abe00700 1 -- 172.21.15.42:0/3505441688 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f00bc00b9e0 con 0x5589fdbf7a70 2024-05-27T10:19:04.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.351+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00bc003c50 con 0x5589fdbf7a70 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00abe00700 1 -- 172.21.15.42:0/3505441688 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00bc003ab0 con 0x5589fdbf7a70 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00df2e3b80 10 client.15315 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589fdb790e0 con 0x5589fdbf7a70 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00abe00700 1 -- 172.21.15.42:0/3505441688 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00bc003c50 con 0x5589fdbf7a70 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00abe00700 1 -- 172.21.15.42:0/3505441688 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f00bc035c80 con 0x5589fdbf7a70 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00df2e3b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00df2e3b80 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00df2e3b80 20 client.15315 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.352+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdca4a00 0x5589fdca6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.353+0000 7f00df2e3b80 4 client.15315 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:04.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.353+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589fdbbae10 con 0x5589fdca4a00 2024-05-27T10:19:04.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.353+0000 7f00cd400700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdca4a00 0x5589fdca6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.354+0000 7f00cd400700 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdca4a00 0x5589fdca6e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5048feea0f8cc697 server_cookie=d3b678eb92a29ea in_seq=0 out_seq=0 2024-05-27T10:19:04.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.354+0000 7f00abe00700 10 client.15315 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:04.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00abe00700 1 -- 172.21.15.42:0/3505441688 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5589fdbbae10 con 0x5589fdca4a00 2024-05-27T10:19:04.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00abe00700 10 client.15315 handle_command_reply: tid=0 2024-05-27T10:19:04.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 client.15315 shutdown 2024-05-27T10:19:04.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 msgr2=0x5589fdc94c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 0x5589fdc94c90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00bc024040 tx=0x7f00bc0057b0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 shutdown_connections 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdca4a00 0x5589fdca6e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdc951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc94750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.356+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3505441688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdbf7a70 0x5589fdc94c90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.357+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 >> 172.21.15.42:0/3505441688 conn(0x5589fdba5410 msgr2=0x5589fdc8a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.357+0000 7f00abe00700 0 client.15315 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.357+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 shutdown_connections 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.357+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3505441688 wait complete. 2024-05-27T10:19:04.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.357+0000 7f00df2e3b80 20 client.15315 trim_cache size 0 max 16384 2024-05-27T10:19:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 Processor -- start 2024-05-27T10:19:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 -- start start 2024-05-27T10:19:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc8ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 0x5589fdc8e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81b3740 con 0x5589fdc8aef0 2024-05-27T10:19:04.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81b0fd0 con 0x5589fdbf7a70 2024-05-27T10:19:04.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.380+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81aee20 con 0x5589fdb0bec0 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.381+0000 7f00cd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 0x5589fdc8e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.381+0000 7f00cd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 0x5589fdc8e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42842/0 (socket says 172.21.15.42:42842) 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.381+0000 7f00cd400700 1 -- 172.21.15.42:0/3064264850 learned_addr learned my addr 172.21.15.42:0/3064264850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.381+0000 7f00cde00700 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.381+0000 7f00cd400700 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc8ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.382+0000 7f00cd400700 1 -- 172.21.15.42:0/3064264850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 msgr2=0x5589fdc8ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.382+0000 7f00cd400700 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc8ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.382+0000 7f00cd400700 1 -- 172.21.15.42:0/3064264850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 msgr2=0x5589fdc8c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.382+0000 7f00cd400700 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.382+0000 7f00cd400700 1 -- 172.21.15.42:0/3064264850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fdc8a800 con 0x5589fdc8aef0 2024-05-27T10:19:04.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.382+0000 7f00cd400700 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 0x5589fdc8e0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f00b80046c0 tx=0x7f00b800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22759f42a67fb75e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.383+0000 7f00b6a00700 1 -- 172.21.15.42:0/3064264850 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8005020 con 0x5589fdc8aef0 2024-05-27T10:19:04.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.383+0000 7f00b6a00700 1 -- 172.21.15.42:0/3064264850 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00b8005650 con 0x5589fdc8aef0 2024-05-27T10:19:04.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.383+0000 7f00b6a00700 1 -- 172.21.15.42:0/3064264850 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8020390 con 0x5589fdc8aef0 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3064264850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 msgr2=0x5589fdc8e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 0x5589fdc8e0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f00b80046c0 tx=0x7f00b800c450 comp rx=0 tx=0).stop 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3064264850 shutdown_connections 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdc8aef0 0x5589fdc8e0b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdb0bec0 0x5589fdc8ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/3064264850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdbf7a70 0x5589fdc8c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3064264850 >> 172.21.15.42:0/3064264850 conn(0x5589fdba5410 msgr2=0x5589fdc8bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3064264850 shutdown_connections 2024-05-27T10:19:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.384+0000 7f00df2e3b80 1 -- 172.21.15.42:0/3064264850 wait complete. 2024-05-27T10:19:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.385+0000 7f00df2e3b80 1 -- start start 2024-05-27T10:19:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.385+0000 7f00df2e3b80 1 Processor -- start 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdca5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdbf7a70 0x5589fdca64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdd0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81af490 con 0x5589fdb0bec0 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81b0900 con 0x5589fdc8aef0 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f00c81afb60 con 0x5589fdbf7a70 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00df2e3b80 10 client.0 _ll_register_callbacks cb 0x5589fdb52cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00cd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdca5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00cd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdca5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42856/0 (socket says 172.21.15.42:42856) 2024-05-27T10:19:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.386+0000 7f00cd400700 1 -- 172.21.15.42:0/224077987 learned_addr learned my addr 172.21.15.42:0/224077987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cde00700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdbf7a70 0x5589fdca64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdd0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 -- 172.21.15.42:0/224077987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdbf7a70 msgr2=0x5589fdca64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdbf7a70 0x5589fdca64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 -- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 msgr2=0x5589fdd0e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdd0e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 -- 172.21.15.42:0/224077987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589fdd0e7d0 con 0x5589fdb0bec0 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00cd400700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdca5fb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c660 tx=0x7f00b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13949960b3d1fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.387+0000 7f00abe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.388+0000 7f00abe00700 1 -- 172.21.15.42:0/224077987 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8004c00 con 0x5589fdb0bec0 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.388+0000 7f00abe00700 1 -- 172.21.15.42:0/224077987 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f00b8004da0 con 0x5589fdb0bec0 2024-05-27T10:19:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.388+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589fdbb45f0 con 0x5589fdb0bec0 2024-05-27T10:19:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.388+0000 7f00abe00700 1 -- 172.21.15.42:0/224077987 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00b8005c20 con 0x5589fdb0bec0 2024-05-27T10:19:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.388+0000 7f00df2e3b80 10 client.15321 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.388+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589fdd0fa70 con 0x5589fdb0bec0 2024-05-27T10:19:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00abe00700 1 -- 172.21.15.42:0/224077987 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5589fdbb45f0 con 0x5589fdb0bec0 2024-05-27T10:19:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00abe00700 1 -- 172.21.15.42:0/224077987 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f00b8020870 con 0x5589fdb0bec0 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00df2e3b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00df2e3b80 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00df2e3b80 20 client.15321 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdd100c0 0x5589fdd124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00df2e3b80 4 client.15321 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.389+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5589fda2b2d0 con 0x5589fdd100c0 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.390+0000 7f00cde00700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdd100c0 0x5589fdd124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.391+0000 7f00cde00700 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdd100c0 0x5589fdd124e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4004f75c0c04dcdc server_cookie=d22766a7e8ef0c9e in_seq=0 out_seq=0 2024-05-27T10:19:04.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.391+0000 7f00abe00700 10 client.15321 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:04.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00abe00700 1 -- 172.21.15.42:0/224077987 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5589fda2b2d0 con 0x5589fdd100c0 2024-05-27T10:19:04.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00abe00700 10 client.15321 handle_command_reply: tid=0 2024-05-27T10:19:04.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 client.15321 shutdown 2024-05-27T10:19:04.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 msgr2=0x5589fdca5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdca5fb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c660 tx=0x7f00b8004270 comp rx=0 tx=0).stop 2024-05-27T10:19:04.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 shutdown_connections 2024-05-27T10:19:04.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5589fdd100c0 0x5589fdd124e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589fdc8aef0 0x5589fdd0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5589fdb0bec0 0x5589fdca5fb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 --2- 172.21.15.42:0/224077987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589fdbf7a70 0x5589fdca64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 >> 172.21.15.42:0/224077987 conn(0x5589fdba5410 msgr2=0x5589fdc94530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00abe00700 0 client.15321 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 shutdown_connections 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.394+0000 7f00df2e3b80 1 -- 172.21.15.42:0/224077987 wait complete. 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.395+0000 7f00df2e3b80 20 client.15321 trim_cache size 0 max 16384 2024-05-27T10:19:04.396 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:19:04.397 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-05-27T10:19:04.397 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "cc420705-6a6a-4d62-96fa-afbcd6b325d7", 2024-05-27T10:19:04.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-05-27T10:19:04.397 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:19:04.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.395+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f00b00b2b10 msgr2=0x7f00b00b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.395+0000 7f00d0800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f00b00b2b10 0x7f00b00b5020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00c40df9c0 tx=0x7f00c400b040 comp rx=0 tx=0).stop 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 msgr2=0x7f00c81a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 0x7f00c81a17a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f00c40094f0 tx=0x7f00c4007f90 comp rx=0 tx=0).stop 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 shutdown_connections 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f00b00b2b10 0x7f00b00b5020 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c8110b90 0x7f00c81a17a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c8075d20 0x7f00c819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 --2- 172.21.15.42:0/3743768153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c8076670 0x7f00c819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 >> 172.21.15.42:0/3743768153 conn(0x7f00c8100030 msgr2=0x7f00c81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 shutdown_connections 2024-05-27T10:19:04.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.396+0000 7f00d0800700 1 -- 172.21.15.42:0/3743768153 wait complete. 2024-05-27T10:19:04.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 -- 172.21.15.42:0/1594127752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32101041f0 msgr2=0x7f32101105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 --2- 172.21.15.42:0/1594127752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32101041f0 0x7f32101105c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3208009ca0 tx=0x7f320802a8c0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 -- 172.21.15.42:0/1594127752 shutdown_connections 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 --2- 172.21.15.42:0/1594127752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32101041f0 0x7f32101105c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 --2- 172.21.15.42:0/1594127752 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3210109f40 0x7f3210103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 --2- 172.21.15.42:0/1594127752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3210109aa0 0x7f3210103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.559+0000 7f320fe00640 1 -- 172.21.15.42:0/1594127752 >> 172.21.15.42:0/1594127752 conn(0x7f32100791a0 msgr2=0x7f32100795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.560+0000 7f320fe00640 1 -- 172.21.15.42:0/1594127752 shutdown_connections 2024-05-27T10:19:04.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.560+0000 7f320fe00640 1 -- 172.21.15.42:0/1594127752 wait complete. 2024-05-27T10:19:04.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.561+0000 7f320fe00640 1 Processor -- start 2024-05-27T10:19:04.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.561+0000 7f320fe00640 1 -- start start 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32101041f0 0x7f32101a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3210109aa0 0x7f32101a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 0x7f321019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3210113bb0 con 0x7f3210109f40 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3210113a30 con 0x7f3210109aa0 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3210113d30 con 0x7f32101041f0 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 0x7f321019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 0x7f321019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42866/0 (socket says 172.21.15.42:42866) 2024-05-27T10:19:04.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.562+0000 7f320f400640 1 -- 172.21.15.42:0/121054297 learned_addr learned my addr 172.21.15.42:0/121054297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:04.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320e000640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3210109aa0 0x7f32101a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320ea00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32101041f0 0x7f32101a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320f400640 1 -- 172.21.15.42:0/121054297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32101041f0 msgr2=0x7f32101a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320f400640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32101041f0 0x7f32101a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320f400640 1 -- 172.21.15.42:0/121054297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3210109aa0 msgr2=0x7f32101a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320f400640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3210109aa0 0x7f32101a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320f400640 1 -- 172.21.15.42:0/121054297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321019cd80 con 0x7f3210109f40 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320f400640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 0x7f321019c650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3208005e30 tx=0x7f32080093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208032d10 con 0x7f3210109f40 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321019cfb0 con 0x7f3210109f40 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3208032eb0 con 0x7f3210109f40 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208030b20 con 0x7f3210109f40 2024-05-27T10:19:04.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.563+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101a9270 con 0x7f3210109f40 2024-05-27T10:19:04.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.565+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3210104d80 con 0x7f3210109f40 2024-05-27T10:19:04.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.567+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f3208030cc0 con 0x7f3210109f40 2024-05-27T10:19:04.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.571+0000 7f3214a00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f31f40b2a70 0x7f31f40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:04.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.571+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3208103e10 con 0x7f3210109f40 2024-05-27T10:19:04.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.571+0000 7f320ea00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f31f40b2a70 0x7f31f40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:04.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.571+0000 7f320ea00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f31f40b2a70 0x7f31f40b4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3210104780 tx=0x7f3200005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:04.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.573+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32080cb6a0 con 0x7f3210109f40 2024-05-27T10:19:04.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.804+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3210101a50 con 0x7f31f40b2a70 2024-05-27T10:19:04.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.821+0000 7f3214a00640 1 -- 172.21.15.42:0/121054297 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3210101a50 con 0x7f31f40b2a70 2024-05-27T10:19:04.821 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T10:19:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-27T10:19:04.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f31f40b2a70 msgr2=0x7f31f40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f31f40b2a70 0x7f31f40b4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3210104780 tx=0x7f3200005f90 comp rx=0 tx=0).stop 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 msgr2=0x7f321019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 0x7f321019c650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3208005e30 tx=0x7f32080093d0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 shutdown_connections 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f31f40b2a70 0x7f31f40b4f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3210109f40 0x7f321019c650 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3210109aa0 0x7f32101a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 --2- 172.21.15.42:0/121054297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32101041f0 0x7f32101a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.823+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 >> 172.21.15.42:0/121054297 conn(0x7f32100791a0 msgr2=0x7f32100779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:04.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.824+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 shutdown_connections 2024-05-27T10:19:04.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:04.824+0000 7f320fe00640 1 -- 172.21.15.42:0/121054297 wait complete. 2024-05-27T10:19:04.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T10:19:05.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:04 smithi143 ceph-mon[35545]: pgmap v13: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:19:05.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:04 smithi143 ceph-mon[35545]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:05.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:04 smithi042 ceph-mon[31661]: pgmap v13: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:19:05.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:04 smithi042 ceph-mon[31661]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:05.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 -- 172.21.15.42:0/3910392506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c078720 msgr2=0x7f0e4c078b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 --2- 172.21.15.42:0/3910392506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c078720 0x7f0e4c078b30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009900 tx=0x7f0e3c02a3f0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 -- 172.21.15.42:0/3910392506 shutdown_connections 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 --2- 172.21.15.42:0/3910392506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c114af0 0x7f0e4c116f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 --2- 172.21.15.42:0/3910392506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4c112190 0x7f0e4c1145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 --2- 172.21.15.42:0/3910392506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c078720 0x7f0e4c078b30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 -- 172.21.15.42:0/3910392506 >> 172.21.15.42:0/3910392506 conn(0x7f0e4c0771f0 msgr2=0x7f0e4c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.169+0000 7f0e51000640 1 -- 172.21.15.42:0/3910392506 shutdown_connections 2024-05-27T10:19:05.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.170+0000 7f0e51000640 1 -- 172.21.15.42:0/3910392506 wait complete. 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 Processor -- start 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 -- start start 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c078720 0x7f0e4c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 0x7f0e4c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4c114af0 0x7f0e4c1a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c11a3e0 con 0x7f0e4c112190 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c11a260 con 0x7f0e4c078720 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.171+0000 7f0e51000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c11a560 con 0x7f0e4c114af0 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 0x7f0e4c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 0x7f0e4c1a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42876/0 (socket says 172.21.15.42:42876) 2024-05-27T10:19:05.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 -- 172.21.15.42:0/297120340 learned_addr learned my addr 172.21.15.42:0/297120340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49e00640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c078720 0x7f0e4c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e4a800640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4c114af0 0x7f0e4c1a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 -- 172.21.15.42:0/297120340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4c114af0 msgr2=0x7f0e4c1a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4c114af0 0x7f0e4c1a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 -- 172.21.15.42:0/297120340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c078720 msgr2=0x7f0e4c1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c078720 0x7f0e4c1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49400640 1 -- 172.21.15.42:0/297120340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c1a5c90 con 0x7f0e4c112190 2024-05-27T10:19:05.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.172+0000 7f0e49e00640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c078720 0x7f0e4c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:05.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.173+0000 7f0e49400640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 0x7f0e4c1a1460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800b480 tx=0x7f0e3800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:05.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.173+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38014070 con 0x7f0e4c112190 2024-05-27T10:19:05.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.173+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e38010d00 con 0x7f0e4c112190 2024-05-27T10:19:05.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.173+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800fd80 con 0x7f0e4c112190 2024-05-27T10:19:05.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.173+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c1a5f80 con 0x7f0e4c112190 2024-05-27T10:19:05.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.173+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c1a6480 con 0x7f0e4c112190 2024-05-27T10:19:05.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.174+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4c050860 con 0x7f0e4c112190 2024-05-27T10:19:05.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.177+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0e38017070 con 0x7f0e4c112190 2024-05-27T10:19:05.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.181+0000 7f0e42a00640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0e300b2a70 0x7f0e300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:05.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.181+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0e380e24a0 con 0x7f0e4c112190 2024-05-27T10:19:05.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.181+0000 7f0e49e00640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0e300b2a70 0x7f0e300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:05.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.181+0000 7f0e49e00640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0e300b2a70 0x7f0e300b4f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c006010 tx=0x7f0e3c005a20 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:05.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.184+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e380a9e00 con 0x7f0e4c112190 2024-05-27T10:19:05.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:04 smithi104 ceph-mon[34812]: pgmap v13: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:19:05.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:04 smithi104 ceph-mon[34812]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.421+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0e4c112cb0 con 0x7f0e300b2a70 2024-05-27T10:19:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.441+0000 7f0e42a00640 1 -- 172.21.15.42:0/297120340 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0e4c112cb0 con 0x7f0e300b2a70 2024-05-27T10:19:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2024-05-27T10:19:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0e300b2a70 msgr2=0x7f0e300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0e300b2a70 0x7f0e300b4f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c006010 tx=0x7f0e3c005a20 comp rx=0 tx=0).stop 2024-05-27T10:19:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 msgr2=0x7f0e4c1a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 0x7f0e4c1a1460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800b480 tx=0x7f0e3800b940 comp rx=0 tx=0).stop 2024-05-27T10:19:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 shutdown_connections 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0e300b2a70 0x7f0e300b4f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4c114af0 0x7f0e4c1a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e4c112190 0x7f0e4c1a1460 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 --2- 172.21.15.42:0/297120340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e4c078720 0x7f0e4c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.444+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 >> 172.21.15.42:0/297120340 conn(0x7f0e4c0771f0 msgr2=0x7f0e4c1125a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.445+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 shutdown_connections 2024-05-27T10:19:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:05.445+0000 7f0e51000640 1 -- 172.21.15.42:0/297120340 wait complete. 2024-05-27T10:19:05.471 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-27T10:19:05.543 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:19:05.554 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-27T10:19:05.566 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-27T10:19:05.593 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:19:05.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-27T10:19:05.599 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-27T10:19:05.628 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T10:19:05.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:19:05.981 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:05 smithi143 ceph-mon[35545]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:05 smithi143 ceph-mon[35545]: scrub queued for path: / 2024-05-27T10:19:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:05 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:19:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:05 smithi143 ceph-mon[35545]: scrub summary: active paths [/] 2024-05-27T10:19:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:05 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:19:06.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:05 smithi143 ceph-mon[35545]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:06.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:05 smithi042 ceph-mon[31661]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:06.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:05 smithi042 ceph-mon[31661]: scrub queued for path: / 2024-05-27T10:19:06.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:05 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:19:06.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:05 smithi042 ceph-mon[31661]: scrub summary: active paths [/] 2024-05-27T10:19:06.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:05 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:19:06.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:05 smithi042 ceph-mon[31661]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:06.100 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:05 smithi104 ceph-mon[34812]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:06.100 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:05 smithi104 ceph-mon[34812]: scrub queued for path: / 2024-05-27T10:19:06.100 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:05 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:19:06.100 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:05 smithi104 ceph-mon[34812]: scrub summary: active paths [/] 2024-05-27T10:19:06.100 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:05 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:19:06.101 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:05 smithi104 ceph-mon[34812]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:06.319 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:07.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:06 smithi042 ceph-mon[31661]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-05-27T10:19:07.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:06 smithi042 ceph-mon[31661]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:07.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:06 smithi104 ceph-mon[34812]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-05-27T10:19:07.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:06 smithi104 ceph-mon[34812]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:07.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:06 smithi143 ceph-mon[35545]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-05-27T10:19:07.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:06 smithi143 ceph-mon[35545]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:19:09.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:08 smithi042 ceph-mon[31661]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T10:19:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:08 smithi104 ceph-mon[34812]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T10:19:09.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:08 smithi143 ceph-mon[35545]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 -- 172.21.15.42:0/1316199416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8108750 msgr2=0x7ff2b81100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1316199416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8108750 0x7ff2b81100b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b0d0 tx=0x7ff2b000b3e0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 -- 172.21.15.42:0/1316199416 shutdown_connections 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1316199416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8108750 0x7ff2b81100b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1316199416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2b81034d0 0x7ff2b81038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1316199416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b8107e00 0x7ff2b8108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 -- 172.21.15.42:0/1316199416 >> 172.21.15.42:0/1316199416 conn(0x7ff2b80fef20 msgr2=0x7ff2b8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.055+0000 7ff2bf600700 1 -- 172.21.15.42:0/1316199416 shutdown_connections 2024-05-27T10:19:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.056+0000 7ff2bf600700 1 -- 172.21.15.42:0/1316199416 wait complete. 2024-05-27T10:19:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.057+0000 7ff2bf600700 1 Processor -- start 2024-05-27T10:19:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.057+0000 7ff2bf600700 1 -- start start 2024-05-27T10:19:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bf600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2b81034d0 0x7ff2b819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bf600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 0x7ff2b819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bf600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b8108750 0x7ff2b81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bf600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81143e0 con 0x7ff2b8107e00 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bf600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8114260 con 0x7ff2b81034d0 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bf600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8114560 con 0x7ff2b8108750 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 0x7ff2b819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 0x7ff2b819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42900/0 (socket says 172.21.15.42:42900) 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1144133344 learned_addr learned my addr 172.21.15.42:0/1144133344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.058+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2b81034d0 0x7ff2b819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1144133344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b8108750 msgr2=0x7ff2b81a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b8108750 0x7ff2b81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1144133344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2b81034d0 msgr2=0x7ff2b819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2b81034d0 0x7ff2b819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1144133344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b81a1e40 con 0x7ff2b8107e00 2024-05-27T10:19:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 0x7ff2b819d510 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d980 tx=0x7ff2a400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2b5600700 1 -- 172.21.15.42:0/1144133344 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4017070 con 0x7ff2b8107e00 2024-05-27T10:19:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.059+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b81a2130 con 0x7ff2b8107e00 2024-05-27T10:19:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.060+0000 7ff2b5600700 1 -- 172.21.15.42:0/1144133344 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2a4010460 con 0x7ff2b8107e00 2024-05-27T10:19:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.060+0000 7ff2b5600700 1 -- 172.21.15.42:0/1144133344 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400de60 con 0x7ff2b8107e00 2024-05-27T10:19:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.060+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81a9a50 con 0x7ff2b8107e00 2024-05-27T10:19:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.064+0000 7ff2b5600700 1 -- 172.21.15.42:0/1144133344 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7ff2a4010610 con 0x7ff2b8107e00 2024-05-27T10:19:10.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.065+0000 7ff2b5600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff2a00b2b90 0x7ff2a00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.065+0000 7ff2b5600700 1 -- 172.21.15.42:0/1144133344 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff2a40e2340 con 0x7ff2b8107e00 2024-05-27T10:19:10.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.066+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff2a00b2b90 0x7ff2a00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.066+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff2a00b2b90 0x7ff2a00b50a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005950 tx=0x7ff2ac0058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.067+0000 7ff2ce080b80 1 Processor -- start 2024-05-27T10:19:10.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.067+0000 7ff2ce080b80 1 -- start start 2024-05-27T10:19:10.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.068+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f5374330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.068+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 0x5630f5374870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.068+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5381110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.068+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5630f52a2e40 con 0x5630f51f5e40 2024-05-27T10:19:10.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.068+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630f52929e0 con 0x5630f5374e70 2024-05-27T10:19:10.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.068+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5630f5296a20 con 0x5630f52e19f0 2024-05-27T10:19:10.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 0x5630f5374870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 0x5630f5374870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42910/0 (socket says 172.21.15.42:42910) 2024-05-27T10:19:10.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 -- 172.21.15.42:0/428027868 learned_addr learned my addr 172.21.15.42:0/428027868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:10.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f5374330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2bd600700 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5381110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 -- 172.21.15.42:0/428027868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 msgr2=0x5630f5374330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f5374330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 -- 172.21.15.42:0/428027868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 msgr2=0x5630f5381110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5381110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.069+0000 7ff2b7e00700 1 -- 172.21.15.42:0/428027868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f525b420 con 0x5630f51f5e40 2024-05-27T10:19:10.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.070+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 0x5630f5374870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4010fd0 tx=0x7ff2a40a9b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac30463579e469c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.070+0000 7ff2aa000700 1 -- 172.21.15.42:0/428027868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4020070 con 0x5630f51f5e40 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.070+0000 7ff2aa000700 1 -- 172.21.15.42:0/428027868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2a40ad770 con 0x5630f51f5e40 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.070+0000 7ff2aa000700 1 -- 172.21.15.42:0/428027868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4009b40 con 0x5630f51f5e40 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.071+0000 7ff2ce080b80 1 -- 172.21.15.42:0/428027868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 msgr2=0x5630f5374870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.071+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 0x5630f5374870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4010fd0 tx=0x7ff2a40a9b90 comp rx=0 tx=0).stop 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.071+0000 7ff2ce080b80 1 -- 172.21.15.42:0/428027868 shutdown_connections 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.071+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5381110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.072+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f5374330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.072+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/428027868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f51f5e40 0x5630f5374870 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.072+0000 7ff2ce080b80 1 -- 172.21.15.42:0/428027868 >> 172.21.15.42:0/428027868 conn(0x5630f528f390 msgr2=0x5630f53704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.072+0000 7ff2ce080b80 1 -- 172.21.15.42:0/428027868 shutdown_connections 2024-05-27T10:19:10.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.072+0000 7ff2ce080b80 1 -- 172.21.15.42:0/428027868 wait complete. 2024-05-27T10:19:10.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.072+0000 7ff2ce080b80 1 -- start start 2024-05-27T10:19:10.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.073+0000 7ff2ce080b80 1 Processor -- start 2024-05-27T10:19:10.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f51f5e40 0x5630f537e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f537ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f537f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5630f5198fa0 con 0x5630f5374e70 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630f5233760 con 0x5630f51f5e40 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5630f5381ac0 con 0x5630f52e19f0 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2ce080b80 10 client.0 _ll_register_callbacks cb 0x5630f523cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2bd600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f537f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2bd600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f537f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42912/0 (socket says 172.21.15.42:42912) 2024-05-27T10:19:10.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2bd600700 1 -- 172.21.15.42:0/3334054128 learned_addr learned my addr 172.21.15.42:0/3334054128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:10.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.074+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f51f5e40 0x5630f537e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f537ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bd600700 1 -- 172.21.15.42:0/3334054128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 msgr2=0x5630f537ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bd600700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f537ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bd600700 1 -- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f51f5e40 msgr2=0x5630f537e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bd600700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f51f5e40 0x5630f537e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bd600700 1 -- 172.21.15.42:0/3334054128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f51e28d0 con 0x5630f5374e70 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f51f5e40 0x5630f537e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff2bd600700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f537f1f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009580 tx=0x7ff2b00095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba38094e8113123 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.075+0000 7ff29ea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.076+0000 7ff29ea00700 1 -- 172.21.15.42:0/3334054128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004e30 con 0x5630f5374e70 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.076+0000 7ff29ea00700 1 -- 172.21.15.42:0/3334054128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff2b0003a00 con 0x5630f5374e70 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.076+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a40ad770 con 0x5630f5374e70 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.076+0000 7ff29ea00700 1 -- 172.21.15.42:0/3334054128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2b0009780 con 0x5630f5374e70 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.076+0000 7ff2ce080b80 10 client.15345 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:10.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.076+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630f5263060 con 0x5630f5374e70 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff29ea00700 1 -- 172.21.15.42:0/3334054128 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2a40ad770 con 0x5630f5374e70 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff29ea00700 1 -- 172.21.15.42:0/3334054128 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff2b0011070 con 0x5630f5374e70 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff2ce080b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff2ce080b80 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff2ce080b80 20 client.15345 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f538eab0 0x5630f5390ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff2ce080b80 4 client.15345 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:10.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.077+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630f52a4d90 con 0x5630f538eab0 2024-05-27T10:19:10.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.078+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f538eab0 0x5630f5390ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.079+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f538eab0 0x5630f5390ed0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fb372b7bdca31b server_cookie=d861c5f894961b in_seq=0 out_seq=0 2024-05-27T10:19:10.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.079+0000 7ff29ea00700 10 client.15345 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:10.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff29ea00700 1 -- 172.21.15.42:0/3334054128 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5630f52a4d90 con 0x5630f538eab0 2024-05-27T10:19:10.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff29ea00700 10 client.15345 handle_command_reply: tid=0 2024-05-27T10:19:10.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 client.15345 shutdown 2024-05-27T10:19:10.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 msgr2=0x5630f537f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f537f1f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009580 tx=0x7ff2b00095b0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 shutdown_connections 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f538eab0 0x5630f5390ed0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f537f1f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f51f5e40 0x5630f537e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3334054128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f52e19f0 0x5630f537ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 >> 172.21.15.42:0/3334054128 conn(0x5630f528f390 msgr2=0x5630f53743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff29ea00700 0 client.15345 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 shutdown_connections 2024-05-27T10:19:10.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.081+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3334054128 wait complete. 2024-05-27T10:19:10.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.082+0000 7ff2ce080b80 20 client.15345 trim_cache size 0 max 16384 2024-05-27T10:19:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.105+0000 7ff2ce080b80 1 Processor -- start 2024-05-27T10:19:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.105+0000 7ff2ce080b80 1 -- start start 2024-05-27T10:19:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f537d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f52e19f0 0x5630f538ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f5380530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b3800 con 0x5630f5374e70 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b1090 con 0x5630f52e19f0 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81aeee0 con 0x5630f51f5e40 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2bd600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f5380530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2bd600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f5380530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42914/0 (socket says 172.21.15.42:42914) 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2bd600700 1 -- 172.21.15.42:0/3250077772 learned_addr learned my addr 172.21.15.42:0/3250077772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.106+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f52e19f0 0x5630f538ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f537d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bd600700 1 -- 172.21.15.42:0/3250077772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 msgr2=0x5630f537d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bd600700 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f537d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bd600700 1 -- 172.21.15.42:0/3250077772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f52e19f0 msgr2=0x5630f538ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bd600700 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f52e19f0 0x5630f538ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bd600700 1 -- 172.21.15.42:0/3250077772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f536c240 con 0x5630f5374e70 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f52e19f0 0x5630f538ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.107+0000 7ff2bd600700 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f5380530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009fd0 tx=0x7ff2b0004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c33805f21956fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.108+0000 7ff2a9600700 1 -- 172.21.15.42:0/3250077772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0025c40 con 0x5630f5374e70 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.108+0000 7ff2a9600700 1 -- 172.21.15.42:0/3250077772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2b0005c60 con 0x5630f5374e70 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.108+0000 7ff2a9600700 1 -- 172.21.15.42:0/3250077772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0007a80 con 0x5630f5374e70 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3250077772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 msgr2=0x5630f5380530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f5380530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009fd0 tx=0x7ff2b0004c00 comp rx=0 tx=0).stop 2024-05-27T10:19:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3250077772 shutdown_connections 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f5374e70 0x5630f5380530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f537d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/3250077772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f52e19f0 0x5630f538ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3250077772 >> 172.21.15.42:0/3250077772 conn(0x5630f528f390 msgr2=0x5630f537fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3250077772 shutdown_connections 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 -- 172.21.15.42:0/3250077772 wait complete. 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.109+0000 7ff2ce080b80 1 -- start start 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 Processor -- start 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f5378ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 0x5630f5379430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5379970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81af550 con 0x5630f52e19f0 2024-05-27T10:19:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b09c0 con 0x5630f5374e70 2024-05-27T10:19:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81afc20 con 0x5630f51f5e40 2024-05-27T10:19:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.110+0000 7ff2ce080b80 10 client.0 _ll_register_callbacks cb 0x5630f523cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 0x5630f5379430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 0x5630f5379430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42926/0 (socket says 172.21.15.42:42926) 2024-05-27T10:19:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1153937050 learned_addr learned my addr 172.21.15.42:0/1153937050 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2bd600700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5379970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f5378ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1153937050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 msgr2=0x5630f5378ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f5378ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 msgr2=0x5630f5379970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5379970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.111+0000 7ff2b7e00700 1 -- 172.21.15.42:0/1153937050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f5385250 con 0x5630f52e19f0 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff2b7e00700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 0x5630f5379430 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40a5440 tx=0x7ff2a4091050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1301f0577f0ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff29ea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff29ea00700 1 -- 172.21.15.42:0/1153937050 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4020070 con 0x5630f52e19f0 2024-05-27T10:19:10.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff29ea00700 1 -- 172.21.15.42:0/1153937050 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff2a40a59b0 con 0x5630f52e19f0 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b0005c60 con 0x5630f52e19f0 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff29ea00700 1 -- 172.21.15.42:0/1153937050 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2a40b1fa0 con 0x5630f52e19f0 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff2ce080b80 10 client.15351 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.112+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630f53f96c0 con 0x5630f52e19f0 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff29ea00700 1 -- 172.21.15.42:0/1153937050 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2b0005c60 con 0x5630f52e19f0 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff29ea00700 1 -- 172.21.15.42:0/1153937050 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff2a40dfd70 con 0x5630f52e19f0 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff2ce080b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff2ce080b80 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff2ce080b80 20 client.15351 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:10.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f53f9a00 0x5630f53fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff2ce080b80 4 client.15351 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:19:10.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.113+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630f529e570 con 0x5630f53f9a00 2024-05-27T10:19:10.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.114+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f53f9a00 0x5630f53fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.115+0000 7ff2bcc00700 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f53f9a00 0x5630f53fbe20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e43451a6bd7c2c server_cookie=fe4968a291c77f17 in_seq=0 out_seq=0 2024-05-27T10:19:10.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.115+0000 7ff29ea00700 10 client.15351 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:10.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.115+0000 7ff29ea00700 1 -- 172.21.15.42:0/1153937050 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630f529e570 con 0x5630f53f9a00 2024-05-27T10:19:10.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.115+0000 7ff29ea00700 10 client.15351 handle_command_reply: tid=0 2024-05-27T10:19:10.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.115+0000 7ff2ce080b80 1 client.15351 shutdown 2024-05-27T10:19:10.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 msgr2=0x5630f5379430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 0x5630f5379430 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40a5440 tx=0x7ff2a4091050 comp rx=0 tx=0).stop 2024-05-27T10:19:10.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 shutdown_connections 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5630f53f9a00 0x5630f53fbe20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff29ea00700 0 client.15351 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630f5374e70 0x5630f5379970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630f51f5e40 0x5630f5378ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 --2- 172.21.15.42:0/1153937050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630f52e19f0 0x5630f5379430 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 >> 172.21.15.42:0/1153937050 conn(0x5630f528f390 msgr2=0x5630f537fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 shutdown_connections 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 1 -- 172.21.15.42:0/1153937050 wait complete. 2024-05-27T10:19:10.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.116+0000 7ff2ce080b80 20 client.15351 trim_cache size 0 max 16384 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "cc420705-6a6a-4d62-96fa-afbcd6b325d7": { 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "cc420705-6a6a-4d62-96fa-afbcd6b325d7", 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:19:10.120 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:19:10.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff2a00b2b90 msgr2=0x7ff2a00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff2a00b2b90 0x7ff2a00b50a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005950 tx=0x7ff2ac0058e0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 msgr2=0x7ff2b819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 0x7ff2b819d510 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d980 tx=0x7ff2a400dd40 comp rx=0 tx=0).stop 2024-05-27T10:19:10.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 shutdown_connections 2024-05-27T10:19:10.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff2a00b2b90 0x7ff2a00b50a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b8108750 0x7ff2b81a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2b81034d0 0x7ff2b819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 --2- 172.21.15.42:0/1144133344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b8107e00 0x7ff2b819d510 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 >> 172.21.15.42:0/1144133344 conn(0x7ff2b80fef20 msgr2=0x7ff2b810fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.117+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 shutdown_connections 2024-05-27T10:19:10.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.118+0000 7ff2bf600700 1 -- 172.21.15.42:0/1144133344 wait complete. 2024-05-27T10:19:10.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 -- 172.21.15.42:0/92547861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70108730 msgr2=0x7f7a70110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 --2- 172.21.15.42:0/92547861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70108730 0x7f7a70110090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800b0d0 tx=0x7f7a6800b3e0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 -- 172.21.15.42:0/92547861 shutdown_connections 2024-05-27T10:19:10.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 --2- 172.21.15.42:0/92547861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70108730 0x7f7a70110090 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 --2- 172.21.15.42:0/92547861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a701034b0 0x7f7a701038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 --2- 172.21.15.42:0/92547861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70107de0 0x7f7a701081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 -- 172.21.15.42:0/92547861 >> 172.21.15.42:0/92547861 conn(0x7f7a700fedb0 msgr2=0x7f7a701011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 -- 172.21.15.42:0/92547861 shutdown_connections 2024-05-27T10:19:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.163+0000 7f7a76400700 1 -- 172.21.15.42:0/92547861 wait complete. 2024-05-27T10:19:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.164+0000 7f7a76400700 1 Processor -- start 2024-05-27T10:19:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 -- start start 2024-05-27T10:19:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 0x7f7a7019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a70107de0 0x7f7a7019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70108730 0x7f7a701a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70113210 con 0x7f7a701034b0 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70113090 con 0x7f7a70107de0 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.165+0000 7f7a76400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70113390 con 0x7f7a70108730 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 0x7f7a7019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 0x7f7a7019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42940/0 (socket says 172.21.15.42:42940) 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 -- 172.21.15.42:0/1708826232 learned_addr learned my addr 172.21.15.42:0/1708826232 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6ea00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a70107de0 0x7f7a7019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6fe00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70108730 0x7f7a701a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 -- 172.21.15.42:0/1708826232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70108730 msgr2=0x7f7a701a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70108730 0x7f7a701a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 -- 172.21.15.42:0/1708826232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a70107de0 msgr2=0x7f7a7019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a70107de0 0x7f7a7019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6f400700 1 -- 172.21.15.42:0/1708826232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a701a1d70 con 0x7f7a701034b0 2024-05-27T10:19:10.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6fe00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70108730 0x7f7a701a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:10.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.166+0000 7f7a6ea00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a70107de0 0x7f7a7019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:10.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.167+0000 7f7a6f400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 0x7f7a7019cea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d950 tx=0x7f7a6000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.167+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60016070 con 0x7f7a701034b0 2024-05-27T10:19:10.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.167+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a60010460 con 0x7f7a701034b0 2024-05-27T10:19:10.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.167+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000f620 con 0x7f7a701034b0 2024-05-27T10:19:10.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.167+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a701a2060 con 0x7f7a701034b0 2024-05-27T10:19:10.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.168+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a9980 con 0x7f7a701034b0 2024-05-27T10:19:10.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.169+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a700512b0 con 0x7f7a701034b0 2024-05-27T10:19:10.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.172+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f7a60019070 con 0x7f7a701034b0 2024-05-27T10:19:10.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.174+0000 7f7a67e00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7a540b2b90 0x7f7a540b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:10.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.174+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7a600e2490 con 0x7f7a701034b0 2024-05-27T10:19:10.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.174+0000 7f7a6ea00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7a540b2b90 0x7f7a540b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:10.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.175+0000 7f7a6ea00700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7a540b2b90 0x7f7a540b50a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005e10 tx=0x7f7a5c005d60 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:10.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.176+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a600a9c10 con 0x7f7a701034b0 2024-05-27T10:19:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.417+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a7019e200 con 0x7f7a540b2b90 2024-05-27T10:19:10.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.428+0000 7f7a67e00700 1 -- 172.21.15.42:0/1708826232 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258054 (secure 0 0 0) 0x7f7a7019e200 con 0x7f7a540b2b90 2024-05-27T10:19:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7a540b2b90 msgr2=0x7f7a540b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7a540b2b90 0x7f7a540b50a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005e10 tx=0x7f7a5c005d60 comp rx=0 tx=0).stop 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 msgr2=0x7f7a7019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 0x7f7a7019cea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d950 tx=0x7f7a6000dd10 comp rx=0 tx=0).stop 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 shutdown_connections 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7a540b2b90 0x7f7a540b50a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a70108730 0x7f7a701a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a701034b0 0x7f7a7019cea0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 --2- 172.21.15.42:0/1708826232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a70107de0 0x7f7a7019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.433+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 >> 172.21.15.42:0/1708826232 conn(0x7f7a700fedb0 msgr2=0x7f7a7010fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.434+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 shutdown_connections 2024-05-27T10:19:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:10.434+0000 7f7a76400700 1 -- 172.21.15.42:0/1708826232 wait complete. 2024-05-27T10:19:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:19:11.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:10 smithi042 ceph-mon[31661]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T10:19:11.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:10 smithi104 ceph-mon[34812]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T10:19:11.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:10 smithi143 ceph-mon[35545]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T10:19:11.389 INFO:tasks.cephfs.filesystem:scrub status for tag:cc420705-6a6a-4d62-96fa-afbcd6b325d7 - {'path': '/', 'tag': 'cc420705-6a6a-4d62-96fa-afbcd6b325d7', 'options': 'recursive,force'} 2024-05-27T10:19:12.022 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-05-27T10:19:08.901784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386679,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1957,"num_read_kb":1312,"num_write":506,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":882,"ondisk_log_size":882,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360284,"kb_used_data":11108,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581540,"statfs":{"total":1151940427776,"available":1151571496960,"internally_reserved":0,"allocated":11374592,"data_stored":9265769,"data_compressed":239239,"data_compressed_allocated":311296,"data_compressed_original":3846144,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":130,"num_write":64,"num_write_kb":652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005129"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866630+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:18:42.866630+0000","last_peered":"2024-05-27T10:18:42.866630+0000","last_clean":"2024-05-27T10:18:42.866630+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:18:42.866630+0000","last_undegraded":"2024-05-27T10:18:42.866630+0000","last_fullsized":"2024-05-27T10:18:42.866630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035777+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:18:43.035777+0000","last_peered":"2024-05-27T10:18:43.035777+0000","last_clean":"2024-05-27T10:18:43.035777+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:18:43.035777+0000","last_undegraded":"2024-05-27T10:18:43.035777+0000","last_fullsized":"2024-05-27T10:18:43.035777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036368+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:18:43.036368+0000","last_peered":"2024-05-27T10:18:43.036368+0000","last_clean":"2024-05-27T10:18:43.036368+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:18:43.036368+0000","last_undegraded":"2024-05-27T10:18:43.036368+0000","last_fullsized":"2024-05-27T10:18:43.036368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011005+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:18:43.011005+0000","last_peered":"2024-05-27T10:18:43.011005+0000","last_clean":"2024-05-27T10:18:43.011005+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:18:43.011005+0000","last_undegraded":"2024-05-27T10:18:43.011005+0000","last_fullsized":"2024-05-27T10:18:43.011005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149389+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:18:43.149389+0000","last_peered":"2024-05-27T10:18:43.149389+0000","last_clean":"2024-05-27T10:18:43.149389+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:18:43.149389+0000","last_undegraded":"2024-05-27T10:18:43.149389+0000","last_fullsized":"2024-05-27T10:18:43.149389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864338+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:18:42.864338+0000","last_peered":"2024-05-27T10:18:42.864338+0000","last_clean":"2024-05-27T10:18:42.864338+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:18:42.864338+0000","last_undegraded":"2024-05-27T10:18:42.864338+0000","last_fullsized":"2024-05-27T10:18:42.864338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914557+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:18:42.914557+0000","last_peered":"2024-05-27T10:18:42.914557+0000","last_clean":"2024-05-27T10:18:42.914557+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:18:42.914557+0000","last_undegraded":"2024-05-27T10:18:42.914557+0000","last_fullsized":"2024-05-27T10:18:42.914557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013138+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:18:43.013138+0000","last_peered":"2024-05-27T10:18:43.013138+0000","last_clean":"2024-05-27T10:18:43.013138+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:18:43.013138+0000","last_undegraded":"2024-05-27T10:18:43.013138+0000","last_fullsized":"2024-05-27T10:18:43.013138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913466+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:18:42.913466+0000","last_peered":"2024-05-27T10:18:42.913466+0000","last_clean":"2024-05-27T10:18:42.913466+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:18:42.913466+0000","last_undegraded":"2024-05-27T10:18:42.913466+0000","last_fullsized":"2024-05-27T10:18:42.913466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865113+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:18:42.865113+0000","last_peered":"2024-05-27T10:18:42.865113+0000","last_clean":"2024-05-27T10:18:42.865113+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:18:42.865113+0000","last_undegraded":"2024-05-27T10:18:42.865113+0000","last_fullsized":"2024-05-27T10:18:42.865113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915691+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:18:42.915691+0000","last_peered":"2024-05-27T10:18:42.915691+0000","last_clean":"2024-05-27T10:18:42.915691+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:18:42.915691+0000","last_undegraded":"2024-05-27T10:18:42.915691+0000","last_fullsized":"2024-05-27T10:18:42.915691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036299+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:18:43.036299+0000","last_peered":"2024-05-27T10:18:43.036299+0000","last_clean":"2024-05-27T10:18:43.036299+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:18:43.036299+0000","last_undegraded":"2024-05-27T10:18:43.036299+0000","last_fullsized":"2024-05-27T10:18:43.036299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858317+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:18:42.858317+0000","last_peered":"2024-05-27T10:18:42.858317+0000","last_clean":"2024-05-27T10:18:42.858317+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:18:42.858317+0000","last_undegraded":"2024-05-27T10:18:42.858317+0000","last_fullsized":"2024-05-27T10:18:42.858317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036021+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:18:43.036021+0000","last_peered":"2024-05-27T10:18:43.036021+0000","last_clean":"2024-05-27T10:18:43.036021+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:18:43.036021+0000","last_undegraded":"2024-05-27T10:18:43.036021+0000","last_fullsized":"2024-05-27T10:18:43.036021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035733+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:18:43.035733+0000","last_peered":"2024-05-27T10:18:43.035733+0000","last_clean":"2024-05-27T10:18:43.035733+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:18:43.035733+0000","last_undegraded":"2024-05-27T10:18:43.035733+0000","last_fullsized":"2024-05-27T10:18:43.035733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.911945+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:18:42.911945+0000","last_peered":"2024-05-27T10:18:42.911945+0000","last_clean":"2024-05-27T10:18:42.911945+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:18:42.911945+0000","last_undegraded":"2024-05-27T10:18:42.911945+0000","last_fullsized":"2024-05-27T10:18:42.911945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.867203+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:18:42.867203+0000","last_peered":"2024-05-27T10:18:42.867203+0000","last_clean":"2024-05-27T10:18:42.867203+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:18:42.867203+0000","last_undegraded":"2024-05-27T10:18:42.867203+0000","last_fullsized":"2024-05-27T10:18:42.867203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011126+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:18:43.011126+0000","last_peered":"2024-05-27T10:18:43.011126+0000","last_clean":"2024-05-27T10:18:43.011126+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:18:43.011126+0000","last_undegraded":"2024-05-27T10:18:43.011126+0000","last_fullsized":"2024-05-27T10:18:43.011126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892271+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:18:42.892271+0000","last_peered":"2024-05-27T10:18:42.892271+0000","last_clean":"2024-05-27T10:18:42.892271+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:18:42.892271+0000","last_undegraded":"2024-05-27T10:18:42.892271+0000","last_fullsized":"2024-05-27T10:18:42.892271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010950+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:18:43.010950+0000","last_peered":"2024-05-27T10:18:43.010950+0000","last_clean":"2024-05-27T10:18:43.010950+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:18:43.010950+0000","last_undegraded":"2024-05-27T10:18:43.010950+0000","last_fullsized":"2024-05-27T10:18:43.010950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013844+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:18:43.013844+0000","last_peered":"2024-05-27T10:18:43.013844+0000","last_clean":"2024-05-27T10:18:43.013844+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:18:43.013844+0000","last_undegraded":"2024-05-27T10:18:43.013844+0000","last_fullsized":"2024-05-27T10:18:43.013844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013898+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:18:43.013898+0000","last_peered":"2024-05-27T10:18:43.013898+0000","last_clean":"2024-05-27T10:18:43.013898+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:18:43.013898+0000","last_undegraded":"2024-05-27T10:18:43.013898+0000","last_fullsized":"2024-05-27T10:18:43.013898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036523+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:18:43.036523+0000","last_peered":"2024-05-27T10:18:43.036523+0000","last_clean":"2024-05-27T10:18:43.036523+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:18:43.036523+0000","last_undegraded":"2024-05-27T10:18:43.036523+0000","last_fullsized":"2024-05-27T10:18:43.036523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150318+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:18:43.150318+0000","last_peered":"2024-05-27T10:18:43.150318+0000","last_clean":"2024-05-27T10:18:43.150318+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:18:43.150318+0000","last_undegraded":"2024-05-27T10:18:43.150318+0000","last_fullsized":"2024-05-27T10:18:43.150318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912126+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:18:42.912126+0000","last_peered":"2024-05-27T10:18:42.912126+0000","last_clean":"2024-05-27T10:18:42.912126+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:18:42.912126+0000","last_undegraded":"2024-05-27T10:18:42.912126+0000","last_fullsized":"2024-05-27T10:18:42.912126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150197+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:18:43.150197+0000","last_peered":"2024-05-27T10:18:43.150197+0000","last_clean":"2024-05-27T10:18:43.150197+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:18:43.150197+0000","last_undegraded":"2024-05-27T10:18:43.150197+0000","last_fullsized":"2024-05-27T10:18:43.150197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029108+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:18:43.029108+0000","last_peered":"2024-05-27T10:18:43.029108+0000","last_clean":"2024-05-27T10:18:43.029108+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:18:43.029108+0000","last_undegraded":"2024-05-27T10:18:43.029108+0000","last_fullsized":"2024-05-27T10:18:43.029108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'11","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:02.491109+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:19:02.491109+0000","last_peered":"2024-05-27T10:19:02.491109+0000","last_clean":"2024-05-27T10:19:02.491109+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:19:02.491109+0000","last_undegraded":"2024-05-27T10:19:02.491109+0000","last_fullsized":"2024-05-27T10:19:02.491109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":268,"num_read_kb":7,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149127+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:18:43.149127+0000","last_peered":"2024-05-27T10:18:43.149127+0000","last_clean":"2024-05-27T10:18:43.149127+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:18:43.149127+0000","last_undegraded":"2024-05-27T10:18:43.149127+0000","last_fullsized":"2024-05-27T10:18:43.149127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864000+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:18:42.864000+0000","last_peered":"2024-05-27T10:18:42.864000+0000","last_clean":"2024-05-27T10:18:42.864000+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:18:42.864000+0000","last_undegraded":"2024-05-27T10:18:42.864000+0000","last_fullsized":"2024-05-27T10:18:42.864000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028881+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:18:43.028881+0000","last_peered":"2024-05-27T10:18:43.028881+0000","last_clean":"2024-05-27T10:18:43.028881+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:18:43.028881+0000","last_undegraded":"2024-05-27T10:18:43.028881+0000","last_fullsized":"2024-05-27T10:18:43.028881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857824+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:18:42.857824+0000","last_peered":"2024-05-27T10:18:42.857824+0000","last_clean":"2024-05-27T10:18:42.857824+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:18:42.857824+0000","last_undegraded":"2024-05-27T10:18:42.857824+0000","last_fullsized":"2024-05-27T10:18:42.857824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013087+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:18:43.013087+0000","last_peered":"2024-05-27T10:18:43.013087+0000","last_clean":"2024-05-27T10:18:43.013087+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:18:43.013087+0000","last_undegraded":"2024-05-27T10:18:43.013087+0000","last_fullsized":"2024-05-27T10:18:43.013087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.027835+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:18:43.027835+0000","last_peered":"2024-05-27T10:18:43.027835+0000","last_clean":"2024-05-27T10:18:43.027835+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:18:43.027835+0000","last_undegraded":"2024-05-27T10:18:43.027835+0000","last_fullsized":"2024-05-27T10:18:43.027835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010824+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:18:43.010824+0000","last_peered":"2024-05-27T10:18:43.010824+0000","last_clean":"2024-05-27T10:18:43.010824+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:18:43.010824+0000","last_undegraded":"2024-05-27T10:18:43.010824+0000","last_fullsized":"2024-05-27T10:18:43.010824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":297,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:02.489779+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:19:02.489779+0000","last_peered":"2024-05-27T10:19:02.489779+0000","last_clean":"2024-05-27T10:19:02.489779+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:19:02.489779+0000","last_undegraded":"2024-05-27T10:19:02.489779+0000","last_fullsized":"2024-05-27T10:19:02.489779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913526+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:18:42.913526+0000","last_peered":"2024-05-27T10:18:42.913526+0000","last_clean":"2024-05-27T10:18:42.913526+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:18:42.913526+0000","last_undegraded":"2024-05-27T10:18:42.913526+0000","last_fullsized":"2024-05-27T10:18:42.913526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915807+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:18:42.915807+0000","last_peered":"2024-05-27T10:18:42.915807+0000","last_clean":"2024-05-27T10:18:42.915807+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:18:42.915807+0000","last_undegraded":"2024-05-27T10:18:42.915807+0000","last_fullsized":"2024-05-27T10:18:42.915807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036613+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:18:43.036613+0000","last_peered":"2024-05-27T10:18:43.036613+0000","last_clean":"2024-05-27T10:18:43.036613+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:18:43.036613+0000","last_undegraded":"2024-05-27T10:18:43.036613+0000","last_fullsized":"2024-05-27T10:18:43.036613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029503+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:18:43.029503+0000","last_peered":"2024-05-27T10:18:43.029503+0000","last_clean":"2024-05-27T10:18:43.029503+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:18:43.029503+0000","last_undegraded":"2024-05-27T10:18:43.029503+0000","last_fullsized":"2024-05-27T10:18:43.029503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912674+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:18:42.912674+0000","last_peered":"2024-05-27T10:18:42.912674+0000","last_clean":"2024-05-27T10:18:42.912674+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:18:42.912674+0000","last_undegraded":"2024-05-27T10:18:42.912674+0000","last_fullsized":"2024-05-27T10:18:42.912674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":604,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:03.440123+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:19:03.440123+0000","last_peered":"2024-05-27T10:19:03.440123+0000","last_clean":"2024-05-27T10:19:03.440123+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:19:03.440123+0000","last_undegraded":"2024-05-27T10:19:03.440123+0000","last_fullsized":"2024-05-27T10:19:03.440123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":537,"num_read_kb":334,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150023+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:18:43.150023+0000","last_peered":"2024-05-27T10:18:43.150023+0000","last_clean":"2024-05-27T10:18:43.150023+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:18:43.150023+0000","last_undegraded":"2024-05-27T10:18:43.150023+0000","last_fullsized":"2024-05-27T10:18:43.150023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892407+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:18:42.892407+0000","last_peered":"2024-05-27T10:18:42.892407+0000","last_clean":"2024-05-27T10:18:42.892407+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:18:42.892407+0000","last_undegraded":"2024-05-27T10:18:42.892407+0000","last_fullsized":"2024-05-27T10:18:42.892407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858066+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:18:42.858066+0000","last_peered":"2024-05-27T10:18:42.858066+0000","last_clean":"2024-05-27T10:18:42.858066+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:18:42.858066+0000","last_undegraded":"2024-05-27T10:18:42.858066+0000","last_fullsized":"2024-05-27T10:18:42.858066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028359+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:18:43.028359+0000","last_peered":"2024-05-27T10:18:43.028359+0000","last_clean":"2024-05-27T10:18:43.028359+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:18:43.028359+0000","last_undegraded":"2024-05-27T10:18:43.028359+0000","last_fullsized":"2024-05-27T10:18:43.028359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915378+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:18:42.915378+0000","last_peered":"2024-05-27T10:18:42.915378+0000","last_clean":"2024-05-27T10:18:42.915378+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:18:42.915378+0000","last_undegraded":"2024-05-27T10:18:42.915378+0000","last_fullsized":"2024-05-27T10:18:42.915378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":305,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:02.475243+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:19:02.475243+0000","last_peered":"2024-05-27T10:19:02.475243+0000","last_clean":"2024-05-27T10:19:02.475243+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:19:02.475243+0000","last_undegraded":"2024-05-27T10:19:02.475243+0000","last_fullsized":"2024-05-27T10:19:02.475243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":269,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914514+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:18:42.914514+0000","last_peered":"2024-05-27T10:18:42.914514+0000","last_clean":"2024-05-27T10:18:42.914514+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:18:42.914514+0000","last_undegraded":"2024-05-27T10:18:42.914514+0000","last_fullsized":"2024-05-27T10:18:42.914514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148905+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:18:43.148905+0000","last_peered":"2024-05-27T10:18:43.148905+0000","last_clean":"2024-05-27T10:18:43.148905+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:18:43.148905+0000","last_undegraded":"2024-05-27T10:18:43.148905+0000","last_fullsized":"2024-05-27T10:18:43.148905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028385+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:18:43.028385+0000","last_peered":"2024-05-27T10:18:43.028385+0000","last_clean":"2024-05-27T10:18:43.028385+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:18:43.028385+0000","last_undegraded":"2024-05-27T10:18:43.028385+0000","last_fullsized":"2024-05-27T10:18:43.028385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.641820+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:18:43.641820+0000","last_peered":"2024-05-27T10:18:43.641820+0000","last_clean":"2024-05-27T10:18:43.641820+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:18:43.641820+0000","last_undegraded":"2024-05-27T10:18:43.641820+0000","last_fullsized":"2024-05-27T10:18:43.641820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865929+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:18:42.865929+0000","last_peered":"2024-05-27T10:18:42.865929+0000","last_clean":"2024-05-27T10:18:42.865929+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:18:42.865929+0000","last_undegraded":"2024-05-27T10:18:42.865929+0000","last_fullsized":"2024-05-27T10:18:42.865929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010814+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:18:43.010814+0000","last_peered":"2024-05-27T10:18:43.010814+0000","last_clean":"2024-05-27T10:18:43.010814+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:18:43.010814+0000","last_undegraded":"2024-05-27T10:18:43.010814+0000","last_fullsized":"2024-05-27T10:18:43.010814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864668+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:18:42.864668+0000","last_peered":"2024-05-27T10:18:42.864668+0000","last_clean":"2024-05-27T10:18:42.864668+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:18:42.864668+0000","last_undegraded":"2024-05-27T10:18:42.864668+0000","last_fullsized":"2024-05-27T10:18:42.864668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914915+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:18:42.914915+0000","last_peered":"2024-05-27T10:18:42.914915+0000","last_clean":"2024-05-27T10:18:42.914915+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:18:42.914915+0000","last_undegraded":"2024-05-27T10:18:42.914915+0000","last_fullsized":"2024-05-27T10:18:42.914915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865657+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:18:42.865657+0000","last_peered":"2024-05-27T10:18:42.865657+0000","last_clean":"2024-05-27T10:18:42.865657+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:18:42.865657+0000","last_undegraded":"2024-05-27T10:18:42.865657+0000","last_fullsized":"2024-05-27T10:18:42.865657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028265+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:18:43.028265+0000","last_peered":"2024-05-27T10:18:43.028265+0000","last_clean":"2024-05-27T10:18:43.028265+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:18:43.028265+0000","last_undegraded":"2024-05-27T10:18:43.028265+0000","last_fullsized":"2024-05-27T10:18:43.028265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":298,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:02.474024+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:19:02.474024+0000","last_peered":"2024-05-27T10:19:02.474024+0000","last_clean":"2024-05-27T10:19:02.474024+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:19:02.474024+0000","last_undegraded":"2024-05-27T10:19:02.474024+0000","last_fullsized":"2024-05-27T10:19:02.474024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013516+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:18:43.013516+0000","last_peered":"2024-05-27T10:18:43.013516+0000","last_clean":"2024-05-27T10:18:43.013516+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:18:43.013516+0000","last_undegraded":"2024-05-27T10:18:43.013516+0000","last_fullsized":"2024-05-27T10:18:43.013516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036179+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:18:43.036179+0000","last_peered":"2024-05-27T10:18:43.036179+0000","last_clean":"2024-05-27T10:18:43.036179+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:18:43.036179+0000","last_undegraded":"2024-05-27T10:18:43.036179+0000","last_fullsized":"2024-05-27T10:18:43.036179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"82'99","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:45.052019+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:18:45.052019+0000","last_peered":"2024-05-27T10:18:45.052019+0000","last_clean":"2024-05-27T10:18:45.052019+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:18:45.052019+0000","last_undegraded":"2024-05-27T10:18:45.052019+0000","last_fullsized":"2024-05-27T10:18:45.052019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857424+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:18:42.857424+0000","last_peered":"2024-05-27T10:18:42.857424+0000","last_clean":"2024-05-27T10:18:42.857424+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:18:42.857424+0000","last_undegraded":"2024-05-27T10:18:42.857424+0000","last_fullsized":"2024-05-27T10:18:42.857424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914885+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:18:42.914885+0000","last_peered":"2024-05-27T10:18:42.914885+0000","last_clean":"2024-05-27T10:18:42.914885+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:18:42.914885+0000","last_undegraded":"2024-05-27T10:18:42.914885+0000","last_fullsized":"2024-05-27T10:18:42.914885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149983+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:18:43.149983+0000","last_peered":"2024-05-27T10:18:43.149983+0000","last_clean":"2024-05-27T10:18:43.149983+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:18:43.149983+0000","last_undegraded":"2024-05-27T10:18:43.149983+0000","last_fullsized":"2024-05-27T10:18:43.149983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858159+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:18:42.858159+0000","last_peered":"2024-05-27T10:18:42.858159+0000","last_clean":"2024-05-27T10:18:42.858159+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:18:42.858159+0000","last_undegraded":"2024-05-27T10:18:42.858159+0000","last_fullsized":"2024-05-27T10:18:42.858159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'74","reported_seq":169,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:03.511455+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:19:03.511455+0000","last_peered":"2024-05-27T10:19:03.511455+0000","last_clean":"2024-05-27T10:19:03.511455+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:19:03.511455+0000","last_undegraded":"2024-05-27T10:19:03.511455+0000","last_fullsized":"2024-05-27T10:19:03.511455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":134,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.014095+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:18:43.014095+0000","last_peered":"2024-05-27T10:18:43.014095+0000","last_clean":"2024-05-27T10:18:43.014095+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:18:43.014095+0000","last_undegraded":"2024-05-27T10:18:43.014095+0000","last_fullsized":"2024-05-27T10:18:43.014095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036143+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:18:43.036143+0000","last_peered":"2024-05-27T10:18:43.036143+0000","last_clean":"2024-05-27T10:18:43.036143+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:18:43.036143+0000","last_undegraded":"2024-05-27T10:18:43.036143+0000","last_fullsized":"2024-05-27T10:18:43.036143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035987+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:18:43.035987+0000","last_peered":"2024-05-27T10:18:43.035987+0000","last_clean":"2024-05-27T10:18:43.035987+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:18:43.035987+0000","last_undegraded":"2024-05-27T10:18:43.035987+0000","last_fullsized":"2024-05-27T10:18:43.035987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910322+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:18:42.910322+0000","last_peered":"2024-05-27T10:18:42.910322+0000","last_clean":"2024-05-27T10:18:42.910322+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:18:42.910322+0000","last_undegraded":"2024-05-27T10:18:42.910322+0000","last_fullsized":"2024-05-27T10:18:42.910322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148975+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:18:43.148975+0000","last_peered":"2024-05-27T10:18:43.148975+0000","last_clean":"2024-05-27T10:18:43.148975+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:18:43.148975+0000","last_undegraded":"2024-05-27T10:18:43.148975+0000","last_fullsized":"2024-05-27T10:18:43.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013173+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:18:43.013173+0000","last_peered":"2024-05-27T10:18:43.013173+0000","last_clean":"2024-05-27T10:18:43.013173+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:18:43.013173+0000","last_undegraded":"2024-05-27T10:18:43.013173+0000","last_fullsized":"2024-05-27T10:18:43.013173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865624+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:18:42.865624+0000","last_peered":"2024-05-27T10:18:42.865624+0000","last_clean":"2024-05-27T10:18:42.865624+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:18:42.865624+0000","last_undegraded":"2024-05-27T10:18:42.865624+0000","last_fullsized":"2024-05-27T10:18:42.865624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865650+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:18:42.865650+0000","last_peered":"2024-05-27T10:18:42.865650+0000","last_clean":"2024-05-27T10:18:42.865650+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:18:42.865650+0000","last_undegraded":"2024-05-27T10:18:42.865650+0000","last_fullsized":"2024-05-27T10:18:42.865650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013727+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:18:43.013727+0000","last_peered":"2024-05-27T10:18:43.013727+0000","last_clean":"2024-05-27T10:18:43.013727+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:18:43.013727+0000","last_undegraded":"2024-05-27T10:18:43.013727+0000","last_fullsized":"2024-05-27T10:18:43.013727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150814+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:18:43.150814+0000","last_peered":"2024-05-27T10:18:43.150814+0000","last_clean":"2024-05-27T10:18:43.150814+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:18:43.150814+0000","last_undegraded":"2024-05-27T10:18:43.150814+0000","last_fullsized":"2024-05-27T10:18:43.150814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915235+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:18:42.915235+0000","last_peered":"2024-05-27T10:18:42.915235+0000","last_clean":"2024-05-27T10:18:42.915235+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:18:42.915235+0000","last_undegraded":"2024-05-27T10:18:42.915235+0000","last_fullsized":"2024-05-27T10:18:42.915235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892674+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:18:42.892674+0000","last_peered":"2024-05-27T10:18:42.892674+0000","last_clean":"2024-05-27T10:18:42.892674+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:18:42.892674+0000","last_undegraded":"2024-05-27T10:18:42.892674+0000","last_fullsized":"2024-05-27T10:18:42.892674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916054+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:18:42.916054+0000","last_peered":"2024-05-27T10:18:42.916054+0000","last_clean":"2024-05-27T10:18:42.916054+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:18:42.916054+0000","last_undegraded":"2024-05-27T10:18:42.916054+0000","last_fullsized":"2024-05-27T10:18:42.916054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'57","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:03.495914+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:19:03.495914+0000","last_peered":"2024-05-27T10:19:03.495914+0000","last_clean":"2024-05-27T10:19:03.495914+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:19:03.495914+0000","last_undegraded":"2024-05-27T10:19:03.495914+0000","last_fullsized":"2024-05-27T10:19:03.495914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915511+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:18:42.915511+0000","last_peered":"2024-05-27T10:18:42.915511+0000","last_clean":"2024-05-27T10:18:42.915511+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:18:42.915511+0000","last_undegraded":"2024-05-27T10:18:42.915511+0000","last_fullsized":"2024-05-27T10:18:42.915511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858229+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:18:42.858229+0000","last_peered":"2024-05-27T10:18:42.858229+0000","last_clean":"2024-05-27T10:18:42.858229+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:18:42.858229+0000","last_undegraded":"2024-05-27T10:18:42.858229+0000","last_fullsized":"2024-05-27T10:18:42.858229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916095+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:18:42.916095+0000","last_peered":"2024-05-27T10:18:42.916095+0000","last_clean":"2024-05-27T10:18:42.916095+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:18:42.916095+0000","last_undegraded":"2024-05-27T10:18:42.916095+0000","last_fullsized":"2024-05-27T10:18:42.916095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865960+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:18:42.865960+0000","last_peered":"2024-05-27T10:18:42.865960+0000","last_clean":"2024-05-27T10:18:42.865960+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:18:42.865960+0000","last_undegraded":"2024-05-27T10:18:42.865960+0000","last_fullsized":"2024-05-27T10:18:42.865960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892084+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:18:42.892084+0000","last_peered":"2024-05-27T10:18:42.892084+0000","last_clean":"2024-05-27T10:18:42.892084+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:18:42.892084+0000","last_undegraded":"2024-05-27T10:18:42.892084+0000","last_fullsized":"2024-05-27T10:18:42.892084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.891669+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:18:42.891669+0000","last_peered":"2024-05-27T10:18:42.891669+0000","last_clean":"2024-05-27T10:18:42.891669+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:18:42.891669+0000","last_undegraded":"2024-05-27T10:18:42.891669+0000","last_fullsized":"2024-05-27T10:18:42.891669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036165+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:18:43.036165+0000","last_peered":"2024-05-27T10:18:43.036165+0000","last_clean":"2024-05-27T10:18:43.036165+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:18:43.036165+0000","last_undegraded":"2024-05-27T10:18:43.036165+0000","last_fullsized":"2024-05-27T10:18:43.036165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858813+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:18:42.858813+0000","last_peered":"2024-05-27T10:18:42.858813+0000","last_clean":"2024-05-27T10:18:42.858813+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:18:42.858813+0000","last_undegraded":"2024-05-27T10:18:42.858813+0000","last_fullsized":"2024-05-27T10:18:42.858813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865913+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:18:42.865913+0000","last_peered":"2024-05-27T10:18:42.865913+0000","last_clean":"2024-05-27T10:18:42.865913+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:18:42.865913+0000","last_undegraded":"2024-05-27T10:18:42.865913+0000","last_fullsized":"2024-05-27T10:18:42.865913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892758+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:18:42.892758+0000","last_peered":"2024-05-27T10:18:42.892758+0000","last_clean":"2024-05-27T10:18:42.892758+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:18:42.892758+0000","last_undegraded":"2024-05-27T10:18:42.892758+0000","last_fullsized":"2024-05-27T10:18:42.892758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010964+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:18:43.010964+0000","last_peered":"2024-05-27T10:18:43.010964+0000","last_clean":"2024-05-27T10:18:43.010964+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:18:43.010964+0000","last_undegraded":"2024-05-27T10:18:43.010964+0000","last_fullsized":"2024-05-27T10:18:43.010964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028084+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:18:43.028084+0000","last_peered":"2024-05-27T10:18:43.028084+0000","last_clean":"2024-05-27T10:18:43.028084+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:18:43.028084+0000","last_undegraded":"2024-05-27T10:18:43.028084+0000","last_fullsized":"2024-05-27T10:18:43.028084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013207+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:18:43.013207+0000","last_peered":"2024-05-27T10:18:43.013207+0000","last_clean":"2024-05-27T10:18:43.013207+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:18:43.013207+0000","last_undegraded":"2024-05-27T10:18:43.013207+0000","last_fullsized":"2024-05-27T10:18:43.013207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913669+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:18:42.913669+0000","last_peered":"2024-05-27T10:18:42.913669+0000","last_clean":"2024-05-27T10:18:42.913669+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:18:42.913669+0000","last_undegraded":"2024-05-27T10:18:42.913669+0000","last_fullsized":"2024-05-27T10:18:42.913669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858143+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:18:42.858143+0000","last_peered":"2024-05-27T10:18:42.858143+0000","last_clean":"2024-05-27T10:18:42.858143+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:18:42.858143+0000","last_undegraded":"2024-05-27T10:18:42.858143+0000","last_fullsized":"2024-05-27T10:18:42.858143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866758+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:18:42.866758+0000","last_peered":"2024-05-27T10:18:42.866758+0000","last_clean":"2024-05-27T10:18:42.866758+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:18:42.866758+0000","last_undegraded":"2024-05-27T10:18:42.866758+0000","last_fullsized":"2024-05-27T10:18:42.866758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796077,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1813,"num_read_kb":1064,"num_write":369,"num_write_kb":2099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2388231,"data_compressed":102345,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678093,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":830200,"data_compressed":22168,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874324,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1425518,"data_compressed":53885,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":830436,"data_compressed":22168,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":437140,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430314,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1032293,"data_compressed":39045,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593842,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":888832,"data_stored":436609,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":503807,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738527,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":437880,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":507355,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149833,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1027276,"data_compressed":30730,"data_compressed_allocated":40960,"data_compressed_original":585728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953603,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":438462,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790074,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1358793,"data_compressed":53885,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34115,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331906,"data_compressed":14840,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":398631,"data_compressed":14840,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19275,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:19:12.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:19:12.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:11 smithi042 ceph-mon[31661]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:12.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:11 smithi104 ceph-mon[34812]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:12.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:11 smithi143 ceph-mon[35545]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:12.519 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:13.053 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:12 smithi042 ceph-mon[31661]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-27T10:19:13.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:12 smithi104 ceph-mon[34812]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-27T10:19:13.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:12 smithi143 ceph-mon[35545]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-27T10:19:15.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:14 smithi042 ceph-mon[31661]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-27T10:19:15.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:14 smithi104 ceph-mon[34812]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-27T10:19:15.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 -- 172.21.15.42:0/2853325213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810b890 msgr2=0x7fe8481131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:15.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 --2- 172.21.15.42:0/2853325213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810b890 0x7fe8481131f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b0d0 tx=0x7fe84000b3e0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 -- 172.21.15.42:0/2853325213 shutdown_connections 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 --2- 172.21.15.42:0/2853325213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810b890 0x7fe8481131f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 --2- 172.21.15.42:0/2853325213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe848106610 0x7fe848106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 --2- 172.21.15.42:0/2853325213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810af40 0x7fe84810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 -- 172.21.15.42:0/2853325213 >> 172.21.15.42:0/2853325213 conn(0x7fe848102020 msgr2=0x7fe848104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 -- 172.21.15.42:0/2853325213 shutdown_connections 2024-05-27T10:19:15.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.212+0000 7fe847e00700 1 -- 172.21.15.42:0/2853325213 wait complete. 2024-05-27T10:19:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.214+0000 7fe847e00700 1 Processor -- start 2024-05-27T10:19:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.214+0000 7fe847e00700 1 -- start start 2024-05-27T10:19:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.215+0000 7fe847e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe848106610 0x7fe84819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.215+0000 7fe847e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 0x7fe84819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.215+0000 7fe847e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810b890 0x7fe8481a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.215+0000 7fe847e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8481163c0 con 0x7fe84810af40 2024-05-27T10:19:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.215+0000 7fe847e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe848116240 con 0x7fe84810b890 2024-05-27T10:19:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.215+0000 7fe847e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe848116540 con 0x7fe848106610 2024-05-27T10:19:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 0x7fe84819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 0x7fe84819d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40128/0 (socket says 172.21.15.42:40128) 2024-05-27T10:19:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 -- 172.21.15.42:0/4234614175 learned_addr learned my addr 172.21.15.42:0/4234614175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe846a00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe848106610 0x7fe84819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe847400700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810b890 0x7fe8481a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 -- 172.21.15.42:0/4234614175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe848106610 msgr2=0x7fe84819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe848106610 0x7fe84819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 -- 172.21.15.42:0/4234614175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810b890 msgr2=0x7fe8481a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810b890 0x7fe8481a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe83e000700 1 -- 172.21.15.42:0/4234614175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8481a1c20 con 0x7fe84810af40 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.216+0000 7fe847400700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810b890 0x7fe8481a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.217+0000 7fe83e000700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 0x7fe84819d2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe83000d980 tx=0x7fe83000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.217+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830017070 con 0x7fe84810af40 2024-05-27T10:19:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.217+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe830010460 con 0x7fe84810af40 2024-05-27T10:19:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.217+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83000de60 con 0x7fe84810af40 2024-05-27T10:19:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.217+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8481a1f10 con 0x7fe84810af40 2024-05-27T10:19:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.217+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8481a9830 con 0x7fe84810af40 2024-05-27T10:19:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.218+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8480512b0 con 0x7fe84810af40 2024-05-27T10:19:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.221+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fe830010610 con 0x7fe84810af40 2024-05-27T10:19:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.224+0000 7fe83fe00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe82c0b2b90 0x7fe82c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.224+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe8300e22f0 con 0x7fe84810af40 2024-05-27T10:19:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.224+0000 7fe846a00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe82c0b2b90 0x7fe82c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.225+0000 7fe846a00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe82c0b2b90 0x7fe82c0b50a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe838005950 tx=0x7fe8380058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:15.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.227+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8300a9af0 con 0x7fe84810af40 2024-05-27T10:19:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:14 smithi143 ceph-mon[35545]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-27T10:19:15.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.463+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe848063bf0 con 0x7fe82c0b2b90 2024-05-27T10:19:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.475+0000 7fe83fe00700 1 -- 172.21.15.42:0/4234614175 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258022 (secure 0 0 0) 0x7fe848063bf0 con 0x7fe82c0b2b90 2024-05-27T10:19:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:15.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe82c0b2b90 msgr2=0x7fe82c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:15.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe82c0b2b90 0x7fe82c0b50a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe838005950 tx=0x7fe8380058e0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 msgr2=0x7fe84819d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 0x7fe84819d2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe83000d980 tx=0x7fe83000dd40 comp rx=0 tx=0).stop 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 shutdown_connections 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe82c0b2b90 0x7fe82c0b50a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe84810b890 0x7fe8481a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe848106610 0x7fe84819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 --2- 172.21.15.42:0/4234614175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84810af40 0x7fe84819d2f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:15.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 >> 172.21.15.42:0/4234614175 conn(0x7fe848102020 msgr2=0x7fe84810a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:15.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 shutdown_connections 2024-05-27T10:19:15.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:15.482+0000 7fe847e00700 1 -- 172.21.15.42:0/4234614175 wait complete. 2024-05-27T10:19:15.486 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:19:16.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:15 smithi104 ceph-mon[34812]: scrub summary: idle 2024-05-27T10:19:16.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:15 smithi143 ceph-mon[35545]: scrub summary: idle 2024-05-27T10:19:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:15 smithi042 ceph-mon[31661]: scrub summary: idle 2024-05-27T10:19:16.514 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-27T10:19:14.904784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386679,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2162,"num_read_kb":1474,"num_write":568,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":918,"ondisk_log_size":918,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360316,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581508,"statfs":{"total":1151940427776,"available":1151571464192,"internally_reserved":0,"allocated":11390976,"data_stored":9396861,"data_compressed":246645,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":277,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005362"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866630+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:18:42.866630+0000","last_peered":"2024-05-27T10:18:42.866630+0000","last_clean":"2024-05-27T10:18:42.866630+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:18:42.866630+0000","last_undegraded":"2024-05-27T10:18:42.866630+0000","last_fullsized":"2024-05-27T10:18:42.866630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035777+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:18:43.035777+0000","last_peered":"2024-05-27T10:18:43.035777+0000","last_clean":"2024-05-27T10:18:43.035777+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:18:43.035777+0000","last_undegraded":"2024-05-27T10:18:43.035777+0000","last_fullsized":"2024-05-27T10:18:43.035777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036368+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:18:43.036368+0000","last_peered":"2024-05-27T10:18:43.036368+0000","last_clean":"2024-05-27T10:18:43.036368+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:18:43.036368+0000","last_undegraded":"2024-05-27T10:18:43.036368+0000","last_fullsized":"2024-05-27T10:18:43.036368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011005+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:18:43.011005+0000","last_peered":"2024-05-27T10:18:43.011005+0000","last_clean":"2024-05-27T10:18:43.011005+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:18:43.011005+0000","last_undegraded":"2024-05-27T10:18:43.011005+0000","last_fullsized":"2024-05-27T10:18:43.011005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149389+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:18:43.149389+0000","last_peered":"2024-05-27T10:18:43.149389+0000","last_clean":"2024-05-27T10:18:43.149389+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:18:43.149389+0000","last_undegraded":"2024-05-27T10:18:43.149389+0000","last_fullsized":"2024-05-27T10:18:43.149389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864557+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:18:42.864557+0000","last_peered":"2024-05-27T10:18:42.864557+0000","last_clean":"2024-05-27T10:18:42.864557+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:18:42.864557+0000","last_undegraded":"2024-05-27T10:18:42.864557+0000","last_fullsized":"2024-05-27T10:18:42.864557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914726+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:18:42.914726+0000","last_peered":"2024-05-27T10:18:42.914726+0000","last_clean":"2024-05-27T10:18:42.914726+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:18:42.914726+0000","last_undegraded":"2024-05-27T10:18:42.914726+0000","last_fullsized":"2024-05-27T10:18:42.914726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013203+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:18:43.013203+0000","last_peered":"2024-05-27T10:18:43.013203+0000","last_clean":"2024-05-27T10:18:43.013203+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:18:43.013203+0000","last_undegraded":"2024-05-27T10:18:43.013203+0000","last_fullsized":"2024-05-27T10:18:43.013203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913626+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:18:42.913626+0000","last_peered":"2024-05-27T10:18:42.913626+0000","last_clean":"2024-05-27T10:18:42.913626+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:18:42.913626+0000","last_undegraded":"2024-05-27T10:18:42.913626+0000","last_fullsized":"2024-05-27T10:18:42.913626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865113+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:18:42.865113+0000","last_peered":"2024-05-27T10:18:42.865113+0000","last_clean":"2024-05-27T10:18:42.865113+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:18:42.865113+0000","last_undegraded":"2024-05-27T10:18:42.865113+0000","last_fullsized":"2024-05-27T10:18:42.865113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915691+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:18:42.915691+0000","last_peered":"2024-05-27T10:18:42.915691+0000","last_clean":"2024-05-27T10:18:42.915691+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:18:42.915691+0000","last_undegraded":"2024-05-27T10:18:42.915691+0000","last_fullsized":"2024-05-27T10:18:42.915691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036299+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:18:43.036299+0000","last_peered":"2024-05-27T10:18:43.036299+0000","last_clean":"2024-05-27T10:18:43.036299+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:18:43.036299+0000","last_undegraded":"2024-05-27T10:18:43.036299+0000","last_fullsized":"2024-05-27T10:18:43.036299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858502+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:18:42.858502+0000","last_peered":"2024-05-27T10:18:42.858502+0000","last_clean":"2024-05-27T10:18:42.858502+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:18:42.858502+0000","last_undegraded":"2024-05-27T10:18:42.858502+0000","last_fullsized":"2024-05-27T10:18:42.858502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036021+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:18:43.036021+0000","last_peered":"2024-05-27T10:18:43.036021+0000","last_clean":"2024-05-27T10:18:43.036021+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:18:43.036021+0000","last_undegraded":"2024-05-27T10:18:43.036021+0000","last_fullsized":"2024-05-27T10:18:43.036021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035733+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:18:43.035733+0000","last_peered":"2024-05-27T10:18:43.035733+0000","last_clean":"2024-05-27T10:18:43.035733+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:18:43.035733+0000","last_undegraded":"2024-05-27T10:18:43.035733+0000","last_fullsized":"2024-05-27T10:18:43.035733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.911945+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:18:42.911945+0000","last_peered":"2024-05-27T10:18:42.911945+0000","last_clean":"2024-05-27T10:18:42.911945+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:18:42.911945+0000","last_undegraded":"2024-05-27T10:18:42.911945+0000","last_fullsized":"2024-05-27T10:18:42.911945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.867203+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:18:42.867203+0000","last_peered":"2024-05-27T10:18:42.867203+0000","last_clean":"2024-05-27T10:18:42.867203+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:18:42.867203+0000","last_undegraded":"2024-05-27T10:18:42.867203+0000","last_fullsized":"2024-05-27T10:18:42.867203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011126+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:18:43.011126+0000","last_peered":"2024-05-27T10:18:43.011126+0000","last_clean":"2024-05-27T10:18:43.011126+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:18:43.011126+0000","last_undegraded":"2024-05-27T10:18:43.011126+0000","last_fullsized":"2024-05-27T10:18:43.011126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892271+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:18:42.892271+0000","last_peered":"2024-05-27T10:18:42.892271+0000","last_clean":"2024-05-27T10:18:42.892271+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:18:42.892271+0000","last_undegraded":"2024-05-27T10:18:42.892271+0000","last_fullsized":"2024-05-27T10:18:42.892271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011023+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:18:43.011023+0000","last_peered":"2024-05-27T10:18:43.011023+0000","last_clean":"2024-05-27T10:18:43.011023+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:18:43.011023+0000","last_undegraded":"2024-05-27T10:18:43.011023+0000","last_fullsized":"2024-05-27T10:18:43.011023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013844+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:18:43.013844+0000","last_peered":"2024-05-27T10:18:43.013844+0000","last_clean":"2024-05-27T10:18:43.013844+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:18:43.013844+0000","last_undegraded":"2024-05-27T10:18:43.013844+0000","last_fullsized":"2024-05-27T10:18:43.013844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013898+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:18:43.013898+0000","last_peered":"2024-05-27T10:18:43.013898+0000","last_clean":"2024-05-27T10:18:43.013898+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:18:43.013898+0000","last_undegraded":"2024-05-27T10:18:43.013898+0000","last_fullsized":"2024-05-27T10:18:43.013898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036523+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:18:43.036523+0000","last_peered":"2024-05-27T10:18:43.036523+0000","last_clean":"2024-05-27T10:18:43.036523+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:18:43.036523+0000","last_undegraded":"2024-05-27T10:18:43.036523+0000","last_fullsized":"2024-05-27T10:18:43.036523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150382+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:18:43.150382+0000","last_peered":"2024-05-27T10:18:43.150382+0000","last_clean":"2024-05-27T10:18:43.150382+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:18:43.150382+0000","last_undegraded":"2024-05-27T10:18:43.150382+0000","last_fullsized":"2024-05-27T10:18:43.150382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912126+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:18:42.912126+0000","last_peered":"2024-05-27T10:18:42.912126+0000","last_clean":"2024-05-27T10:18:42.912126+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:18:42.912126+0000","last_undegraded":"2024-05-27T10:18:42.912126+0000","last_fullsized":"2024-05-27T10:18:42.912126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150270+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:18:43.150270+0000","last_peered":"2024-05-27T10:18:43.150270+0000","last_clean":"2024-05-27T10:18:43.150270+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:18:43.150270+0000","last_undegraded":"2024-05-27T10:18:43.150270+0000","last_fullsized":"2024-05-27T10:18:43.150270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029108+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:18:43.029108+0000","last_peered":"2024-05-27T10:18:43.029108+0000","last_clean":"2024-05-27T10:18:43.029108+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:18:43.029108+0000","last_undegraded":"2024-05-27T10:18:43.029108+0000","last_fullsized":"2024-05-27T10:18:43.029108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'11","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:13.527006+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:19:13.527006+0000","last_peered":"2024-05-27T10:19:13.527006+0000","last_clean":"2024-05-27T10:19:13.527006+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:19:13.527006+0000","last_undegraded":"2024-05-27T10:19:13.527006+0000","last_fullsized":"2024-05-27T10:19:13.527006+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":278,"num_read_kb":7,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149192+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:18:43.149192+0000","last_peered":"2024-05-27T10:18:43.149192+0000","last_clean":"2024-05-27T10:18:43.149192+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:18:43.149192+0000","last_undegraded":"2024-05-27T10:18:43.149192+0000","last_fullsized":"2024-05-27T10:18:43.149192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864166+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:18:42.864166+0000","last_peered":"2024-05-27T10:18:42.864166+0000","last_clean":"2024-05-27T10:18:42.864166+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:18:42.864166+0000","last_undegraded":"2024-05-27T10:18:42.864166+0000","last_fullsized":"2024-05-27T10:18:42.864166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028881+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:18:43.028881+0000","last_peered":"2024-05-27T10:18:43.028881+0000","last_clean":"2024-05-27T10:18:43.028881+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:18:43.028881+0000","last_undegraded":"2024-05-27T10:18:43.028881+0000","last_fullsized":"2024-05-27T10:18:43.028881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857824+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:18:42.857824+0000","last_peered":"2024-05-27T10:18:42.857824+0000","last_clean":"2024-05-27T10:18:42.857824+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:18:42.857824+0000","last_undegraded":"2024-05-27T10:18:42.857824+0000","last_fullsized":"2024-05-27T10:18:42.857824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013087+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:18:43.013087+0000","last_peered":"2024-05-27T10:18:43.013087+0000","last_clean":"2024-05-27T10:18:43.013087+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:18:43.013087+0000","last_undegraded":"2024-05-27T10:18:43.013087+0000","last_fullsized":"2024-05-27T10:18:43.013087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.027835+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:18:43.027835+0000","last_peered":"2024-05-27T10:18:43.027835+0000","last_clean":"2024-05-27T10:18:43.027835+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:18:43.027835+0000","last_undegraded":"2024-05-27T10:18:43.027835+0000","last_fullsized":"2024-05-27T10:18:43.027835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010885+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:18:43.010885+0000","last_peered":"2024-05-27T10:18:43.010885+0000","last_clean":"2024-05-27T10:18:43.010885+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:18:43.010885+0000","last_undegraded":"2024-05-27T10:18:43.010885+0000","last_fullsized":"2024-05-27T10:18:43.010885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":307,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:12.522270+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:19:12.522270+0000","last_peered":"2024-05-27T10:19:12.522270+0000","last_clean":"2024-05-27T10:19:12.522270+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:19:12.522270+0000","last_undegraded":"2024-05-27T10:19:12.522270+0000","last_fullsized":"2024-05-27T10:19:12.522270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":272,"num_read_kb":272,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913526+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:18:42.913526+0000","last_peered":"2024-05-27T10:18:42.913526+0000","last_clean":"2024-05-27T10:18:42.913526+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:18:42.913526+0000","last_undegraded":"2024-05-27T10:18:42.913526+0000","last_fullsized":"2024-05-27T10:18:42.913526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915807+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:18:42.915807+0000","last_peered":"2024-05-27T10:18:42.915807+0000","last_clean":"2024-05-27T10:18:42.915807+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:18:42.915807+0000","last_undegraded":"2024-05-27T10:18:42.915807+0000","last_fullsized":"2024-05-27T10:18:42.915807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036613+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:18:43.036613+0000","last_peered":"2024-05-27T10:18:43.036613+0000","last_clean":"2024-05-27T10:18:43.036613+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:18:43.036613+0000","last_undegraded":"2024-05-27T10:18:43.036613+0000","last_fullsized":"2024-05-27T10:18:43.036613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029503+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:18:43.029503+0000","last_peered":"2024-05-27T10:18:43.029503+0000","last_clean":"2024-05-27T10:18:43.029503+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:18:43.029503+0000","last_undegraded":"2024-05-27T10:18:43.029503+0000","last_fullsized":"2024-05-27T10:18:43.029503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912674+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:18:42.912674+0000","last_peered":"2024-05-27T10:18:42.912674+0000","last_clean":"2024-05-27T10:18:42.912674+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:18:42.912674+0000","last_undegraded":"2024-05-27T10:18:42.912674+0000","last_fullsized":"2024-05-27T10:18:42.912674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":625,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:13.469599+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:19:13.469599+0000","last_peered":"2024-05-27T10:19:13.469599+0000","last_clean":"2024-05-27T10:19:13.469599+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:19:13.469599+0000","last_undegraded":"2024-05-27T10:19:13.469599+0000","last_fullsized":"2024-05-27T10:19:13.469599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":345,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150023+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:18:43.150023+0000","last_peered":"2024-05-27T10:18:43.150023+0000","last_clean":"2024-05-27T10:18:43.150023+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:18:43.150023+0000","last_undegraded":"2024-05-27T10:18:43.150023+0000","last_fullsized":"2024-05-27T10:18:43.150023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892407+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:18:42.892407+0000","last_peered":"2024-05-27T10:18:42.892407+0000","last_clean":"2024-05-27T10:18:42.892407+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:18:42.892407+0000","last_undegraded":"2024-05-27T10:18:42.892407+0000","last_fullsized":"2024-05-27T10:18:42.892407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858066+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:18:42.858066+0000","last_peered":"2024-05-27T10:18:42.858066+0000","last_clean":"2024-05-27T10:18:42.858066+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:18:42.858066+0000","last_undegraded":"2024-05-27T10:18:42.858066+0000","last_fullsized":"2024-05-27T10:18:42.858066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028359+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:18:43.028359+0000","last_peered":"2024-05-27T10:18:43.028359+0000","last_clean":"2024-05-27T10:18:43.028359+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:18:43.028359+0000","last_undegraded":"2024-05-27T10:18:43.028359+0000","last_fullsized":"2024-05-27T10:18:43.028359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915378+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:18:42.915378+0000","last_peered":"2024-05-27T10:18:42.915378+0000","last_clean":"2024-05-27T10:18:42.915378+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:18:42.915378+0000","last_undegraded":"2024-05-27T10:18:42.915378+0000","last_fullsized":"2024-05-27T10:18:42.915378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":315,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:12.506010+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:19:12.506010+0000","last_peered":"2024-05-27T10:19:12.506010+0000","last_clean":"2024-05-27T10:19:12.506010+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:19:12.506010+0000","last_undegraded":"2024-05-27T10:19:12.506010+0000","last_fullsized":"2024-05-27T10:19:12.506010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":279,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914514+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:18:42.914514+0000","last_peered":"2024-05-27T10:18:42.914514+0000","last_clean":"2024-05-27T10:18:42.914514+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:18:42.914514+0000","last_undegraded":"2024-05-27T10:18:42.914514+0000","last_fullsized":"2024-05-27T10:18:42.914514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148905+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:18:43.148905+0000","last_peered":"2024-05-27T10:18:43.148905+0000","last_clean":"2024-05-27T10:18:43.148905+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:18:43.148905+0000","last_undegraded":"2024-05-27T10:18:43.148905+0000","last_fullsized":"2024-05-27T10:18:43.148905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028385+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:18:43.028385+0000","last_peered":"2024-05-27T10:18:43.028385+0000","last_clean":"2024-05-27T10:18:43.028385+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:18:43.028385+0000","last_undegraded":"2024-05-27T10:18:43.028385+0000","last_fullsized":"2024-05-27T10:18:43.028385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.641820+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:18:43.641820+0000","last_peered":"2024-05-27T10:18:43.641820+0000","last_clean":"2024-05-27T10:18:43.641820+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:18:43.641820+0000","last_undegraded":"2024-05-27T10:18:43.641820+0000","last_fullsized":"2024-05-27T10:18:43.641820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865929+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:18:42.865929+0000","last_peered":"2024-05-27T10:18:42.865929+0000","last_clean":"2024-05-27T10:18:42.865929+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:18:42.865929+0000","last_undegraded":"2024-05-27T10:18:42.865929+0000","last_fullsized":"2024-05-27T10:18:42.865929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010814+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:18:43.010814+0000","last_peered":"2024-05-27T10:18:43.010814+0000","last_clean":"2024-05-27T10:18:43.010814+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:18:43.010814+0000","last_undegraded":"2024-05-27T10:18:43.010814+0000","last_fullsized":"2024-05-27T10:18:43.010814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864668+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:18:42.864668+0000","last_peered":"2024-05-27T10:18:42.864668+0000","last_clean":"2024-05-27T10:18:42.864668+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:18:42.864668+0000","last_undegraded":"2024-05-27T10:18:42.864668+0000","last_fullsized":"2024-05-27T10:18:42.864668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914915+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:18:42.914915+0000","last_peered":"2024-05-27T10:18:42.914915+0000","last_clean":"2024-05-27T10:18:42.914915+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:18:42.914915+0000","last_undegraded":"2024-05-27T10:18:42.914915+0000","last_fullsized":"2024-05-27T10:18:42.914915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865657+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:18:42.865657+0000","last_peered":"2024-05-27T10:18:42.865657+0000","last_clean":"2024-05-27T10:18:42.865657+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:18:42.865657+0000","last_undegraded":"2024-05-27T10:18:42.865657+0000","last_fullsized":"2024-05-27T10:18:42.865657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028265+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:18:43.028265+0000","last_peered":"2024-05-27T10:18:43.028265+0000","last_clean":"2024-05-27T10:18:43.028265+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:18:43.028265+0000","last_undegraded":"2024-05-27T10:18:43.028265+0000","last_fullsized":"2024-05-27T10:18:43.028265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:12.504424+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:19:12.504424+0000","last_peered":"2024-05-27T10:19:12.504424+0000","last_clean":"2024-05-27T10:19:12.504424+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:19:12.504424+0000","last_undegraded":"2024-05-27T10:19:12.504424+0000","last_fullsized":"2024-05-27T10:19:12.504424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":272,"num_read_kb":272,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013516+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:18:43.013516+0000","last_peered":"2024-05-27T10:18:43.013516+0000","last_clean":"2024-05-27T10:18:43.013516+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:18:43.013516+0000","last_undegraded":"2024-05-27T10:18:43.013516+0000","last_fullsized":"2024-05-27T10:18:43.013516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036179+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:18:43.036179+0000","last_peered":"2024-05-27T10:18:43.036179+0000","last_clean":"2024-05-27T10:18:43.036179+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:18:43.036179+0000","last_undegraded":"2024-05-27T10:18:43.036179+0000","last_fullsized":"2024-05-27T10:18:43.036179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"82'99","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:45.052019+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:18:45.052019+0000","last_peered":"2024-05-27T10:18:45.052019+0000","last_clean":"2024-05-27T10:18:45.052019+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:18:45.052019+0000","last_undegraded":"2024-05-27T10:18:45.052019+0000","last_fullsized":"2024-05-27T10:18:45.052019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857424+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:18:42.857424+0000","last_peered":"2024-05-27T10:18:42.857424+0000","last_clean":"2024-05-27T10:18:42.857424+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:18:42.857424+0000","last_undegraded":"2024-05-27T10:18:42.857424+0000","last_fullsized":"2024-05-27T10:18:42.857424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914885+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:18:42.914885+0000","last_peered":"2024-05-27T10:18:42.914885+0000","last_clean":"2024-05-27T10:18:42.914885+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:18:42.914885+0000","last_undegraded":"2024-05-27T10:18:42.914885+0000","last_fullsized":"2024-05-27T10:18:42.914885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149983+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:18:43.149983+0000","last_peered":"2024-05-27T10:18:43.149983+0000","last_clean":"2024-05-27T10:18:43.149983+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:18:43.149983+0000","last_undegraded":"2024-05-27T10:18:43.149983+0000","last_fullsized":"2024-05-27T10:18:43.149983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858159+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:18:42.858159+0000","last_peered":"2024-05-27T10:18:42.858159+0000","last_clean":"2024-05-27T10:18:42.858159+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:18:42.858159+0000","last_undegraded":"2024-05-27T10:18:42.858159+0000","last_fullsized":"2024-05-27T10:18:42.858159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'102","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:05.435234+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:19:05.435234+0000","last_peered":"2024-05-27T10:19:05.435234+0000","last_clean":"2024-05-27T10:19:05.435234+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:19:05.435234+0000","last_undegraded":"2024-05-27T10:19:05.435234+0000","last_fullsized":"2024-05-27T10:19:05.435234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.014095+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:18:43.014095+0000","last_peered":"2024-05-27T10:18:43.014095+0000","last_clean":"2024-05-27T10:18:43.014095+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:18:43.014095+0000","last_undegraded":"2024-05-27T10:18:43.014095+0000","last_fullsized":"2024-05-27T10:18:43.014095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036143+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:18:43.036143+0000","last_peered":"2024-05-27T10:18:43.036143+0000","last_clean":"2024-05-27T10:18:43.036143+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:18:43.036143+0000","last_undegraded":"2024-05-27T10:18:43.036143+0000","last_fullsized":"2024-05-27T10:18:43.036143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035987+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:18:43.035987+0000","last_peered":"2024-05-27T10:18:43.035987+0000","last_clean":"2024-05-27T10:18:43.035987+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:18:43.035987+0000","last_undegraded":"2024-05-27T10:18:43.035987+0000","last_fullsized":"2024-05-27T10:18:43.035987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910322+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:18:42.910322+0000","last_peered":"2024-05-27T10:18:42.910322+0000","last_clean":"2024-05-27T10:18:42.910322+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:18:42.910322+0000","last_undegraded":"2024-05-27T10:18:42.910322+0000","last_fullsized":"2024-05-27T10:18:42.910322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148975+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:18:43.148975+0000","last_peered":"2024-05-27T10:18:43.148975+0000","last_clean":"2024-05-27T10:18:43.148975+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:18:43.148975+0000","last_undegraded":"2024-05-27T10:18:43.148975+0000","last_fullsized":"2024-05-27T10:18:43.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013173+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:18:43.013173+0000","last_peered":"2024-05-27T10:18:43.013173+0000","last_clean":"2024-05-27T10:18:43.013173+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:18:43.013173+0000","last_undegraded":"2024-05-27T10:18:43.013173+0000","last_fullsized":"2024-05-27T10:18:43.013173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865624+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:18:42.865624+0000","last_peered":"2024-05-27T10:18:42.865624+0000","last_clean":"2024-05-27T10:18:42.865624+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:18:42.865624+0000","last_undegraded":"2024-05-27T10:18:42.865624+0000","last_fullsized":"2024-05-27T10:18:42.865624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865650+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:18:42.865650+0000","last_peered":"2024-05-27T10:18:42.865650+0000","last_clean":"2024-05-27T10:18:42.865650+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:18:42.865650+0000","last_undegraded":"2024-05-27T10:18:42.865650+0000","last_fullsized":"2024-05-27T10:18:42.865650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013727+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:18:43.013727+0000","last_peered":"2024-05-27T10:18:43.013727+0000","last_clean":"2024-05-27T10:18:43.013727+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:18:43.013727+0000","last_undegraded":"2024-05-27T10:18:43.013727+0000","last_fullsized":"2024-05-27T10:18:43.013727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150814+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:18:43.150814+0000","last_peered":"2024-05-27T10:18:43.150814+0000","last_clean":"2024-05-27T10:18:43.150814+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:18:43.150814+0000","last_undegraded":"2024-05-27T10:18:43.150814+0000","last_fullsized":"2024-05-27T10:18:43.150814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915235+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:18:42.915235+0000","last_peered":"2024-05-27T10:18:42.915235+0000","last_clean":"2024-05-27T10:18:42.915235+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:18:42.915235+0000","last_undegraded":"2024-05-27T10:18:42.915235+0000","last_fullsized":"2024-05-27T10:18:42.915235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892674+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:18:42.892674+0000","last_peered":"2024-05-27T10:18:42.892674+0000","last_clean":"2024-05-27T10:18:42.892674+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:18:42.892674+0000","last_undegraded":"2024-05-27T10:18:42.892674+0000","last_fullsized":"2024-05-27T10:18:42.892674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916054+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:18:42.916054+0000","last_peered":"2024-05-27T10:18:42.916054+0000","last_clean":"2024-05-27T10:18:42.916054+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:18:42.916054+0000","last_undegraded":"2024-05-27T10:18:42.916054+0000","last_fullsized":"2024-05-27T10:18:42.916054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'65","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:04.813909+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:19:04.813909+0000","last_peered":"2024-05-27T10:19:04.813909+0000","last_clean":"2024-05-27T10:19:04.813909+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:19:04.813909+0000","last_undegraded":"2024-05-27T10:19:04.813909+0000","last_fullsized":"2024-05-27T10:19:04.813909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915511+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:18:42.915511+0000","last_peered":"2024-05-27T10:18:42.915511+0000","last_clean":"2024-05-27T10:18:42.915511+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:18:42.915511+0000","last_undegraded":"2024-05-27T10:18:42.915511+0000","last_fullsized":"2024-05-27T10:18:42.915511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858229+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:18:42.858229+0000","last_peered":"2024-05-27T10:18:42.858229+0000","last_clean":"2024-05-27T10:18:42.858229+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:18:42.858229+0000","last_undegraded":"2024-05-27T10:18:42.858229+0000","last_fullsized":"2024-05-27T10:18:42.858229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916095+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:18:42.916095+0000","last_peered":"2024-05-27T10:18:42.916095+0000","last_clean":"2024-05-27T10:18:42.916095+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:18:42.916095+0000","last_undegraded":"2024-05-27T10:18:42.916095+0000","last_fullsized":"2024-05-27T10:18:42.916095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865960+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:18:42.865960+0000","last_peered":"2024-05-27T10:18:42.865960+0000","last_clean":"2024-05-27T10:18:42.865960+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:18:42.865960+0000","last_undegraded":"2024-05-27T10:18:42.865960+0000","last_fullsized":"2024-05-27T10:18:42.865960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892084+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:18:42.892084+0000","last_peered":"2024-05-27T10:18:42.892084+0000","last_clean":"2024-05-27T10:18:42.892084+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:18:42.892084+0000","last_undegraded":"2024-05-27T10:18:42.892084+0000","last_fullsized":"2024-05-27T10:18:42.892084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.891669+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:18:42.891669+0000","last_peered":"2024-05-27T10:18:42.891669+0000","last_clean":"2024-05-27T10:18:42.891669+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:18:42.891669+0000","last_undegraded":"2024-05-27T10:18:42.891669+0000","last_fullsized":"2024-05-27T10:18:42.891669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036165+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:18:43.036165+0000","last_peered":"2024-05-27T10:18:43.036165+0000","last_clean":"2024-05-27T10:18:43.036165+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:18:43.036165+0000","last_undegraded":"2024-05-27T10:18:43.036165+0000","last_fullsized":"2024-05-27T10:18:43.036165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858813+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:18:42.858813+0000","last_peered":"2024-05-27T10:18:42.858813+0000","last_clean":"2024-05-27T10:18:42.858813+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:18:42.858813+0000","last_undegraded":"2024-05-27T10:18:42.858813+0000","last_fullsized":"2024-05-27T10:18:42.858813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865913+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:18:42.865913+0000","last_peered":"2024-05-27T10:18:42.865913+0000","last_clean":"2024-05-27T10:18:42.865913+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:18:42.865913+0000","last_undegraded":"2024-05-27T10:18:42.865913+0000","last_fullsized":"2024-05-27T10:18:42.865913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892758+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:18:42.892758+0000","last_peered":"2024-05-27T10:18:42.892758+0000","last_clean":"2024-05-27T10:18:42.892758+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:18:42.892758+0000","last_undegraded":"2024-05-27T10:18:42.892758+0000","last_fullsized":"2024-05-27T10:18:42.892758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010964+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:18:43.010964+0000","last_peered":"2024-05-27T10:18:43.010964+0000","last_clean":"2024-05-27T10:18:43.010964+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:18:43.010964+0000","last_undegraded":"2024-05-27T10:18:43.010964+0000","last_fullsized":"2024-05-27T10:18:43.010964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028084+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:18:43.028084+0000","last_peered":"2024-05-27T10:18:43.028084+0000","last_clean":"2024-05-27T10:18:43.028084+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:18:43.028084+0000","last_undegraded":"2024-05-27T10:18:43.028084+0000","last_fullsized":"2024-05-27T10:18:43.028084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013207+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:18:43.013207+0000","last_peered":"2024-05-27T10:18:43.013207+0000","last_clean":"2024-05-27T10:18:43.013207+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:18:43.013207+0000","last_undegraded":"2024-05-27T10:18:43.013207+0000","last_fullsized":"2024-05-27T10:18:43.013207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913669+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:18:42.913669+0000","last_peered":"2024-05-27T10:18:42.913669+0000","last_clean":"2024-05-27T10:18:42.913669+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:18:42.913669+0000","last_undegraded":"2024-05-27T10:18:42.913669+0000","last_fullsized":"2024-05-27T10:18:42.913669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858143+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:18:42.858143+0000","last_peered":"2024-05-27T10:18:42.858143+0000","last_clean":"2024-05-27T10:18:42.858143+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:18:42.858143+0000","last_undegraded":"2024-05-27T10:18:42.858143+0000","last_fullsized":"2024-05-27T10:18:42.858143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866758+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:18:42.866758+0000","last_peered":"2024-05-27T10:18:42.866758+0000","last_clean":"2024-05-27T10:18:42.866758+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:18:42.866758+0000","last_undegraded":"2024-05-27T10:18:42.866758+0000","last_fullsized":"2024-05-27T10:18:42.866758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796077,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2018,"num_read_kb":1226,"num_write":431,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2388231,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678095,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":830200,"data_compressed":22182,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874326,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1425518,"data_compressed":54247,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":830436,"data_compressed":22182,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234086,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":437140,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430316,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1032293,"data_compressed":39045,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593844,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":888832,"data_stored":436609,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":503807,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738528,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":437880,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411476,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":507355,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149835,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1158368,"data_compressed":37384,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953605,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":438462,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790076,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1358793,"data_compressed":54247,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331906,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":398631,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:19:16.519 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T10:19:16.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T10:19:16.990 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:17.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:16 smithi042 ceph-mon[31661]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-27T10:19:17.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:16 smithi042 ceph-mon[31661]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:17.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:16 smithi104 ceph-mon[34812]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-27T10:19:17.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:16 smithi104 ceph-mon[34812]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:17.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:16 smithi143 ceph-mon[35545]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-27T10:19:17.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:16 smithi143 ceph-mon[35545]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 -- 172.21.15.42:0/1188978603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410c170 msgr2=0x7efeb4105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 --2- 172.21.15.42:0/1188978603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410c170 0x7efeb4105db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efea0009900 tx=0x7efea002a410 comp rx=0 tx=0).stop 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 -- 172.21.15.42:0/1188978603 shutdown_connections 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 --2- 172.21.15.42:0/1188978603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4106420 0x7efeb41127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 --2- 172.21.15.42:0/1188978603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410c170 0x7efeb4105db0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 --2- 172.21.15.42:0/1188978603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efeb410bcd0 0x7efeb4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.931+0000 7efeb3200640 1 -- 172.21.15.42:0/1188978603 >> 172.21.15.42:0/1188978603 conn(0x7efeb41013d0 msgr2=0x7efeb4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.932+0000 7efeb3200640 1 -- 172.21.15.42:0/1188978603 shutdown_connections 2024-05-27T10:19:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.932+0000 7efeb3200640 1 -- 172.21.15.42:0/1188978603 wait complete. 2024-05-27T10:19:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.934+0000 7efeb3200640 1 Processor -- start 2024-05-27T10:19:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 -- start start 2024-05-27T10:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4106420 0x7efeb41a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 0x7efeb41a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efeb410c170 0x7efeb419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4115d70 con 0x7efeb410bcd0 2024-05-27T10:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4115bf0 con 0x7efeb410c170 2024-05-27T10:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.935+0000 7efeb3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4115ef0 con 0x7efeb4106420 2024-05-27T10:19:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 0x7efeb41a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 0x7efeb41a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40140/0 (socket says 172.21.15.42:40140) 2024-05-27T10:19:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 -- 172.21.15.42:0/4062190505 learned_addr learned my addr 172.21.15.42:0/4062190505 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1e00640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4106420 0x7efeb41a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb2800640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efeb410c170 0x7efeb419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 -- 172.21.15.42:0/4062190505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4106420 msgr2=0x7efeb41a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4106420 0x7efeb41a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 -- 172.21.15.42:0/4062190505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efeb410c170 msgr2=0x7efeb419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efeb410c170 0x7efeb419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.936+0000 7efeb1400640 1 -- 172.21.15.42:0/4062190505 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb419cda0 con 0x7efeb410bcd0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.937+0000 7efeb1400640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 0x7efeb41a2a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efea0005940 tx=0x7efea0005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.937+0000 7efeaaa00640 1 -- 172.21.15.42:0/4062190505 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea003f030 con 0x7efeb410bcd0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.937+0000 7efeaaa00640 1 -- 172.21.15.42:0/4062190505 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efea00043d0 con 0x7efeb410bcd0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.937+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb419cfd0 con 0x7efeb410bcd0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.937+0000 7efeaaa00640 1 -- 172.21.15.42:0/4062190505 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0035070 con 0x7efeb410bcd0 2024-05-27T10:19:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.937+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb41a9290 con 0x7efeb410bcd0 2024-05-27T10:19:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 Processor -- start 2024-05-27T10:19:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 -- start start 2024-05-27T10:19:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efeaaa00640 1 -- 172.21.15.42:0/4062190505 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7efea0042030 con 0x7efeb410bcd0 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 0x55a91b5e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a91b551b90 con 0x55a91b23e1b0 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a91b541730 con 0x55a91b21d4d0 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.941+0000 7efec0384740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a91b545770 con 0x55a91b2152f0 2024-05-27T10:19:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 0x55a91b5e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 0x55a91b5e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40156/0 (socket says 172.21.15.42:40156) 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 -- 172.21.15.42:0/3412581865 learned_addr learned my addr 172.21.15.42:0/3412581865 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb1400640 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb1e00640 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 -- 172.21.15.42:0/3412581865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 msgr2=0x55a91b5e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 -- 172.21.15.42:0/3412581865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 msgr2=0x55a91b5e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb2800640 1 -- 172.21.15.42:0/3412581865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91b4da8b0 con 0x55a91b23e1b0 2024-05-27T10:19:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.942+0000 7efeb1e00640 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.943+0000 7efeb2800640 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 0x55a91b5e0a60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efeac009fc0 tx=0x7efeac010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75de2310be53e71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.943+0000 7efe9f400640 1 -- 172.21.15.42:0/3412581865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac003b70 con 0x55a91b23e1b0 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.943+0000 7efe9f400640 1 -- 172.21.15.42:0/3412581865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeac003d10 con 0x55a91b23e1b0 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.943+0000 7efe9f400640 1 -- 172.21.15.42:0/3412581865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac011920 con 0x55a91b23e1b0 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.943+0000 7efeaaa00640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efe980b2af0 0x7efe980b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.943+0000 7efeaaa00640 1 -- 172.21.15.42:0/4062190505 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7efea0102b90 con 0x7efeb410bcd0 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efeb1e00640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efe980b2af0 0x7efe980b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 -- 172.21.15.42:0/3412581865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 msgr2=0x55a91b5e0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 0x55a91b5e0a60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efeac009fc0 tx=0x7efeac010860 comp rx=0 tx=0).stop 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 -- 172.21.15.42:0/3412581865 shutdown_connections 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 --2- 172.21.15.42:0/3412581865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b23e1b0 0x55a91b5e0a60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efec0384740 1 -- 172.21.15.42:0/3412581865 >> 172.21.15.42:0/3412581865 conn(0x55a91b232300 msgr2=0x55a91b5dd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.944+0000 7efeb1e00640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efe980b2af0 0x7efe980b5000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55a91b5e1e10 tx=0x7efea4005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.945+0000 7efec0384740 1 -- 172.21.15.42:0/3412581865 shutdown_connections 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.945+0000 7efec0384740 1 -- 172.21.15.42:0/3412581865 wait complete. 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.945+0000 7efec0384740 1 -- start start 2024-05-27T10:19:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.945+0000 7efec0384740 1 Processor -- start 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b23e1b0 0x55a91b5f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a91b227790 con 0x55a91b21d4d0 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a91b4e9bc0 con 0x55a91b23e1b0 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a91b5edb90 con 0x55a91b2152f0 2024-05-27T10:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efec0384740 10 client.0 _ll_register_callbacks cb 0x55a91b4c7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efeb1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.946+0000 7efeb1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40172/0 (socket says 172.21.15.42:40172) 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 -- 172.21.15.42:0/1235683155 learned_addr learned my addr 172.21.15.42:0/1235683155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1e00640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb2800640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b23e1b0 0x55a91b5f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 -- 172.21.15.42:0/1235683155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 msgr2=0x55a91b5eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5eac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 -- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b23e1b0 msgr2=0x55a91b5f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b23e1b0 0x55a91b5f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 -- 172.21.15.42:0/1235683155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91b256210 con 0x55a91b21d4d0 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb2800640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b23e1b0 0x55a91b5f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efeb1400640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5eb190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efea00c6f50 tx=0x7efea0106170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5f4eccc4e2ccaf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efe93e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.947+0000 7efe93e00640 1 -- 172.21.15.42:0/1235683155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0049070 con 0x55a91b21d4d0 2024-05-27T10:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efe93e00640 1 -- 172.21.15.42:0/1235683155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7efea00c9f40 con 0x55a91b21d4d0 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeac003d10 con 0x55a91b21d4d0 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efe93e00640 1 -- 172.21.15.42:0/1235683155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efea00c9630 con 0x55a91b21d4d0 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 10 client.15375 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a91b322620 con 0x55a91b21d4d0 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efe93e00640 1 -- 172.21.15.42:0/1235683155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeac003d10 con 0x55a91b21d4d0 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efe93e00640 1 -- 172.21.15.42:0/1235683155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efea00e4080 con 0x55a91b21d4d0 2024-05-27T10:19:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 10 client.15375 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 20 client.15375 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.948+0000 7efec0384740 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1fcd80 0x55a91b5fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.949+0000 7efec0384740 4 client.15375 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.949+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a91b553ae0 con 0x55a91b1fcd80 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.949+0000 7efeb1e00640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1fcd80 0x55a91b5fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.950+0000 7efeb1e00640 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1fcd80 0x55a91b5fcf60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662436760e57d120 server_cookie=df225fa296e3837d in_seq=0 out_seq=0 2024-05-27T10:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.950+0000 7efe93e00640 10 client.15375 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efe93e00640 1 -- 172.21.15.42:0/1235683155 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a91b553ae0 con 0x55a91b1fcd80 2024-05-27T10:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efe93e00640 10 client.15375 handle_command_reply: tid=0 2024-05-27T10:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efec0384740 1 client.15375 shutdown 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 msgr2=0x55a91b5eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efec0384740 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5eb190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efea00c6f50 tx=0x7efea0106170 comp rx=0 tx=0).stop 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 shutdown_connections 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efec0384740 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b23e1b0 0x55a91b5f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.952+0000 7efec0384740 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5eb190 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efec0384740 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1fcd80 0x55a91b5fcf60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efec0384740 1 --2- 172.21.15.42:0/1235683155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b2152f0 0x55a91b5eac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 >> 172.21.15.42:0/1235683155 conn(0x55a91b232300 msgr2=0x55a91b5e0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efe93e00640 0 client.15375 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 shutdown_connections 2024-05-27T10:19:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efec0384740 1 -- 172.21.15.42:0/1235683155 wait complete. 2024-05-27T10:19:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.953+0000 7efec0384740 20 client.15375 trim_cache size 0 max 16384 2024-05-27T10:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 Processor -- start 2024-05-27T10:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 -- start start 2024-05-27T10:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 0x55a91b5e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 0x55a91b602720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41b4250 con 0x55a91b2152f0 2024-05-27T10:19:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41b1b20 con 0x55a91b21d4d0 2024-05-27T10:19:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.970+0000 7efec0384740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41af970 con 0x55a91b23e1b0 2024-05-27T10:19:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 0x55a91b5e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 0x55a91b5e63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40180/0 (socket says 172.21.15.42:40180) 2024-05-27T10:19:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 -- 172.21.15.42:0/3622211300 learned_addr learned my addr 172.21.15.42:0/3622211300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1400640 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 -- 172.21.15.42:0/3622211300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 msgr2=0x55a91b602720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 0x55a91b602720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 -- 172.21.15.42:0/3622211300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 msgr2=0x55a91b5ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5ecda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 -- 172.21.15.42:0/3622211300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91b5e91a0 con 0x55a91b2152f0 2024-05-27T10:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.971+0000 7efeb1e00640 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 0x55a91b5e63a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efeb41069b0 tx=0x7efea4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e8d0b4d01a4df0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.972+0000 7efe9ea00640 1 -- 172.21.15.42:0/3622211300 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea402a5a0 con 0x55a91b2152f0 2024-05-27T10:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.972+0000 7efe9ea00640 1 -- 172.21.15.42:0/3622211300 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efea402abd0 con 0x55a91b2152f0 2024-05-27T10:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.972+0000 7efe9ea00640 1 -- 172.21.15.42:0/3622211300 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4021b60 con 0x55a91b2152f0 2024-05-27T10:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 -- 172.21.15.42:0/3622211300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 msgr2=0x55a91b5e63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 0x55a91b5e63a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efeb41069b0 tx=0x7efea4015040 comp rx=0 tx=0).stop 2024-05-27T10:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 -- 172.21.15.42:0/3622211300 shutdown_connections 2024-05-27T10:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 0x55a91b602720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b21d4d0 0x55a91b5ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 --2- 172.21.15.42:0/3622211300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b2152f0 0x55a91b5e63a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 -- 172.21.15.42:0/3622211300 >> 172.21.15.42:0/3622211300 conn(0x55a91b232300 msgr2=0x55a91b5ea190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 -- 172.21.15.42:0/3622211300 shutdown_connections 2024-05-27T10:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 -- 172.21.15.42:0/3622211300 wait complete. 2024-05-27T10:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 -- start start 2024-05-27T10:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.973+0000 7efec0384740 1 Processor -- start 2024-05-27T10:19:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b2152f0 0x55a91b5f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 0x55a91b65d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41affe0 con 0x55a91b21d4d0 2024-05-27T10:19:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41b1450 con 0x55a91b2152f0 2024-05-27T10:19:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41b06b0 con 0x55a91b23e1b0 2024-05-27T10:19:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efec0384740 10 client.0 _ll_register_callbacks cb 0x7efea01001e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efeb1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efeb1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40186/0 (socket says 172.21.15.42:40186) 2024-05-27T10:19:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.974+0000 7efeb1400640 1 -- 172.21.15.42:0/1640038629 learned_addr learned my addr 172.21.15.42:0/1640038629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1e00640 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b2152f0 0x55a91b5f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1400640 1 -- 172.21.15.42:0/1640038629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 msgr2=0x55a91b65d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1400640 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 0x55a91b65d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1400640 1 -- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b2152f0 msgr2=0x55a91b5f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1400640 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b2152f0 0x55a91b5f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1400640 1 -- 172.21.15.42:0/1640038629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91b65db00 con 0x55a91b21d4d0 2024-05-27T10:19:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efeb1400640 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5f0940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efea003d040 tx=0x7efea00c9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ccaa44ea4c44f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efe93e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efe93e00640 1 -- 172.21.15.42:0/1640038629 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00e8020 con 0x55a91b21d4d0 2024-05-27T10:19:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efe93e00640 1 -- 172.21.15.42:0/1640038629 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7efea00c62d0 con 0x55a91b21d4d0 2024-05-27T10:19:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea402abd0 con 0x55a91b21d4d0 2024-05-27T10:19:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.975+0000 7efe93e00640 1 -- 172.21.15.42:0/1640038629 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efea0049070 con 0x55a91b21d4d0 2024-05-27T10:19:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 10 client.15381 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a91b664de0 con 0x55a91b21d4d0 2024-05-27T10:19:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efe93e00640 1 -- 172.21.15.42:0/1640038629 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efea402abd0 con 0x55a91b21d4d0 2024-05-27T10:19:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efe93e00640 1 -- 172.21.15.42:0/1640038629 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efea00e4080 con 0x55a91b21d4d0 2024-05-27T10:19:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 10 client.15381 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 20 client.15381 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1d21c0 0x55a91b667410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 4 client.15381 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.976+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a91b54d2c0 con 0x55a91b1d21c0 2024-05-27T10:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.977+0000 7efeb1e00640 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1d21c0 0x55a91b667410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.978+0000 7efeb1e00640 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1d21c0 0x55a91b667410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b49051b60ba42be server_cookie=c619ed960815cb0d in_seq=0 out_seq=0 2024-05-27T10:19:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:18.978+0000 7efe93e00640 10 client.15381 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efe93e00640 1 -- 172.21.15.42:0/1640038629 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a91b54d2c0 con 0x55a91b1d21c0 2024-05-27T10:19:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efe93e00640 10 client.15381 handle_command_reply: tid=0 2024-05-27T10:19:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 client.15381 shutdown 2024-05-27T10:19:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 msgr2=0x55a91b5f0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5f0940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efea003d040 tx=0x7efea00c9b70 comp rx=0 tx=0).stop 2024-05-27T10:19:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 shutdown_connections 2024-05-27T10:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a91b1d21c0 0x55a91b667410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a91b23e1b0 0x55a91b65d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a91b21d4d0 0x55a91b5f0940 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efe93e00640 0 client.15381 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 --2- 172.21.15.42:0/1640038629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a91b2152f0 0x55a91b5f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.012+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 >> 172.21.15.42:0/1640038629 conn(0x55a91b232300 msgr2=0x55a91b5ec9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.013+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 shutdown_connections 2024-05-27T10:19:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.013+0000 7efec0384740 1 -- 172.21.15.42:0/1640038629 wait complete. 2024-05-27T10:19:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.013+0000 7efec0384740 20 client.15381 trim_cache size 0 max 16384 2024-05-27T10:19:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:19:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:19:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:19:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efe980b2af0 msgr2=0x7efe980b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efe980b2af0 0x7efe980b5000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55a91b5e1e10 tx=0x7efea4005f90 comp rx=0 tx=0).stop 2024-05-27T10:19:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 msgr2=0x7efeb41a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 0x7efeb41a2a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efea0005940 tx=0x7efea0005e10 comp rx=0 tx=0).stop 2024-05-27T10:19:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 shutdown_connections 2024-05-27T10:19:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efe980b2af0 0x7efe980b5000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efeb410c170 0x7efeb419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeb410bcd0 0x7efeb41a2a30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 --2- 172.21.15.42:0/4062190505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4106420 0x7efeb41a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 >> 172.21.15.42:0/4062190505 conn(0x7efeb41013d0 msgr2=0x7efeb41017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 shutdown_connections 2024-05-27T10:19:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:19.014+0000 7efeb3200640 1 -- 172.21.15.42:0/4062190505 wait complete. 2024-05-27T10:19:19.131 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:18 smithi042 ceph-mon[31661]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:19:19.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:18 smithi104 ceph-mon[34812]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:19:19.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:18 smithi143 ceph-mon[35545]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.552+0000 7f938dc00700 1 -- 172.21.15.42:0/1983290264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 msgr2=0x7f9388104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.552+0000 7f938dc00700 1 --2- 172.21.15.42:0/1983290264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f9388104a50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f93780099c0 tx=0x7f9378009cd0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 -- 172.21.15.42:0/1983290264 shutdown_connections 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 --2- 172.21.15.42:0/1983290264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388109870 0x7f93881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 --2- 172.21.15.42:0/1983290264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f9388104a50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 --2- 172.21.15.42:0/1983290264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108f20 0x7f9388109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 -- 172.21.15.42:0/1983290264 >> 172.21.15.42:0/1983290264 conn(0x7f9388079ef0 msgr2=0x7f938807a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 -- 172.21.15.42:0/1983290264 shutdown_connections 2024-05-27T10:19:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.553+0000 7f938dc00700 1 -- 172.21.15.42:0/1983290264 wait complete. 2024-05-27T10:19:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 Processor -- start 2024-05-27T10:19:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 -- start start 2024-05-27T10:19:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f938819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388108f20 0x7f938819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388109870 0x7f938819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9388114350 con 0x7f9388104640 2024-05-27T10:19:19.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93881141d0 con 0x7f9388109870 2024-05-27T10:19:19.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.554+0000 7f938dc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93881144d0 con 0x7f9388108f20 2024-05-27T10:19:19.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f938819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:19.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f938819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40208/0 (socket says 172.21.15.42:40208) 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 -- 172.21.15.42:0/3059695765 learned_addr learned my addr 172.21.15.42:0/3059695765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386000700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388108f20 0x7f938819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9387400700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388109870 0x7f938819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 -- 172.21.15.42:0/3059695765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388108f20 msgr2=0x7f938819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388108f20 0x7f938819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 -- 172.21.15.42:0/3059695765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388109870 msgr2=0x7f938819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.555+0000 7f9386a00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388109870 0x7f938819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f9386a00700 1 -- 172.21.15.42:0/3059695765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938819fc90 con 0x7f9388104640 2024-05-27T10:19:19.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f9387400700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388109870 0x7f938819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:19.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f9386000700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388108f20 0x7f938819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:19.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f9386a00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f938819adc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9378000c00 tx=0x7f937800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:19.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937800f940 con 0x7f9388104640 2024-05-27T10:19:19.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9378004830 con 0x7f9388104640 2024-05-27T10:19:19.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938819ff20 con 0x7f9388104640 2024-05-27T10:19:19.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780054b0 con 0x7f9388104640 2024-05-27T10:19:19.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.556+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a03a0 con 0x7f9388104640 2024-05-27T10:19:19.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.558+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93880512b0 con 0x7f9388104640 2024-05-27T10:19:19.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.560+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f9378029060 con 0x7f9388104640 2024-05-27T10:19:19.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.562+0000 7f937f400700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f936c0b2b10 0x7f936c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:19.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.562+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f93780e3420 con 0x7f9388104640 2024-05-27T10:19:19.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.562+0000 7f9386000700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f936c0b2b10 0x7f936c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:19.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.563+0000 7f9386000700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f936c0b2b10 0x7f936c0b5020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9374005950 tx=0x7f93740058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:19.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.567+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9378013070 con 0x7f9388104640 2024-05-27T10:19:19.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.806+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9388063bf0 con 0x7f936c0b2b10 2024-05-27T10:19:19.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.816+0000 7f937f400700 1 -- 172.21.15.42:0/3059695765 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258016 (secure 0 0 0) 0x7f9388063bf0 con 0x7f936c0b2b10 2024-05-27T10:19:19.820 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.821+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f936c0b2b10 msgr2=0x7f936c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.821+0000 7f938dc00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f936c0b2b10 0x7f936c0b5020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9374005950 tx=0x7f93740058e0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.821+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 msgr2=0x7f938819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.821+0000 7f938dc00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f938819adc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9378000c00 tx=0x7f937800f510 comp rx=0 tx=0).stop 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 shutdown_connections 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f936c0b2b10 0x7f936c0b5020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388109870 0x7f938819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9388104640 0x7f938819adc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 --2- 172.21.15.42:0/3059695765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388108f20 0x7f938819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 >> 172.21.15.42:0/3059695765 conn(0x7f9388079ef0 msgr2=0x7f9388102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 shutdown_connections 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:19.822+0000 7f938dc00700 1 -- 172.21.15.42:0/3059695765 wait complete. 2024-05-27T10:19:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-27T10:19:20.952 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-27T10:19:18.906706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386679,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2162,"num_read_kb":1474,"num_write":568,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":918,"ondisk_log_size":918,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360316,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581508,"statfs":{"total":1151940427776,"available":1151571464192,"internally_reserved":0,"allocated":11390976,"data_stored":9396861,"data_compressed":246645,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":162,"num_write":62,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005815"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866630+0000","last_change":"2024-05-27T10:14:14.839287+0000","last_active":"2024-05-27T10:18:42.866630+0000","last_peered":"2024-05-27T10:18:42.866630+0000","last_clean":"2024-05-27T10:18:42.866630+0000","last_became_active":"2024-05-27T10:14:14.838918+0000","last_became_peered":"2024-05-27T10:14:14.838918+0000","last_unstale":"2024-05-27T10:18:42.866630+0000","last_undegraded":"2024-05-27T10:18:42.866630+0000","last_fullsized":"2024-05-27T10:18:42.866630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:51:20.861675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035777+0000","last_change":"2024-05-27T10:14:14.823907+0000","last_active":"2024-05-27T10:18:43.035777+0000","last_peered":"2024-05-27T10:18:43.035777+0000","last_clean":"2024-05-27T10:18:43.035777+0000","last_became_active":"2024-05-27T10:14:14.823514+0000","last_became_peered":"2024-05-27T10:14:14.823514+0000","last_unstale":"2024-05-27T10:18:43.035777+0000","last_undegraded":"2024-05-27T10:18:43.035777+0000","last_fullsized":"2024-05-27T10:18:43.035777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:27.880827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036368+0000","last_change":"2024-05-27T10:14:14.839650+0000","last_active":"2024-05-27T10:18:43.036368+0000","last_peered":"2024-05-27T10:18:43.036368+0000","last_clean":"2024-05-27T10:18:43.036368+0000","last_became_active":"2024-05-27T10:14:14.839197+0000","last_became_peered":"2024-05-27T10:14:14.839197+0000","last_unstale":"2024-05-27T10:18:43.036368+0000","last_undegraded":"2024-05-27T10:18:43.036368+0000","last_fullsized":"2024-05-27T10:18:43.036368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:13.230278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011005+0000","last_change":"2024-05-27T10:14:41.652377+0000","last_active":"2024-05-27T10:18:43.011005+0000","last_peered":"2024-05-27T10:18:43.011005+0000","last_clean":"2024-05-27T10:18:43.011005+0000","last_became_active":"2024-05-27T10:14:41.651417+0000","last_became_peered":"2024-05-27T10:14:41.651417+0000","last_unstale":"2024-05-27T10:18:43.011005+0000","last_undegraded":"2024-05-27T10:18:43.011005+0000","last_fullsized":"2024-05-27T10:18:43.011005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:58:59.517831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149389+0000","last_change":"2024-05-27T10:14:14.826618+0000","last_active":"2024-05-27T10:18:43.149389+0000","last_peered":"2024-05-27T10:18:43.149389+0000","last_clean":"2024-05-27T10:18:43.149389+0000","last_became_active":"2024-05-27T10:14:14.824744+0000","last_became_peered":"2024-05-27T10:14:14.824744+0000","last_unstale":"2024-05-27T10:18:43.149389+0000","last_undegraded":"2024-05-27T10:18:43.149389+0000","last_fullsized":"2024-05-27T10:18:43.149389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:16:20.951397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864557+0000","last_change":"2024-05-27T10:14:41.651541+0000","last_active":"2024-05-27T10:18:42.864557+0000","last_peered":"2024-05-27T10:18:42.864557+0000","last_clean":"2024-05-27T10:18:42.864557+0000","last_became_active":"2024-05-27T10:14:41.651103+0000","last_became_peered":"2024-05-27T10:14:41.651103+0000","last_unstale":"2024-05-27T10:18:42.864557+0000","last_undegraded":"2024-05-27T10:18:42.864557+0000","last_fullsized":"2024-05-27T10:18:42.864557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:45:35.834138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914726+0000","last_change":"2024-05-27T10:14:14.827610+0000","last_active":"2024-05-27T10:18:42.914726+0000","last_peered":"2024-05-27T10:18:42.914726+0000","last_clean":"2024-05-27T10:18:42.914726+0000","last_became_active":"2024-05-27T10:14:14.824794+0000","last_became_peered":"2024-05-27T10:14:14.824794+0000","last_unstale":"2024-05-27T10:18:42.914726+0000","last_undegraded":"2024-05-27T10:18:42.914726+0000","last_fullsized":"2024-05-27T10:18:42.914726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:37.084757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013203+0000","last_change":"2024-05-27T10:14:14.822857+0000","last_active":"2024-05-27T10:18:43.013203+0000","last_peered":"2024-05-27T10:18:43.013203+0000","last_clean":"2024-05-27T10:18:43.013203+0000","last_became_active":"2024-05-27T10:14:14.820365+0000","last_became_peered":"2024-05-27T10:14:14.820365+0000","last_unstale":"2024-05-27T10:18:43.013203+0000","last_undegraded":"2024-05-27T10:18:43.013203+0000","last_fullsized":"2024-05-27T10:18:43.013203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:19.413074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913626+0000","last_change":"2024-05-27T10:14:14.824904+0000","last_active":"2024-05-27T10:18:42.913626+0000","last_peered":"2024-05-27T10:18:42.913626+0000","last_clean":"2024-05-27T10:18:42.913626+0000","last_became_active":"2024-05-27T10:14:14.820946+0000","last_became_peered":"2024-05-27T10:14:14.820946+0000","last_unstale":"2024-05-27T10:18:42.913626+0000","last_undegraded":"2024-05-27T10:18:42.913626+0000","last_fullsized":"2024-05-27T10:18:42.913626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:52.617468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865113+0000","last_change":"2024-05-27T10:14:14.833109+0000","last_active":"2024-05-27T10:18:42.865113+0000","last_peered":"2024-05-27T10:18:42.865113+0000","last_clean":"2024-05-27T10:18:42.865113+0000","last_became_active":"2024-05-27T10:14:14.831059+0000","last_became_peered":"2024-05-27T10:14:14.831059+0000","last_unstale":"2024-05-27T10:18:42.865113+0000","last_undegraded":"2024-05-27T10:18:42.865113+0000","last_fullsized":"2024-05-27T10:18:42.865113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:58.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915691+0000","last_change":"2024-05-27T10:14:14.837844+0000","last_active":"2024-05-27T10:18:42.915691+0000","last_peered":"2024-05-27T10:18:42.915691+0000","last_clean":"2024-05-27T10:18:42.915691+0000","last_became_active":"2024-05-27T10:14:14.836811+0000","last_became_peered":"2024-05-27T10:14:14.836811+0000","last_unstale":"2024-05-27T10:18:42.915691+0000","last_undegraded":"2024-05-27T10:18:42.915691+0000","last_fullsized":"2024-05-27T10:18:42.915691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:50:47.211097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036299+0000","last_change":"2024-05-27T10:14:14.825359+0000","last_active":"2024-05-27T10:18:43.036299+0000","last_peered":"2024-05-27T10:18:43.036299+0000","last_clean":"2024-05-27T10:18:43.036299+0000","last_became_active":"2024-05-27T10:14:14.824551+0000","last_became_peered":"2024-05-27T10:14:14.824551+0000","last_unstale":"2024-05-27T10:18:43.036299+0000","last_undegraded":"2024-05-27T10:18:43.036299+0000","last_fullsized":"2024-05-27T10:18:43.036299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:44.889780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858502+0000","last_change":"2024-05-27T10:14:14.832332+0000","last_active":"2024-05-27T10:18:42.858502+0000","last_peered":"2024-05-27T10:18:42.858502+0000","last_clean":"2024-05-27T10:18:42.858502+0000","last_became_active":"2024-05-27T10:14:14.821944+0000","last_became_peered":"2024-05-27T10:14:14.821944+0000","last_unstale":"2024-05-27T10:18:42.858502+0000","last_undegraded":"2024-05-27T10:18:42.858502+0000","last_fullsized":"2024-05-27T10:18:42.858502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:02:53.309968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036021+0000","last_change":"2024-05-27T10:14:14.818237+0000","last_active":"2024-05-27T10:18:43.036021+0000","last_peered":"2024-05-27T10:18:43.036021+0000","last_clean":"2024-05-27T10:18:43.036021+0000","last_became_active":"2024-05-27T10:14:14.817887+0000","last_became_peered":"2024-05-27T10:14:14.817887+0000","last_unstale":"2024-05-27T10:18:43.036021+0000","last_undegraded":"2024-05-27T10:18:43.036021+0000","last_fullsized":"2024-05-27T10:18:43.036021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:23:09.453857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035733+0000","last_change":"2024-05-27T10:14:14.829011+0000","last_active":"2024-05-27T10:18:43.035733+0000","last_peered":"2024-05-27T10:18:43.035733+0000","last_clean":"2024-05-27T10:18:43.035733+0000","last_became_active":"2024-05-27T10:14:14.828460+0000","last_became_peered":"2024-05-27T10:14:14.828460+0000","last_unstale":"2024-05-27T10:18:43.035733+0000","last_undegraded":"2024-05-27T10:18:43.035733+0000","last_fullsized":"2024-05-27T10:18:43.035733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:42.971037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.911945+0000","last_change":"2024-05-27T10:14:14.828338+0000","last_active":"2024-05-27T10:18:42.911945+0000","last_peered":"2024-05-27T10:18:42.911945+0000","last_clean":"2024-05-27T10:18:42.911945+0000","last_became_active":"2024-05-27T10:14:14.827995+0000","last_became_peered":"2024-05-27T10:14:14.827995+0000","last_unstale":"2024-05-27T10:18:42.911945+0000","last_undegraded":"2024-05-27T10:18:42.911945+0000","last_fullsized":"2024-05-27T10:18:42.911945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:57:31.195550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.867203+0000","last_change":"2024-05-27T10:14:14.841228+0000","last_active":"2024-05-27T10:18:42.867203+0000","last_peered":"2024-05-27T10:18:42.867203+0000","last_clean":"2024-05-27T10:18:42.867203+0000","last_became_active":"2024-05-27T10:14:14.840501+0000","last_became_peered":"2024-05-27T10:14:14.840501+0000","last_unstale":"2024-05-27T10:18:42.867203+0000","last_undegraded":"2024-05-27T10:18:42.867203+0000","last_fullsized":"2024-05-27T10:18:42.867203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:08.861679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011126+0000","last_change":"2024-05-27T10:14:14.819773+0000","last_active":"2024-05-27T10:18:43.011126+0000","last_peered":"2024-05-27T10:18:43.011126+0000","last_clean":"2024-05-27T10:18:43.011126+0000","last_became_active":"2024-05-27T10:14:14.816703+0000","last_became_peered":"2024-05-27T10:14:14.816703+0000","last_unstale":"2024-05-27T10:18:43.011126+0000","last_undegraded":"2024-05-27T10:18:43.011126+0000","last_fullsized":"2024-05-27T10:18:43.011126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:44.068994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892271+0000","last_change":"2024-05-27T10:14:14.825115+0000","last_active":"2024-05-27T10:18:42.892271+0000","last_peered":"2024-05-27T10:18:42.892271+0000","last_clean":"2024-05-27T10:18:42.892271+0000","last_became_active":"2024-05-27T10:14:14.824702+0000","last_became_peered":"2024-05-27T10:14:14.824702+0000","last_unstale":"2024-05-27T10:18:42.892271+0000","last_undegraded":"2024-05-27T10:18:42.892271+0000","last_fullsized":"2024-05-27T10:18:42.892271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:35.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.011023+0000","last_change":"2024-05-27T10:14:09.763305+0000","last_active":"2024-05-27T10:18:43.011023+0000","last_peered":"2024-05-27T10:18:43.011023+0000","last_clean":"2024-05-27T10:18:43.011023+0000","last_became_active":"2024-05-27T10:14:09.762791+0000","last_became_peered":"2024-05-27T10:14:09.762791+0000","last_unstale":"2024-05-27T10:18:43.011023+0000","last_undegraded":"2024-05-27T10:18:43.011023+0000","last_fullsized":"2024-05-27T10:18:43.011023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:50:13.946980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013844+0000","last_change":"2024-05-27T10:14:14.828107+0000","last_active":"2024-05-27T10:18:43.013844+0000","last_peered":"2024-05-27T10:18:43.013844+0000","last_clean":"2024-05-27T10:18:43.013844+0000","last_became_active":"2024-05-27T10:14:14.827113+0000","last_became_peered":"2024-05-27T10:14:14.827113+0000","last_unstale":"2024-05-27T10:18:43.013844+0000","last_undegraded":"2024-05-27T10:18:43.013844+0000","last_fullsized":"2024-05-27T10:18:43.013844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:33.710845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013898+0000","last_change":"2024-05-27T10:14:09.924639+0000","last_active":"2024-05-27T10:18:43.013898+0000","last_peered":"2024-05-27T10:18:43.013898+0000","last_clean":"2024-05-27T10:18:43.013898+0000","last_became_active":"2024-05-27T10:14:09.923211+0000","last_became_peered":"2024-05-27T10:14:09.923211+0000","last_unstale":"2024-05-27T10:18:43.013898+0000","last_undegraded":"2024-05-27T10:18:43.013898+0000","last_fullsized":"2024-05-27T10:18:43.013898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:09.752150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036523+0000","last_change":"2024-05-27T10:14:14.827782+0000","last_active":"2024-05-27T10:18:43.036523+0000","last_peered":"2024-05-27T10:18:43.036523+0000","last_clean":"2024-05-27T10:18:43.036523+0000","last_became_active":"2024-05-27T10:14:14.825578+0000","last_became_peered":"2024-05-27T10:14:14.825578+0000","last_unstale":"2024-05-27T10:18:43.036523+0000","last_undegraded":"2024-05-27T10:18:43.036523+0000","last_fullsized":"2024-05-27T10:18:43.036523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:03.434405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150382+0000","last_change":"2024-05-27T10:14:09.768054+0000","last_active":"2024-05-27T10:18:43.150382+0000","last_peered":"2024-05-27T10:18:43.150382+0000","last_clean":"2024-05-27T10:18:43.150382+0000","last_became_active":"2024-05-27T10:14:09.767078+0000","last_became_peered":"2024-05-27T10:14:09.767078+0000","last_unstale":"2024-05-27T10:18:43.150382+0000","last_undegraded":"2024-05-27T10:18:43.150382+0000","last_fullsized":"2024-05-27T10:18:43.150382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:04:16.102726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912126+0000","last_change":"2024-05-27T10:14:14.823627+0000","last_active":"2024-05-27T10:18:42.912126+0000","last_peered":"2024-05-27T10:18:42.912126+0000","last_clean":"2024-05-27T10:18:42.912126+0000","last_became_active":"2024-05-27T10:14:14.822414+0000","last_became_peered":"2024-05-27T10:14:14.822414+0000","last_unstale":"2024-05-27T10:18:42.912126+0000","last_undegraded":"2024-05-27T10:18:42.912126+0000","last_fullsized":"2024-05-27T10:18:42.912126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:28:37.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150270+0000","last_change":"2024-05-27T10:14:09.763662+0000","last_active":"2024-05-27T10:18:43.150270+0000","last_peered":"2024-05-27T10:18:43.150270+0000","last_clean":"2024-05-27T10:18:43.150270+0000","last_became_active":"2024-05-27T10:14:09.763199+0000","last_became_peered":"2024-05-27T10:14:09.763199+0000","last_unstale":"2024-05-27T10:18:43.150270+0000","last_undegraded":"2024-05-27T10:18:43.150270+0000","last_fullsized":"2024-05-27T10:18:43.150270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:50.564291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029108+0000","last_change":"2024-05-27T10:14:14.827945+0000","last_active":"2024-05-27T10:18:43.029108+0000","last_peered":"2024-05-27T10:18:43.029108+0000","last_clean":"2024-05-27T10:18:43.029108+0000","last_became_active":"2024-05-27T10:14:14.827544+0000","last_became_peered":"2024-05-27T10:14:14.827544+0000","last_unstale":"2024-05-27T10:18:43.029108+0000","last_undegraded":"2024-05-27T10:18:43.029108+0000","last_fullsized":"2024-05-27T10:18:43.029108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:26.052991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'11","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:13.527006+0000","last_change":"2024-05-27T10:14:09.764720+0000","last_active":"2024-05-27T10:19:13.527006+0000","last_peered":"2024-05-27T10:19:13.527006+0000","last_clean":"2024-05-27T10:19:13.527006+0000","last_became_active":"2024-05-27T10:14:09.763510+0000","last_became_peered":"2024-05-27T10:14:09.763510+0000","last_unstale":"2024-05-27T10:19:13.527006+0000","last_undegraded":"2024-05-27T10:19:13.527006+0000","last_fullsized":"2024-05-27T10:19:13.527006+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:13:11.580154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":278,"num_read_kb":7,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149192+0000","last_change":"2024-05-27T10:14:14.817766+0000","last_active":"2024-05-27T10:18:43.149192+0000","last_peered":"2024-05-27T10:18:43.149192+0000","last_clean":"2024-05-27T10:18:43.149192+0000","last_became_active":"2024-05-27T10:14:14.813197+0000","last_became_peered":"2024-05-27T10:14:14.813197+0000","last_unstale":"2024-05-27T10:18:43.149192+0000","last_undegraded":"2024-05-27T10:18:43.149192+0000","last_fullsized":"2024-05-27T10:18:43.149192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:13.876501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864166+0000","last_change":"2024-05-27T10:14:09.766798+0000","last_active":"2024-05-27T10:18:42.864166+0000","last_peered":"2024-05-27T10:18:42.864166+0000","last_clean":"2024-05-27T10:18:42.864166+0000","last_became_active":"2024-05-27T10:14:09.764587+0000","last_became_peered":"2024-05-27T10:14:09.764587+0000","last_unstale":"2024-05-27T10:18:42.864166+0000","last_undegraded":"2024-05-27T10:18:42.864166+0000","last_fullsized":"2024-05-27T10:18:42.864166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:15.671011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028881+0000","last_change":"2024-05-27T10:14:14.829602+0000","last_active":"2024-05-27T10:18:43.028881+0000","last_peered":"2024-05-27T10:18:43.028881+0000","last_clean":"2024-05-27T10:18:43.028881+0000","last_became_active":"2024-05-27T10:14:14.828694+0000","last_became_peered":"2024-05-27T10:14:14.828694+0000","last_unstale":"2024-05-27T10:18:43.028881+0000","last_undegraded":"2024-05-27T10:18:43.028881+0000","last_fullsized":"2024-05-27T10:18:43.028881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857824+0000","last_change":"2024-05-27T10:14:09.763411+0000","last_active":"2024-05-27T10:18:42.857824+0000","last_peered":"2024-05-27T10:18:42.857824+0000","last_clean":"2024-05-27T10:18:42.857824+0000","last_became_active":"2024-05-27T10:14:09.761440+0000","last_became_peered":"2024-05-27T10:14:09.761440+0000","last_unstale":"2024-05-27T10:18:42.857824+0000","last_undegraded":"2024-05-27T10:18:42.857824+0000","last_fullsized":"2024-05-27T10:18:42.857824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:30:58.940755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013087+0000","last_change":"2024-05-27T10:14:14.819826+0000","last_active":"2024-05-27T10:18:43.013087+0000","last_peered":"2024-05-27T10:18:43.013087+0000","last_clean":"2024-05-27T10:18:43.013087+0000","last_became_active":"2024-05-27T10:14:14.819429+0000","last_became_peered":"2024-05-27T10:14:14.819429+0000","last_unstale":"2024-05-27T10:18:43.013087+0000","last_undegraded":"2024-05-27T10:18:43.013087+0000","last_fullsized":"2024-05-27T10:18:43.013087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:44.490378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.027835+0000","last_change":"2024-05-27T10:14:09.763213+0000","last_active":"2024-05-27T10:18:43.027835+0000","last_peered":"2024-05-27T10:18:43.027835+0000","last_clean":"2024-05-27T10:18:43.027835+0000","last_became_active":"2024-05-27T10:14:09.762001+0000","last_became_peered":"2024-05-27T10:14:09.762001+0000","last_unstale":"2024-05-27T10:18:43.027835+0000","last_undegraded":"2024-05-27T10:18:43.027835+0000","last_fullsized":"2024-05-27T10:18:43.027835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:59:39.728739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010885+0000","last_change":"2024-05-27T10:14:14.829970+0000","last_active":"2024-05-27T10:18:43.010885+0000","last_peered":"2024-05-27T10:18:43.010885+0000","last_clean":"2024-05-27T10:18:43.010885+0000","last_became_active":"2024-05-27T10:14:14.829593+0000","last_became_peered":"2024-05-27T10:14:14.829593+0000","last_unstale":"2024-05-27T10:18:43.010885+0000","last_undegraded":"2024-05-27T10:18:43.010885+0000","last_fullsized":"2024-05-27T10:18:43.010885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:36.656453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":307,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:12.522270+0000","last_change":"2024-05-27T10:14:09.758400+0000","last_active":"2024-05-27T10:19:12.522270+0000","last_peered":"2024-05-27T10:19:12.522270+0000","last_clean":"2024-05-27T10:19:12.522270+0000","last_became_active":"2024-05-27T10:14:09.756020+0000","last_became_peered":"2024-05-27T10:14:09.756020+0000","last_unstale":"2024-05-27T10:19:12.522270+0000","last_undegraded":"2024-05-27T10:19:12.522270+0000","last_fullsized":"2024-05-27T10:19:12.522270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:40:58.345653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":272,"num_read_kb":272,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913526+0000","last_change":"2024-05-27T10:14:14.822607+0000","last_active":"2024-05-27T10:18:42.913526+0000","last_peered":"2024-05-27T10:18:42.913526+0000","last_clean":"2024-05-27T10:18:42.913526+0000","last_became_active":"2024-05-27T10:14:14.818801+0000","last_became_peered":"2024-05-27T10:14:14.818801+0000","last_unstale":"2024-05-27T10:18:42.913526+0000","last_undegraded":"2024-05-27T10:18:42.913526+0000","last_fullsized":"2024-05-27T10:18:42.913526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:18:25.032265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915807+0000","last_change":"2024-05-27T10:14:09.756973+0000","last_active":"2024-05-27T10:18:42.915807+0000","last_peered":"2024-05-27T10:18:42.915807+0000","last_clean":"2024-05-27T10:18:42.915807+0000","last_became_active":"2024-05-27T10:14:09.755809+0000","last_became_peered":"2024-05-27T10:14:09.755809+0000","last_unstale":"2024-05-27T10:18:42.915807+0000","last_undegraded":"2024-05-27T10:18:42.915807+0000","last_fullsized":"2024-05-27T10:18:42.915807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:07.099987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036613+0000","last_change":"2024-05-27T10:14:14.829595+0000","last_active":"2024-05-27T10:18:43.036613+0000","last_peered":"2024-05-27T10:18:43.036613+0000","last_clean":"2024-05-27T10:18:43.036613+0000","last_became_active":"2024-05-27T10:14:14.829147+0000","last_became_peered":"2024-05-27T10:14:14.829147+0000","last_unstale":"2024-05-27T10:18:43.036613+0000","last_undegraded":"2024-05-27T10:18:43.036613+0000","last_fullsized":"2024-05-27T10:18:43.036613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:47.613733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.029503+0000","last_change":"2024-05-27T10:14:09.920755+0000","last_active":"2024-05-27T10:18:43.029503+0000","last_peered":"2024-05-27T10:18:43.029503+0000","last_clean":"2024-05-27T10:18:43.029503+0000","last_became_active":"2024-05-27T10:14:09.919667+0000","last_became_peered":"2024-05-27T10:14:09.919667+0000","last_unstale":"2024-05-27T10:18:43.029503+0000","last_undegraded":"2024-05-27T10:18:43.029503+0000","last_fullsized":"2024-05-27T10:18:43.029503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:13:07.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.912674+0000","last_change":"2024-05-27T10:14:14.820113+0000","last_active":"2024-05-27T10:18:42.912674+0000","last_peered":"2024-05-27T10:18:42.912674+0000","last_clean":"2024-05-27T10:18:42.912674+0000","last_became_active":"2024-05-27T10:14:14.818592+0000","last_became_peered":"2024-05-27T10:14:14.818592+0000","last_unstale":"2024-05-27T10:18:42.912674+0000","last_undegraded":"2024-05-27T10:18:42.912674+0000","last_fullsized":"2024-05-27T10:18:42.912674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:16:14.743437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":625,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:13.469599+0000","last_change":"2024-05-27T10:14:09.766339+0000","last_active":"2024-05-27T10:19:13.469599+0000","last_peered":"2024-05-27T10:19:13.469599+0000","last_clean":"2024-05-27T10:19:13.469599+0000","last_became_active":"2024-05-27T10:14:09.765812+0000","last_became_peered":"2024-05-27T10:14:09.765812+0000","last_unstale":"2024-05-27T10:19:13.469599+0000","last_undegraded":"2024-05-27T10:19:13.469599+0000","last_fullsized":"2024-05-27T10:19:13.469599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:26.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":345,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150023+0000","last_change":"2024-05-27T10:14:14.817616+0000","last_active":"2024-05-27T10:18:43.150023+0000","last_peered":"2024-05-27T10:18:43.150023+0000","last_clean":"2024-05-27T10:18:43.150023+0000","last_became_active":"2024-05-27T10:14:14.817226+0000","last_became_peered":"2024-05-27T10:14:14.817226+0000","last_unstale":"2024-05-27T10:18:43.150023+0000","last_undegraded":"2024-05-27T10:18:43.150023+0000","last_fullsized":"2024-05-27T10:18:43.150023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:33.711009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892407+0000","last_change":"2024-05-27T10:14:09.757974+0000","last_active":"2024-05-27T10:18:42.892407+0000","last_peered":"2024-05-27T10:18:42.892407+0000","last_clean":"2024-05-27T10:18:42.892407+0000","last_became_active":"2024-05-27T10:14:09.754790+0000","last_became_peered":"2024-05-27T10:14:09.754790+0000","last_unstale":"2024-05-27T10:18:42.892407+0000","last_undegraded":"2024-05-27T10:18:42.892407+0000","last_fullsized":"2024-05-27T10:18:42.892407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:39.839122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858066+0000","last_change":"2024-05-27T10:14:14.835777+0000","last_active":"2024-05-27T10:18:42.858066+0000","last_peered":"2024-05-27T10:18:42.858066+0000","last_clean":"2024-05-27T10:18:42.858066+0000","last_became_active":"2024-05-27T10:14:14.835306+0000","last_became_peered":"2024-05-27T10:14:14.835306+0000","last_unstale":"2024-05-27T10:18:42.858066+0000","last_undegraded":"2024-05-27T10:18:42.858066+0000","last_fullsized":"2024-05-27T10:18:42.858066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:05.771167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028359+0000","last_change":"2024-05-27T10:14:09.766513+0000","last_active":"2024-05-27T10:18:43.028359+0000","last_peered":"2024-05-27T10:18:43.028359+0000","last_clean":"2024-05-27T10:18:43.028359+0000","last_became_active":"2024-05-27T10:14:09.766108+0000","last_became_peered":"2024-05-27T10:14:09.766108+0000","last_unstale":"2024-05-27T10:18:43.028359+0000","last_undegraded":"2024-05-27T10:18:43.028359+0000","last_fullsized":"2024-05-27T10:18:43.028359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:37:23.266123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915378+0000","last_change":"2024-05-27T10:14:14.840687+0000","last_active":"2024-05-27T10:18:42.915378+0000","last_peered":"2024-05-27T10:18:42.915378+0000","last_clean":"2024-05-27T10:18:42.915378+0000","last_became_active":"2024-05-27T10:14:14.840057+0000","last_became_peered":"2024-05-27T10:14:14.840057+0000","last_unstale":"2024-05-27T10:18:42.915378+0000","last_undegraded":"2024-05-27T10:18:42.915378+0000","last_fullsized":"2024-05-27T10:18:42.915378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:53.137056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":315,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:12.506010+0000","last_change":"2024-05-27T10:14:09.762194+0000","last_active":"2024-05-27T10:19:12.506010+0000","last_peered":"2024-05-27T10:19:12.506010+0000","last_clean":"2024-05-27T10:19:12.506010+0000","last_became_active":"2024-05-27T10:14:09.760764+0000","last_became_peered":"2024-05-27T10:14:09.760764+0000","last_unstale":"2024-05-27T10:19:12.506010+0000","last_undegraded":"2024-05-27T10:19:12.506010+0000","last_fullsized":"2024-05-27T10:19:12.506010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:21:28.256988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":279,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914514+0000","last_change":"2024-05-27T10:14:41.654514+0000","last_active":"2024-05-27T10:18:42.914514+0000","last_peered":"2024-05-27T10:18:42.914514+0000","last_clean":"2024-05-27T10:18:42.914514+0000","last_became_active":"2024-05-27T10:14:41.654050+0000","last_became_peered":"2024-05-27T10:14:41.654050+0000","last_unstale":"2024-05-27T10:18:42.914514+0000","last_undegraded":"2024-05-27T10:18:42.914514+0000","last_fullsized":"2024-05-27T10:18:42.914514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:13.077228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148905+0000","last_change":"2024-05-27T10:14:09.922680+0000","last_active":"2024-05-27T10:18:43.148905+0000","last_peered":"2024-05-27T10:18:43.148905+0000","last_clean":"2024-05-27T10:18:43.148905+0000","last_became_active":"2024-05-27T10:14:09.921448+0000","last_became_peered":"2024-05-27T10:14:09.921448+0000","last_unstale":"2024-05-27T10:18:43.148905+0000","last_undegraded":"2024-05-27T10:18:43.148905+0000","last_fullsized":"2024-05-27T10:18:43.148905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:13.216336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028385+0000","last_change":"2024-05-27T10:14:14.826493+0000","last_active":"2024-05-27T10:18:43.028385+0000","last_peered":"2024-05-27T10:18:43.028385+0000","last_clean":"2024-05-27T10:18:43.028385+0000","last_became_active":"2024-05-27T10:14:14.826110+0000","last_became_peered":"2024-05-27T10:14:14.826110+0000","last_unstale":"2024-05-27T10:18:43.028385+0000","last_undegraded":"2024-05-27T10:18:43.028385+0000","last_fullsized":"2024-05-27T10:18:43.028385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:10.405322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.641820+0000","last_change":"2024-05-27T10:14:09.765933+0000","last_active":"2024-05-27T10:18:43.641820+0000","last_peered":"2024-05-27T10:18:43.641820+0000","last_clean":"2024-05-27T10:18:43.641820+0000","last_became_active":"2024-05-27T10:14:09.765216+0000","last_became_peered":"2024-05-27T10:14:09.765216+0000","last_unstale":"2024-05-27T10:18:43.641820+0000","last_undegraded":"2024-05-27T10:18:43.641820+0000","last_fullsized":"2024-05-27T10:18:43.641820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:30:47.099708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865929+0000","last_change":"2024-05-27T10:14:14.827446+0000","last_active":"2024-05-27T10:18:42.865929+0000","last_peered":"2024-05-27T10:18:42.865929+0000","last_clean":"2024-05-27T10:18:42.865929+0000","last_became_active":"2024-05-27T10:14:14.826974+0000","last_became_peered":"2024-05-27T10:14:14.826974+0000","last_unstale":"2024-05-27T10:18:42.865929+0000","last_undegraded":"2024-05-27T10:18:42.865929+0000","last_fullsized":"2024-05-27T10:18:42.865929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:36.094987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010814+0000","last_change":"2024-05-27T10:14:09.928741+0000","last_active":"2024-05-27T10:18:43.010814+0000","last_peered":"2024-05-27T10:18:43.010814+0000","last_clean":"2024-05-27T10:18:43.010814+0000","last_became_active":"2024-05-27T10:14:09.925500+0000","last_became_peered":"2024-05-27T10:14:09.925500+0000","last_unstale":"2024-05-27T10:18:43.010814+0000","last_undegraded":"2024-05-27T10:18:43.010814+0000","last_fullsized":"2024-05-27T10:18:43.010814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:24.719898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.864668+0000","last_change":"2024-05-27T10:14:14.823118+0000","last_active":"2024-05-27T10:18:42.864668+0000","last_peered":"2024-05-27T10:18:42.864668+0000","last_clean":"2024-05-27T10:18:42.864668+0000","last_became_active":"2024-05-27T10:14:14.822789+0000","last_became_peered":"2024-05-27T10:14:14.822789+0000","last_unstale":"2024-05-27T10:18:42.864668+0000","last_undegraded":"2024-05-27T10:18:42.864668+0000","last_fullsized":"2024-05-27T10:18:42.864668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:08.875533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914915+0000","last_change":"2024-05-27T10:14:09.757268+0000","last_active":"2024-05-27T10:18:42.914915+0000","last_peered":"2024-05-27T10:18:42.914915+0000","last_clean":"2024-05-27T10:18:42.914915+0000","last_became_active":"2024-05-27T10:14:09.756243+0000","last_became_peered":"2024-05-27T10:14:09.756243+0000","last_unstale":"2024-05-27T10:18:42.914915+0000","last_undegraded":"2024-05-27T10:18:42.914915+0000","last_fullsized":"2024-05-27T10:18:42.914915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:04:15.628972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865657+0000","last_change":"2024-05-27T10:14:14.827130+0000","last_active":"2024-05-27T10:18:42.865657+0000","last_peered":"2024-05-27T10:18:42.865657+0000","last_clean":"2024-05-27T10:18:42.865657+0000","last_became_active":"2024-05-27T10:14:14.826746+0000","last_became_peered":"2024-05-27T10:14:14.826746+0000","last_unstale":"2024-05-27T10:18:42.865657+0000","last_undegraded":"2024-05-27T10:18:42.865657+0000","last_fullsized":"2024-05-27T10:18:42.865657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:51:37.159337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028265+0000","last_change":"2024-05-27T10:14:14.832500+0000","last_active":"2024-05-27T10:18:43.028265+0000","last_peered":"2024-05-27T10:18:43.028265+0000","last_clean":"2024-05-27T10:18:43.028265+0000","last_became_active":"2024-05-27T10:14:14.832000+0000","last_became_peered":"2024-05-27T10:14:14.832000+0000","last_unstale":"2024-05-27T10:18:43.028265+0000","last_undegraded":"2024-05-27T10:18:43.028265+0000","last_fullsized":"2024-05-27T10:18:43.028265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:45:37.313176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:12.504424+0000","last_change":"2024-05-27T10:14:09.923057+0000","last_active":"2024-05-27T10:19:12.504424+0000","last_peered":"2024-05-27T10:19:12.504424+0000","last_clean":"2024-05-27T10:19:12.504424+0000","last_became_active":"2024-05-27T10:14:09.922600+0000","last_became_peered":"2024-05-27T10:14:09.922600+0000","last_unstale":"2024-05-27T10:19:12.504424+0000","last_undegraded":"2024-05-27T10:19:12.504424+0000","last_fullsized":"2024-05-27T10:19:12.504424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:56:57.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":272,"num_read_kb":272,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013516+0000","last_change":"2024-05-27T10:14:14.832026+0000","last_active":"2024-05-27T10:18:43.013516+0000","last_peered":"2024-05-27T10:18:43.013516+0000","last_clean":"2024-05-27T10:18:43.013516+0000","last_became_active":"2024-05-27T10:14:14.831353+0000","last_became_peered":"2024-05-27T10:14:14.831353+0000","last_unstale":"2024-05-27T10:18:43.013516+0000","last_undegraded":"2024-05-27T10:18:43.013516+0000","last_fullsized":"2024-05-27T10:18:43.013516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:41.939043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036179+0000","last_change":"2024-05-27T10:14:09.765600+0000","last_active":"2024-05-27T10:18:43.036179+0000","last_peered":"2024-05-27T10:18:43.036179+0000","last_clean":"2024-05-27T10:18:43.036179+0000","last_became_active":"2024-05-27T10:14:09.765017+0000","last_became_peered":"2024-05-27T10:14:09.765017+0000","last_unstale":"2024-05-27T10:18:43.036179+0000","last_undegraded":"2024-05-27T10:18:43.036179+0000","last_fullsized":"2024-05-27T10:18:43.036179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:06.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"82'99","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:45.052019+0000","last_change":"2024-05-27T10:12:11.104089+0000","last_active":"2024-05-27T10:18:45.052019+0000","last_peered":"2024-05-27T10:18:45.052019+0000","last_clean":"2024-05-27T10:18:45.052019+0000","last_became_active":"2024-05-27T10:12:11.095070+0000","last_became_peered":"2024-05-27T10:12:11.095070+0000","last_unstale":"2024-05-27T10:18:45.052019+0000","last_undegraded":"2024-05-27T10:18:45.052019+0000","last_fullsized":"2024-05-27T10:18:45.052019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:07:06.610748+0000","last_clean_scrub_stamp":"2024-05-27T10:07:06.610748+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:36.168276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.857424+0000","last_change":"2024-05-27T10:14:14.837710+0000","last_active":"2024-05-27T10:18:42.857424+0000","last_peered":"2024-05-27T10:18:42.857424+0000","last_clean":"2024-05-27T10:18:42.857424+0000","last_became_active":"2024-05-27T10:14:14.837296+0000","last_became_peered":"2024-05-27T10:14:14.837296+0000","last_unstale":"2024-05-27T10:18:42.857424+0000","last_undegraded":"2024-05-27T10:18:42.857424+0000","last_fullsized":"2024-05-27T10:18:42.857424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:13.687195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.914885+0000","last_change":"2024-05-27T10:14:09.762348+0000","last_active":"2024-05-27T10:18:42.914885+0000","last_peered":"2024-05-27T10:18:42.914885+0000","last_clean":"2024-05-27T10:18:42.914885+0000","last_became_active":"2024-05-27T10:14:09.760843+0000","last_became_peered":"2024-05-27T10:14:09.760843+0000","last_unstale":"2024-05-27T10:18:42.914885+0000","last_undegraded":"2024-05-27T10:18:42.914885+0000","last_fullsized":"2024-05-27T10:18:42.914885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:11.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.149983+0000","last_change":"2024-05-27T10:14:09.925220+0000","last_active":"2024-05-27T10:18:43.149983+0000","last_peered":"2024-05-27T10:18:43.149983+0000","last_clean":"2024-05-27T10:18:43.149983+0000","last_became_active":"2024-05-27T10:14:09.923421+0000","last_became_peered":"2024-05-27T10:14:09.923421+0000","last_unstale":"2024-05-27T10:18:43.149983+0000","last_undegraded":"2024-05-27T10:18:43.149983+0000","last_fullsized":"2024-05-27T10:18:43.149983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:58:56.855166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858159+0000","last_change":"2024-05-27T10:14:14.830173+0000","last_active":"2024-05-27T10:18:42.858159+0000","last_peered":"2024-05-27T10:18:42.858159+0000","last_clean":"2024-05-27T10:18:42.858159+0000","last_became_active":"2024-05-27T10:14:14.825378+0000","last_became_peered":"2024-05-27T10:14:14.825378+0000","last_unstale":"2024-05-27T10:18:42.858159+0000","last_undegraded":"2024-05-27T10:18:42.858159+0000","last_fullsized":"2024-05-27T10:18:42.858159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:49.236898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'102","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:05.435234+0000","last_change":"2024-05-27T10:14:41.665781+0000","last_active":"2024-05-27T10:19:05.435234+0000","last_peered":"2024-05-27T10:19:05.435234+0000","last_clean":"2024-05-27T10:19:05.435234+0000","last_became_active":"2024-05-27T10:14:41.655442+0000","last_became_peered":"2024-05-27T10:14:41.655442+0000","last_unstale":"2024-05-27T10:19:05.435234+0000","last_undegraded":"2024-05-27T10:19:05.435234+0000","last_fullsized":"2024-05-27T10:19:05.435234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:00.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.014095+0000","last_change":"2024-05-27T10:14:14.825856+0000","last_active":"2024-05-27T10:18:43.014095+0000","last_peered":"2024-05-27T10:18:43.014095+0000","last_clean":"2024-05-27T10:18:43.014095+0000","last_became_active":"2024-05-27T10:14:14.822384+0000","last_became_peered":"2024-05-27T10:14:14.822384+0000","last_unstale":"2024-05-27T10:18:43.014095+0000","last_undegraded":"2024-05-27T10:18:43.014095+0000","last_fullsized":"2024-05-27T10:18:43.014095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:36.330100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036143+0000","last_change":"2024-05-27T10:14:10.491878+0000","last_active":"2024-05-27T10:18:43.036143+0000","last_peered":"2024-05-27T10:18:43.036143+0000","last_clean":"2024-05-27T10:18:43.036143+0000","last_became_active":"2024-05-27T10:14:10.491081+0000","last_became_peered":"2024-05-27T10:14:10.491081+0000","last_unstale":"2024-05-27T10:18:43.036143+0000","last_undegraded":"2024-05-27T10:18:43.036143+0000","last_fullsized":"2024-05-27T10:18:43.036143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:48.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.035987+0000","last_change":"2024-05-27T10:14:14.828156+0000","last_active":"2024-05-27T10:18:43.035987+0000","last_peered":"2024-05-27T10:18:43.035987+0000","last_clean":"2024-05-27T10:18:43.035987+0000","last_became_active":"2024-05-27T10:14:14.827484+0000","last_became_peered":"2024-05-27T10:14:14.827484+0000","last_unstale":"2024-05-27T10:18:43.035987+0000","last_undegraded":"2024-05-27T10:18:43.035987+0000","last_fullsized":"2024-05-27T10:18:43.035987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:03:36.546483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.910322+0000","last_change":"2024-05-27T10:14:09.924791+0000","last_active":"2024-05-27T10:18:42.910322+0000","last_peered":"2024-05-27T10:18:42.910322+0000","last_clean":"2024-05-27T10:18:42.910322+0000","last_became_active":"2024-05-27T10:14:09.923818+0000","last_became_peered":"2024-05-27T10:14:09.923818+0000","last_unstale":"2024-05-27T10:18:42.910322+0000","last_undegraded":"2024-05-27T10:18:42.910322+0000","last_fullsized":"2024-05-27T10:18:42.910322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:27:18.303437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.148975+0000","last_change":"2024-05-27T10:14:14.813965+0000","last_active":"2024-05-27T10:18:43.148975+0000","last_peered":"2024-05-27T10:18:43.148975+0000","last_clean":"2024-05-27T10:18:43.148975+0000","last_became_active":"2024-05-27T10:14:14.812390+0000","last_became_peered":"2024-05-27T10:14:14.812390+0000","last_unstale":"2024-05-27T10:18:43.148975+0000","last_undegraded":"2024-05-27T10:18:43.148975+0000","last_fullsized":"2024-05-27T10:18:43.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:24:36.869873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013173+0000","last_change":"2024-05-27T10:14:09.756108+0000","last_active":"2024-05-27T10:18:43.013173+0000","last_peered":"2024-05-27T10:18:43.013173+0000","last_clean":"2024-05-27T10:18:43.013173+0000","last_became_active":"2024-05-27T10:14:09.755646+0000","last_became_peered":"2024-05-27T10:14:09.755646+0000","last_unstale":"2024-05-27T10:18:43.013173+0000","last_undegraded":"2024-05-27T10:18:43.013173+0000","last_fullsized":"2024-05-27T10:18:43.013173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:08.399409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865624+0000","last_change":"2024-05-27T10:14:14.841131+0000","last_active":"2024-05-27T10:18:42.865624+0000","last_peered":"2024-05-27T10:18:42.865624+0000","last_clean":"2024-05-27T10:18:42.865624+0000","last_became_active":"2024-05-27T10:14:14.840419+0000","last_became_peered":"2024-05-27T10:14:14.840419+0000","last_unstale":"2024-05-27T10:18:42.865624+0000","last_undegraded":"2024-05-27T10:18:42.865624+0000","last_fullsized":"2024-05-27T10:18:42.865624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:59.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865650+0000","last_change":"2024-05-27T10:14:41.659057+0000","last_active":"2024-05-27T10:18:42.865650+0000","last_peered":"2024-05-27T10:18:42.865650+0000","last_clean":"2024-05-27T10:18:42.865650+0000","last_became_active":"2024-05-27T10:14:41.650636+0000","last_became_peered":"2024-05-27T10:14:41.650636+0000","last_unstale":"2024-05-27T10:18:42.865650+0000","last_undegraded":"2024-05-27T10:18:42.865650+0000","last_fullsized":"2024-05-27T10:18:42.865650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:09:15.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013727+0000","last_change":"2024-05-27T10:14:14.840418+0000","last_active":"2024-05-27T10:18:43.013727+0000","last_peered":"2024-05-27T10:18:43.013727+0000","last_clean":"2024-05-27T10:18:43.013727+0000","last_became_active":"2024-05-27T10:14:14.840023+0000","last_became_peered":"2024-05-27T10:14:14.840023+0000","last_unstale":"2024-05-27T10:18:43.013727+0000","last_undegraded":"2024-05-27T10:18:43.013727+0000","last_fullsized":"2024-05-27T10:18:43.013727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:18:16.847768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.150814+0000","last_change":"2024-05-27T10:14:41.663013+0000","last_active":"2024-05-27T10:18:43.150814+0000","last_peered":"2024-05-27T10:18:43.150814+0000","last_clean":"2024-05-27T10:18:43.150814+0000","last_became_active":"2024-05-27T10:14:41.655978+0000","last_became_peered":"2024-05-27T10:14:41.655978+0000","last_unstale":"2024-05-27T10:18:43.150814+0000","last_undegraded":"2024-05-27T10:18:43.150814+0000","last_fullsized":"2024-05-27T10:18:43.150814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:05.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915235+0000","last_change":"2024-05-27T10:14:14.835391+0000","last_active":"2024-05-27T10:18:42.915235+0000","last_peered":"2024-05-27T10:18:42.915235+0000","last_clean":"2024-05-27T10:18:42.915235+0000","last_became_active":"2024-05-27T10:14:14.833566+0000","last_became_peered":"2024-05-27T10:14:14.833566+0000","last_unstale":"2024-05-27T10:18:42.915235+0000","last_undegraded":"2024-05-27T10:18:42.915235+0000","last_fullsized":"2024-05-27T10:18:42.915235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:02.482584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892674+0000","last_change":"2024-05-27T10:14:10.492426+0000","last_active":"2024-05-27T10:18:42.892674+0000","last_peered":"2024-05-27T10:18:42.892674+0000","last_clean":"2024-05-27T10:18:42.892674+0000","last_became_active":"2024-05-27T10:14:10.490798+0000","last_became_peered":"2024-05-27T10:14:10.490798+0000","last_unstale":"2024-05-27T10:18:42.892674+0000","last_undegraded":"2024-05-27T10:18:42.892674+0000","last_fullsized":"2024-05-27T10:18:42.892674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:53.803187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916054+0000","last_change":"2024-05-27T10:14:14.835895+0000","last_active":"2024-05-27T10:18:42.916054+0000","last_peered":"2024-05-27T10:18:42.916054+0000","last_clean":"2024-05-27T10:18:42.916054+0000","last_became_active":"2024-05-27T10:14:14.835530+0000","last_became_peered":"2024-05-27T10:14:14.835530+0000","last_unstale":"2024-05-27T10:18:42.916054+0000","last_undegraded":"2024-05-27T10:18:42.916054+0000","last_fullsized":"2024-05-27T10:18:42.916054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:38.380555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'65","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:19:04.813909+0000","last_change":"2024-05-27T10:14:09.760927+0000","last_active":"2024-05-27T10:19:04.813909+0000","last_peered":"2024-05-27T10:19:04.813909+0000","last_clean":"2024-05-27T10:19:04.813909+0000","last_became_active":"2024-05-27T10:14:09.758944+0000","last_became_peered":"2024-05-27T10:14:09.758944+0000","last_unstale":"2024-05-27T10:19:04.813909+0000","last_undegraded":"2024-05-27T10:19:04.813909+0000","last_fullsized":"2024-05-27T10:19:04.813909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:12:05.155937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.915511+0000","last_change":"2024-05-27T10:14:14.831048+0000","last_active":"2024-05-27T10:18:42.915511+0000","last_peered":"2024-05-27T10:18:42.915511+0000","last_clean":"2024-05-27T10:18:42.915511+0000","last_became_active":"2024-05-27T10:14:14.830604+0000","last_became_peered":"2024-05-27T10:14:14.830604+0000","last_unstale":"2024-05-27T10:18:42.915511+0000","last_undegraded":"2024-05-27T10:18:42.915511+0000","last_fullsized":"2024-05-27T10:18:42.915511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:29:15.213365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858229+0000","last_change":"2024-05-27T10:14:09.750668+0000","last_active":"2024-05-27T10:18:42.858229+0000","last_peered":"2024-05-27T10:18:42.858229+0000","last_clean":"2024-05-27T10:18:42.858229+0000","last_became_active":"2024-05-27T10:14:09.749739+0000","last_became_peered":"2024-05-27T10:14:09.749739+0000","last_unstale":"2024-05-27T10:18:42.858229+0000","last_undegraded":"2024-05-27T10:18:42.858229+0000","last_fullsized":"2024-05-27T10:18:42.858229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:08.710033+0000","last_clean_scrub_stamp":"2024-05-27T10:14:08.710033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:33:47.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.916095+0000","last_change":"2024-05-27T10:14:14.827088+0000","last_active":"2024-05-27T10:18:42.916095+0000","last_peered":"2024-05-27T10:18:42.916095+0000","last_clean":"2024-05-27T10:18:42.916095+0000","last_became_active":"2024-05-27T10:14:14.824166+0000","last_became_peered":"2024-05-27T10:14:14.824166+0000","last_unstale":"2024-05-27T10:18:42.916095+0000","last_undegraded":"2024-05-27T10:18:42.916095+0000","last_fullsized":"2024-05-27T10:18:42.916095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:49.972864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865960+0000","last_change":"2024-05-27T10:14:14.824147+0000","last_active":"2024-05-27T10:18:42.865960+0000","last_peered":"2024-05-27T10:18:42.865960+0000","last_clean":"2024-05-27T10:18:42.865960+0000","last_became_active":"2024-05-27T10:14:14.823651+0000","last_became_peered":"2024-05-27T10:14:14.823651+0000","last_unstale":"2024-05-27T10:18:42.865960+0000","last_undegraded":"2024-05-27T10:18:42.865960+0000","last_fullsized":"2024-05-27T10:18:42.865960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:07.950306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892084+0000","last_change":"2024-05-27T10:14:14.821198+0000","last_active":"2024-05-27T10:18:42.892084+0000","last_peered":"2024-05-27T10:18:42.892084+0000","last_clean":"2024-05-27T10:18:42.892084+0000","last_became_active":"2024-05-27T10:14:14.820764+0000","last_became_peered":"2024-05-27T10:14:14.820764+0000","last_unstale":"2024-05-27T10:18:42.892084+0000","last_undegraded":"2024-05-27T10:18:42.892084+0000","last_fullsized":"2024-05-27T10:18:42.892084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:55:34.499740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.891669+0000","last_change":"2024-05-27T10:14:14.831572+0000","last_active":"2024-05-27T10:18:42.891669+0000","last_peered":"2024-05-27T10:18:42.891669+0000","last_clean":"2024-05-27T10:18:42.891669+0000","last_became_active":"2024-05-27T10:14:14.831167+0000","last_became_peered":"2024-05-27T10:14:14.831167+0000","last_unstale":"2024-05-27T10:18:42.891669+0000","last_undegraded":"2024-05-27T10:18:42.891669+0000","last_fullsized":"2024-05-27T10:18:42.891669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:45.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.036165+0000","last_change":"2024-05-27T10:14:14.832122+0000","last_active":"2024-05-27T10:18:43.036165+0000","last_peered":"2024-05-27T10:18:43.036165+0000","last_clean":"2024-05-27T10:18:43.036165+0000","last_became_active":"2024-05-27T10:14:14.831588+0000","last_became_peered":"2024-05-27T10:14:14.831588+0000","last_unstale":"2024-05-27T10:18:43.036165+0000","last_undegraded":"2024-05-27T10:18:43.036165+0000","last_fullsized":"2024-05-27T10:18:43.036165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:16.651734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858813+0000","last_change":"2024-05-27T10:14:14.842336+0000","last_active":"2024-05-27T10:18:42.858813+0000","last_peered":"2024-05-27T10:18:42.858813+0000","last_clean":"2024-05-27T10:18:42.858813+0000","last_became_active":"2024-05-27T10:14:14.841924+0000","last_became_peered":"2024-05-27T10:14:14.841924+0000","last_unstale":"2024-05-27T10:18:42.858813+0000","last_undegraded":"2024-05-27T10:18:42.858813+0000","last_fullsized":"2024-05-27T10:18:42.858813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:23.889881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.865913+0000","last_change":"2024-05-27T10:14:14.820734+0000","last_active":"2024-05-27T10:18:42.865913+0000","last_peered":"2024-05-27T10:18:42.865913+0000","last_clean":"2024-05-27T10:18:42.865913+0000","last_became_active":"2024-05-27T10:14:14.818406+0000","last_became_peered":"2024-05-27T10:14:14.818406+0000","last_unstale":"2024-05-27T10:18:42.865913+0000","last_undegraded":"2024-05-27T10:18:42.865913+0000","last_fullsized":"2024-05-27T10:18:42.865913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:20.905355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.892758+0000","last_change":"2024-05-27T10:14:14.827905+0000","last_active":"2024-05-27T10:18:42.892758+0000","last_peered":"2024-05-27T10:18:42.892758+0000","last_clean":"2024-05-27T10:18:42.892758+0000","last_became_active":"2024-05-27T10:14:14.827023+0000","last_became_peered":"2024-05-27T10:14:14.827023+0000","last_unstale":"2024-05-27T10:18:42.892758+0000","last_undegraded":"2024-05-27T10:18:42.892758+0000","last_fullsized":"2024-05-27T10:18:42.892758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:09.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.010964+0000","last_change":"2024-05-27T10:14:41.655357+0000","last_active":"2024-05-27T10:18:43.010964+0000","last_peered":"2024-05-27T10:18:43.010964+0000","last_clean":"2024-05-27T10:18:43.010964+0000","last_became_active":"2024-05-27T10:14:41.654533+0000","last_became_peered":"2024-05-27T10:14:41.654533+0000","last_unstale":"2024-05-27T10:18:43.010964+0000","last_undegraded":"2024-05-27T10:18:43.010964+0000","last_fullsized":"2024-05-27T10:18:43.010964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:10.126802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.028084+0000","last_change":"2024-05-27T10:14:14.830584+0000","last_active":"2024-05-27T10:18:43.028084+0000","last_peered":"2024-05-27T10:18:43.028084+0000","last_clean":"2024-05-27T10:18:43.028084+0000","last_became_active":"2024-05-27T10:14:14.830059+0000","last_became_peered":"2024-05-27T10:14:14.830059+0000","last_unstale":"2024-05-27T10:18:43.028084+0000","last_undegraded":"2024-05-27T10:18:43.028084+0000","last_fullsized":"2024-05-27T10:18:43.028084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:41.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:43.013207+0000","last_change":"2024-05-27T10:14:14.817864+0000","last_active":"2024-05-27T10:18:43.013207+0000","last_peered":"2024-05-27T10:18:43.013207+0000","last_clean":"2024-05-27T10:18:43.013207+0000","last_became_active":"2024-05-27T10:14:14.817411+0000","last_became_peered":"2024-05-27T10:14:14.817411+0000","last_unstale":"2024-05-27T10:18:43.013207+0000","last_undegraded":"2024-05-27T10:18:43.013207+0000","last_fullsized":"2024-05-27T10:18:43.013207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:43:13.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.913669+0000","last_change":"2024-05-27T10:14:14.831673+0000","last_active":"2024-05-27T10:18:42.913669+0000","last_peered":"2024-05-27T10:18:42.913669+0000","last_clean":"2024-05-27T10:18:42.913669+0000","last_became_active":"2024-05-27T10:14:14.829620+0000","last_became_peered":"2024-05-27T10:14:14.829620+0000","last_unstale":"2024-05-27T10:18:42.913669+0000","last_undegraded":"2024-05-27T10:18:42.913669+0000","last_fullsized":"2024-05-27T10:18:42.913669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:39.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.858143+0000","last_change":"2024-05-27T10:14:14.841100+0000","last_active":"2024-05-27T10:18:42.858143+0000","last_peered":"2024-05-27T10:18:42.858143+0000","last_clean":"2024-05-27T10:18:42.858143+0000","last_became_active":"2024-05-27T10:14:14.840594+0000","last_became_peered":"2024-05-27T10:14:14.840594+0000","last_unstale":"2024-05-27T10:18:42.858143+0000","last_undegraded":"2024-05-27T10:18:42.858143+0000","last_fullsized":"2024-05-27T10:18:42.858143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:48:22.869849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-27T10:18:42.866758+0000","last_change":"2024-05-27T10:14:14.832750+0000","last_active":"2024-05-27T10:18:42.866758+0000","last_peered":"2024-05-27T10:18:42.866758+0000","last_clean":"2024-05-27T10:18:42.866758+0000","last_became_active":"2024-05-27T10:14:14.830996+0000","last_became_peered":"2024-05-27T10:14:14.830996+0000","last_unstale":"2024-05-27T10:18:42.866758+0000","last_undegraded":"2024-05-27T10:18:42.866758+0000","last_fullsized":"2024-05-27T10:18:42.866758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T10:14:13.756804+0000","last_clean_scrub_stamp":"2024-05-27T10:14:13.756804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:27.686223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108456,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796077,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2018,"num_read_kb":1226,"num_write":431,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2388231,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678095,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":830200,"data_compressed":22182,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874326,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1425518,"data_compressed":54247,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":830436,"data_compressed":22182,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234086,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":437140,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430316,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":1032293,"data_compressed":39045,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593844,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":888832,"data_stored":436609,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":503807,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738529,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":437880,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411476,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":507355,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149836,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1158368,"data_compressed":37384,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953605,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":438462,"data_compressed":2893,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790076,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1358793,"data_compressed":54247,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36152,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331906,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":398631,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T10:19:20.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:19:21.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:20 smithi104 ceph-mon[34812]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:19:21.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:20 smithi104 ceph-mon[34812]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:21.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:20 smithi143 ceph-mon[35545]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:19:21.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:20 smithi143 ceph-mon[35545]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:21.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:20 smithi042 ceph-mon[31661]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:19:21.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:20 smithi042 ceph-mon[31661]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T10:19:21.471 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:23.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:22 smithi042 ceph-mon[31661]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-27T10:19:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:22 smithi104 ceph-mon[34812]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-27T10:19:23.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:22 smithi143 ceph-mon[35545]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-27T10:19:23.925 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:23 smithi042 ceph-mon[31661]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:19:24.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:23 smithi104 ceph-mon[34812]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:19:24.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:23 smithi143 ceph-mon[35545]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:19:24.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.333+0000 7f78dac00700 1 -- 172.21.15.42:0/1261366265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d4106610 msgr2=0x7f78d4106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:24.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.333+0000 7f78dac00700 1 --2- 172.21.15.42:0/1261366265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d4106610 0x7f78d4106a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f78c800b0d0 tx=0x7f78c800b3e0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 -- 172.21.15.42:0/1261366265 shutdown_connections 2024-05-27T10:19:24.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 --2- 172.21.15.42:0/1261366265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d410b890 0x7f78d41131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 --2- 172.21.15.42:0/1261366265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d4106610 0x7f78d4106a20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 --2- 172.21.15.42:0/1261366265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 0x7f78d410b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 -- 172.21.15.42:0/1261366265 >> 172.21.15.42:0/1261366265 conn(0x7f78d4102020 msgr2=0x7f78d4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 -- 172.21.15.42:0/1261366265 shutdown_connections 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.334+0000 7f78dac00700 1 -- 172.21.15.42:0/1261366265 wait complete. 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.335+0000 7f78dac00700 1 Processor -- start 2024-05-27T10:19:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.335+0000 7f78dac00700 1 -- start start 2024-05-27T10:19:24.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78dac00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d4106610 0x7f78d419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:24.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78dac00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 0x7f78d419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:24.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78dac00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 0x7f78d41a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:24.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78dac00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41163c0 con 0x7f78d410b890 2024-05-27T10:19:24.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78dac00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4116240 con 0x7f78d4106610 2024-05-27T10:19:24.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78dac00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4116540 con 0x7f78d410af40 2024-05-27T10:19:24.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78da200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 0x7f78d41a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78da200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 0x7f78d41a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41250/0 (socket says 172.21.15.42:41250) 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78da200700 1 -- 172.21.15.42:0/1154570392 learned_addr learned my addr 172.21.15.42:0/1154570392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78d9800700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d4106610 0x7f78d419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.336+0000 7f78d8e00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 0x7f78d419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78da200700 1 -- 172.21.15.42:0/1154570392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 msgr2=0x7f78d419d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78da200700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 0x7f78d419d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78da200700 1 -- 172.21.15.42:0/1154570392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d4106610 msgr2=0x7f78d419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78da200700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d4106610 0x7f78d419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78da200700 1 -- 172.21.15.42:0/1154570392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d41a1bb0 con 0x7f78d410b890 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78d8e00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 0x7f78d419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78da200700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 0x7f78d41a15c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c900 tx=0x7f78cc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.337+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc007b00 con 0x7f78d410b890 2024-05-27T10:19:24.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.338+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78cc00ce50 con 0x7f78d410b890 2024-05-27T10:19:24.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.338+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d41a1ea0 con 0x7f78d410b890 2024-05-27T10:19:24.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.338+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc018660 con 0x7f78d410b890 2024-05-27T10:19:24.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.338+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d41a23e0 con 0x7f78d410b890 2024-05-27T10:19:24.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.339+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d40512b0 con 0x7f78d410b890 2024-05-27T10:19:24.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.341+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f78cc00f450 con 0x7f78d410b890 2024-05-27T10:19:24.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.343+0000 7f78d2000700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78bc0b2b90 0x7f78bc0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:24.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.343+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f78cc0e2710 con 0x7f78d410b890 2024-05-27T10:19:24.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.343+0000 7f78d9800700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78bc0b2b90 0x7f78bc0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:24.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.344+0000 7f78d9800700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78bc0b2b90 0x7f78bc0b50a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c800b040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:24.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.345+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78cc014030 con 0x7f78d410b890 2024-05-27T10:19:24.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.637+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d402e5e0 con 0x7f78d410b890 2024-05-27T10:19:24.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.638+0000 7f78d2000700 1 -- 172.21.15.42:0/1154570392 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f78cc0a9fa0 con 0x7f78d410b890 2024-05-27T10:19:24.638 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:24.638 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:19:24.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78bc0b2b90 msgr2=0x7f78bc0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78bc0b2b90 0x7f78bc0b50a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c800b040 comp rx=0 tx=0).stop 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 msgr2=0x7f78d41a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 0x7f78d41a15c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c900 tx=0x7f78cc00cc10 comp rx=0 tx=0).stop 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 shutdown_connections 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78bc0b2b90 0x7f78bc0b50a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78d410b890 0x7f78d41a15c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d4106610 0x7f78d419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 --2- 172.21.15.42:0/1154570392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d410af40 0x7f78d419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.641+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 >> 172.21.15.42:0/1154570392 conn(0x7f78d4102020 msgr2=0x7f78d410a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.642+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 shutdown_connections 2024-05-27T10:19:24.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:24.642+0000 7f78dac00700 1 -- 172.21.15.42:0/1154570392 wait complete. 2024-05-27T10:19:24.644 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:19:25.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:24 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1154570392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:25.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:24 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1154570392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:25.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:24 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1154570392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:25.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:19:26.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:25 smithi042 ceph-mon[31661]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T10:19:26.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:25 smithi104 ceph-mon[34812]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T10:19:26.237 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:26.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:25 smithi143 ceph-mon[35545]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T10:19:28.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:27 smithi104 ceph-mon[34812]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:28.220 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:27 smithi042 ceph-mon[31661]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:28.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:27 smithi143 ceph-mon[35545]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 -- 172.21.15.42:0/3409611543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410af40 msgr2=0x7fbdf410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/3409611543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410af40 0x7fbdf410b350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbde00099c0 tx=0x7fbde0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 -- 172.21.15.42:0/3409611543 shutdown_connections 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/3409611543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf410b890 0x7fbdf41131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/3409611543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf4106610 0x7fbdf4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/3409611543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410af40 0x7fbdf410b350 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 -- 172.21.15.42:0/3409611543 >> 172.21.15.42:0/3409611543 conn(0x7fbdf4102020 msgr2=0x7fbdf4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.870+0000 7fbdf3e00700 1 -- 172.21.15.42:0/3409611543 shutdown_connections 2024-05-27T10:19:28.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.871+0000 7fbdf3e00700 1 -- 172.21.15.42:0/3409611543 wait complete. 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 Processor -- start 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 -- start start 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf4106610 0x7fbdf419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf410af40 0x7fbdf419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 0x7fbdf41a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41163c0 con 0x7fbdf410b890 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4116240 con 0x7fbdf410af40 2024-05-27T10:19:28.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.872+0000 7fbdf3e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4116540 con 0x7fbdf4106610 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 0x7fbdf41a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 0x7fbdf41a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41268/0 (socket says 172.21.15.42:41268) 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 -- 172.21.15.42:0/4240823783 learned_addr learned my addr 172.21.15.42:0/4240823783 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf2a00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf4106610 0x7fbdf419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf2000700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf410af40 0x7fbdf419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 -- 172.21.15.42:0/4240823783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf4106610 msgr2=0x7fbdf419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf4106610 0x7fbdf419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 -- 172.21.15.42:0/4240823783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf410af40 msgr2=0x7fbdf419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf410af40 0x7fbdf419d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:28.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.873+0000 7fbdf3400700 1 -- 172.21.15.42:0/4240823783 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf41a1c80 con 0x7fbdf410b890 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.874+0000 7fbdf3400700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 0x7fbdf41a15e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbdec007ee0 tx=0x7fbdec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.874+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00dbd0 con 0x7fbdf410b890 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.874+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdec00dd70 con 0x7fbdf410b890 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.874+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf41a1f70 con 0x7fbdf410b890 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.874+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0058d0 con 0x7fbdf410b890 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.874+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf41a9890 con 0x7fbdf410b890 2024-05-27T10:19:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.876+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf40512b0 con 0x7fbdf410b890 2024-05-27T10:19:28.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.878+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fbdec01a020 con 0x7fbdf410b890 2024-05-27T10:19:28.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.880+0000 7fbdeb400700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbdd80b2b10 0x7fbdd80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:28.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.880+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbdec0e20c0 con 0x7fbdf410b890 2024-05-27T10:19:28.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.880+0000 7fbdf2a00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbdd80b2b10 0x7fbdd80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:28.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.881+0000 7fbdf2a00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbdd80b2b10 0x7fbdd80b5020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbde4005950 tx=0x7fbde40058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:28.885+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdec010070 con 0x7fbdf410b890 2024-05-27T10:19:29.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.181+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdf402e5e0 con 0x7fbdf410b890 2024-05-27T10:19:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.182+0000 7fbdeb400700 1 -- 172.21.15.42:0/4240823783 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fbdec0a9840 con 0x7fbdf410b890 2024-05-27T10:19:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:29.183 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:19:29.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.185+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbdd80b2b10 msgr2=0x7fbdd80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.185+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbdd80b2b10 0x7fbdd80b5020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbde4005950 tx=0x7fbde40058e0 comp rx=0 tx=0).stop 2024-05-27T10:19:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.185+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 msgr2=0x7fbdf41a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.185+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 0x7fbdf41a15e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbdec007ee0 tx=0x7fbdec00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:19:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 shutdown_connections 2024-05-27T10:19:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbdd80b2b10 0x7fbdd80b5020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf410b890 0x7fbdf41a15e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf4106610 0x7fbdf419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 --2- 172.21.15.42:0/4240823783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf410af40 0x7fbdf419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 >> 172.21.15.42:0/4240823783 conn(0x7fbdf4102020 msgr2=0x7fbdf410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 shutdown_connections 2024-05-27T10:19:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:29.186+0000 7fbdf3e00700 1 -- 172.21.15.42:0/4240823783 wait complete. 2024-05-27T10:19:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:19:30.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:29 smithi104 ceph-mon[34812]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:30.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:29 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4240823783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:30.219 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:29 smithi042 ceph-mon[31661]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:30.219 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:29 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4240823783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:30.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T10:19:30.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:29 smithi143 ceph-mon[35545]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:30.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:29 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4240823783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:30.782 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:32.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:31 smithi104 ceph-mon[34812]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T10:19:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.251+0000 7f1e65000640 1 -- 172.21.15.42:0/2650031622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 msgr2=0x7f1e600769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.251+0000 7f1e65000640 1 --2- 172.21.15.42:0/2650031622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e600769a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009900 tx=0x7f1e5002a3f0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 -- 172.21.15.42:0/2650031622 shutdown_connections 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 --2- 172.21.15.42:0/2650031622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 0x7f1e60116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 --2- 172.21.15.42:0/2650031622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e60112110 0x7f1e60114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 --2- 172.21.15.42:0/2650031622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e600769a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 -- 172.21.15.42:0/2650031622 >> 172.21.15.42:0/2650031622 conn(0x7f1e600ff170 msgr2=0x7f1e601015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 -- 172.21.15.42:0/2650031622 shutdown_connections 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.252+0000 7f1e65000640 1 -- 172.21.15.42:0/2650031622 wait complete. 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.253+0000 7f1e65000640 1 Processor -- start 2024-05-27T10:19:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.253+0000 7f1e65000640 1 -- start start 2024-05-27T10:19:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e65000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e601a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e65000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e60112110 0x7f1e601a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e65000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 0x7f1e601a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e65000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6011a210 con 0x7f1e60076590 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e65000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6011a090 con 0x7f1e60112110 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e65000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6011a390 con 0x7f1e60114a70 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e601a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e601a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41278/0 (socket says 172.21.15.42:41278) 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 -- 172.21.15.42:0/74527511 learned_addr learned my addr 172.21.15.42:0/74527511 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e60112110 0x7f1e601a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 0x7f1e601a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 -- 172.21.15.42:0/74527511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 msgr2=0x7f1e601a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 0x7f1e601a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 -- 172.21.15.42:0/74527511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e60112110 msgr2=0x7f1e601a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e60112110 0x7f1e601a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5f400640 1 -- 172.21.15.42:0/74527511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e601a1110 con 0x7f1e60076590 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.254+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 0x7f1e601a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.255+0000 7f1e5f400640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e601a6860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1e5002a910 tx=0x7f1e50003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.255+0000 7f1e57e00640 1 -- 172.21.15.42:0/74527511 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5003f070 con 0x7f1e60076590 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.255+0000 7f1e57e00640 1 -- 172.21.15.42:0/74527511 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e50003e10 con 0x7f1e60076590 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.255+0000 7f1e57e00640 1 -- 172.21.15.42:0/74527511 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50035030 con 0x7f1e60076590 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.255+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e601a13a0 con 0x7f1e60076590 2024-05-27T10:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.255+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e601a1840 con 0x7f1e60076590 2024-05-27T10:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 Processor -- start 2024-05-27T10:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 -- start start 2024-05-27T10:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e57e00640 1 -- 172.21.15.42:0/74527511 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f1e50042070 con 0x7f1e60076590 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1d0421b90 con 0x55a1d00ed4d0 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a1d0411730 con 0x55a1d010e1b0 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.259+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a1d0415770 con 0x55a1d00e52f0 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41282/0 (socket says 172.21.15.42:41282) 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 -- 172.21.15.42:0/2150935662 learned_addr learned my addr 172.21.15.42:0/2150935662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5f400640 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 -- 172.21.15.42:0/2150935662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 msgr2=0x55a1d04b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 -- 172.21.15.42:0/2150935662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 msgr2=0x55a1d04b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.260+0000 7f1e5ea00640 1 -- 172.21.15.42:0/2150935662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1d03aa8b0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.261+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b14e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800bc10 tx=0x7f1e4800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdfe06bc68f8c29 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.261+0000 7f1e54c00640 1 -- 172.21.15.42:0/2150935662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48015070 con 0x55a1d00ed4d0 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.261+0000 7f1e54c00640 1 -- 172.21.15.42:0/2150935662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e4801a8f0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.261+0000 7f1e54c00640 1 -- 172.21.15.42:0/2150935662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480117c0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.261+0000 7f1e57e00640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1e440b2af0 0x7f1e440b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.262+0000 7f1e57e00640 1 -- 172.21.15.42:0/74527511 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1e50102ff0 con 0x7f1e60076590 2024-05-27T10:19:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.262+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1e440b2af0 0x7f1e440b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.262+0000 7f1e6d59c740 1 -- 172.21.15.42:0/2150935662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 msgr2=0x55a1d04b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.262+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b14e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800bc10 tx=0x7f1e4800f730 comp rx=0 tx=0).stop 2024-05-27T10:19:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 -- 172.21.15.42:0/2150935662 shutdown_connections 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b14e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/2150935662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 -- 172.21.15.42:0/2150935662 >> 172.21.15.42:0/2150935662 conn(0x55a1d0102300 msgr2=0x55a1d04ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1e440b2af0 0x7f1e440b5000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55a1d04adbd0 tx=0x7f1e58009700 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 -- 172.21.15.42:0/2150935662 shutdown_connections 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 -- 172.21.15.42:0/2150935662 wait complete. 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 -- start start 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.263+0000 7f1e6d59c740 1 Processor -- start 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04babe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1d00f7790 con 0x55a1d00ed4d0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a1d03b9bc0 con 0x55a1d010e1b0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a1d04bdb90 con 0x55a1d00e52f0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e6d59c740 10 client.0 _ll_register_callbacks cb 0x55a1d0397400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04bb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41284/0 (socket says 172.21.15.42:41284) 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.264+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1362826964 learned_addr learned my addr 172.21.15.42:0/1362826964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04babe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1362826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 msgr2=0x55a1d04babe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04babe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 msgr2=0x55a1d04c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1362826964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1d0126210 con 0x55a1d00ed4d0 2024-05-27T10:19:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04babe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04bb120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e48005780 tx=0x7f1e4800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b971d626db99c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e4d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.265+0000 7f1e4d600640 1 -- 172.21.15.42:0/1362826964 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48027060 con 0x55a1d00ed4d0 2024-05-27T10:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e4d600640 1 -- 172.21.15.42:0/1362826964 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1e4801aca0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4801a8f0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e4d600640 1 -- 172.21.15.42:0/1362826964 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e48015040 con 0x55a1d00ed4d0 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e6d59c740 10 client.15411 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1d01f2620 con 0x55a1d00ed4d0 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e4d600640 1 -- 172.21.15.42:0/1362826964 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e4801a8f0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.266+0000 7f1e4d600640 1 -- 172.21.15.42:0/1362826964 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e48022070 con 0x55a1d00ed4d0 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e6d59c740 10 client.15411 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e6d59c740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e6d59c740 20 client.15411 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00ccd80 0x55a1d04ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e6d59c740 4 client.15411 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1d0423ae0 con 0x55a1d00ccd80 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.267+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00ccd80 0x55a1d04ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.268+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00ccd80 0x55a1d04ccce0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc796da09e70de server_cookie=271b432bc141587c in_seq=0 out_seq=0 2024-05-27T10:19:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.268+0000 7f1e4d600640 10 client.15411 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.270+0000 7f1e4d600640 1 -- 172.21.15.42:0/1362826964 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a1d0423ae0 con 0x55a1d00ccd80 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.270+0000 7f1e4d600640 10 client.15411 handle_command_reply: tid=0 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.270+0000 7f1e6d59c740 1 client.15411 shutdown 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 msgr2=0x55a1d04bb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04bb120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e48005780 tx=0x7f1e4800bfa0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 shutdown_connections 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d010e1b0 0x55a1d04c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04bb120 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00ccd80 0x55a1d04ccce0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1362826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d00e52f0 0x55a1d04babe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 >> 172.21.15.42:0/1362826964 conn(0x55a1d0102300 msgr2=0x55a1d04b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e4d600640 0 client.15411 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 shutdown_connections 2024-05-27T10:19:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1362826964 wait complete. 2024-05-27T10:19:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.271+0000 7f1e6d59c740 20 client.15411 trim_cache size 0 max 16384 2024-05-27T10:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.291+0000 7f1e6d59c740 1 Processor -- start 2024-05-27T10:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 -- start start 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00e52f0 0x55a1d04b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601b8610 con 0x55a1d00ed4d0 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601b5ee0 con 0x55a1d00e52f0 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601b3d30 con 0x55a1d010e1b0 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41288/0 (socket says 172.21.15.42:41288) 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.292+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1542082900 learned_addr learned my addr 172.21.15.42:0/1542082900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00e52f0 0x55a1d04b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1542082900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 msgr2=0x55a1d04b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1542082900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00e52f0 msgr2=0x55a1d04b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00e52f0 0x55a1d04b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5ea00640 1 -- 172.21.15.42:0/1542082900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1d04adde0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00e52f0 0x55a1d04b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.293+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b47e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800bd40 tx=0x7f1e4800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525ac3b440634ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.294+0000 7f1e4fe00640 1 -- 172.21.15.42:0/1542082900 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48027040 con 0x55a1d00ed4d0 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.294+0000 7f1e4fe00640 1 -- 172.21.15.42:0/1542082900 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e480037f0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.294+0000 7f1e4fe00640 1 -- 172.21.15.42:0/1542082900 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48011ad0 con 0x55a1d00ed4d0 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.294+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1542082900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 msgr2=0x55a1d04b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.294+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b47e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800bd40 tx=0x7f1e4800f730 comp rx=0 tx=0).stop 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1542082900 shutdown_connections 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b47e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1542082900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00e52f0 0x55a1d04b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1542082900 >> 172.21.15.42:0/1542082900 conn(0x55a1d0102300 msgr2=0x55a1d04baea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1542082900 shutdown_connections 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1542082900 wait complete. 2024-05-27T10:19:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 -- start start 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.295+0000 7f1e6d59c740 1 Processor -- start 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 0x55a1d04b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601b43a0 con 0x55a1d00e52f0 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601b5810 con 0x55a1d00ed4d0 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601b4a70 con 0x55a1d010e1b0 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e6d59c740 10 client.0 _ll_register_callbacks cb 0x7f1e580095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e5f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 0x55a1d04b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e5f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 0x55a1d04b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41300/0 (socket says 172.21.15.42:41300) 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.296+0000 7f1e5f400640 1 -- 172.21.15.42:0/1515056820 learned_addr learned my addr 172.21.15.42:0/1515056820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5fe00640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5f400640 1 -- 172.21.15.42:0/1515056820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 msgr2=0x55a1d04b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5f400640 1 -- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00ed4d0 msgr2=0x55a1d04b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5f400640 1 -- 172.21.15.42:0/1515056820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1d04c1fd0 con 0x55a1d00e52f0 2024-05-27T10:19:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e5f400640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 0x55a1d04b4b60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e500ca130 tx=0x7f1e50100670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6047314242febcc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.297+0000 7f1e4d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e4d600640 1 -- 172.21.15.42:0/1515056820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50049030 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e4d600640 1 -- 172.21.15.42:0/1515056820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1e501050b0 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e480037f0 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e4d600640 1 -- 172.21.15.42:0/1515056820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e500ca8a0 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e6d59c740 10 client.15417 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1d04c32c0 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.298+0000 7f1e4d600640 1 -- 172.21.15.42:0/1515056820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e480037f0 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e4d600640 1 -- 172.21.15.42:0/1515056820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e500ce130 con 0x55a1d00e52f0 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e6d59c740 10 client.15417 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e6d59c740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e6d59c740 20 client.15417 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00a21c0 0x55a1d0537160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e6d59c740 4 client.15417 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1d041d2c0 con 0x55a1d00a21c0 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.299+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00a21c0 0x55a1d0537160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.300+0000 7f1e5ea00640 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00a21c0 0x55a1d0537160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01e437045447fce server_cookie=c999e6dc87a2f085 in_seq=0 out_seq=0 2024-05-27T10:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.300+0000 7f1e4d600640 10 client.15417 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:32.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:31 smithi143 ceph-mon[35545]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T10:19:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.321+0000 7f1e4d600640 1 -- 172.21.15.42:0/1515056820 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a1d041d2c0 con 0x55a1d00a21c0 2024-05-27T10:19:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.321+0000 7f1e4d600640 10 client.15417 handle_command_reply: tid=0 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.321+0000 7f1e6d59c740 1 client.15417 shutdown 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.321+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 msgr2=0x55a1d04b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.321+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 0x55a1d04b4b60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e500ca130 tx=0x7f1e50100670 comp rx=0 tx=0).stop 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 shutdown_connections 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a1d00a21c0 0x55a1d0537160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1d010e1b0 0x55a1d04b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a1d00ed4d0 0x55a1d04b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 --2- 172.21.15.42:0/1515056820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1d00e52f0 0x55a1d04b4b60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 >> 172.21.15.42:0/1515056820 conn(0x55a1d0102300 msgr2=0x55a1d04bae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e4d600640 0 client.15417 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 shutdown_connections 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 1 -- 172.21.15.42:0/1515056820 wait complete. 2024-05-27T10:19:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.322+0000 7f1e6d59c740 20 client.15417 trim_cache size 0 max 16384 2024-05-27T10:19:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:19:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:19:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1e440b2af0 msgr2=0x7f1e440b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1e440b2af0 0x7f1e440b5000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55a1d04adbd0 tx=0x7f1e58009700 comp rx=0 tx=0).stop 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 msgr2=0x7f1e601a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e601a6860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1e5002a910 tx=0x7f1e50003c20 comp rx=0 tx=0).stop 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 shutdown_connections 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1e440b2af0 0x7f1e440b5000 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e60114a70 0x7f1e601a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e60112110 0x7f1e601a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 --2- 172.21.15.42:0/74527511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e60076590 0x7f1e601a6860 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.323+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 >> 172.21.15.42:0/74527511 conn(0x7f1e600ff170 msgr2=0x7f1e600ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.324+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 shutdown_connections 2024-05-27T10:19:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:32.324+0000 7f1e65000640 1 -- 172.21.15.42:0/74527511 wait complete. 2024-05-27T10:19:32.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:31 smithi042 ceph-mon[31661]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T10:19:33.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 -- 172.21.15.42:0/4065280927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f64108ef0 msgr2=0x7f4f64109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:33.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 --2- 172.21.15.42:0/4065280927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f64108ef0 0x7f4f64109300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4f500099c0 tx=0x7f4f50009cd0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 -- 172.21.15.42:0/4065280927 shutdown_connections 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 --2- 172.21.15.42:0/4065280927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f64109840 0x7f4f641111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 --2- 172.21.15.42:0/4065280927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f641045c0 0x7f4f641049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 --2- 172.21.15.42:0/4065280927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f64108ef0 0x7f4f64109300 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 -- 172.21.15.42:0/4065280927 >> 172.21.15.42:0/4065280927 conn(0x7f4f640ffe90 msgr2=0x7f4f641022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 -- 172.21.15.42:0/4065280927 shutdown_connections 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.428+0000 7f4f6c000700 1 -- 172.21.15.42:0/4065280927 wait complete. 2024-05-27T10:19:33.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.430+0000 7f4f6c000700 1 Processor -- start 2024-05-27T10:19:33.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.430+0000 7f4f6c000700 1 -- start start 2024-05-27T10:19:33.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6c000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 0x7f4f6419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:33.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6c000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f64108ef0 0x7f4f6419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:33.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6c000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f64109840 0x7f4f6419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:33.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6c000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64114300 con 0x7f4f641045c0 2024-05-27T10:19:33.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6c000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64114180 con 0x7f4f64109840 2024-05-27T10:19:33.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6c000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64114480 con 0x7f4f64108ef0 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f69600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 0x7f4f6419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f69600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 0x7f4f6419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45780/0 (socket says 172.21.15.42:45780) 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f69600700 1 -- 172.21.15.42:0/3514866091 learned_addr learned my addr 172.21.15.42:0/3514866091 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f68c00700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f64108ef0 0x7f4f6419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f6a000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f64109840 0x7f4f6419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f69600700 1 -- 172.21.15.42:0/3514866091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f64108ef0 msgr2=0x7f4f6419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:33.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f69600700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f64108ef0 0x7f4f6419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.431+0000 7f4f69600700 1 -- 172.21.15.42:0/3514866091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f64109840 msgr2=0x7f4f6419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f69600700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f64109840 0x7f4f6419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f69600700 1 -- 172.21.15.42:0/3514866091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6419fbe0 con 0x7f4f641045c0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f69600700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 0x7f4f6419ad10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800d950 tx=0x7f4f5800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58016070 con 0x7f4f641045c0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f58010460 con 0x7f4f641045c0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800f620 con 0x7f4f641045c0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6419fed0 con 0x7f4f641045c0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.432+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641a77f0 con 0x7f4f641045c0 2024-05-27T10:19:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.434+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f640512b0 con 0x7f4f641045c0 2024-05-27T10:19:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.437+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f4f58019070 con 0x7f4f641045c0 2024-05-27T10:19:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.438+0000 7f4f62000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b2b90 0x7f4f4c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.438+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4f580e2440 con 0x7f4f641045c0 2024-05-27T10:19:33.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.438+0000 7f4f68c00700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b2b90 0x7f4f4c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.439+0000 7f4f68c00700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b2b90 0x7f4f4c0b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009990 tx=0x7f4f50005940 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:33.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.441+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f580a9c20 con 0x7f4f641045c0 2024-05-27T10:19:33.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.684+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f6402e5e0 con 0x7f4f641045c0 2024-05-27T10:19:33.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.686+0000 7f4f62000700 1 -- 172.21.15.42:0/3514866091 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22940 (secure 0 0 0) 0x7f4f580a9520 con 0x7f4f641045c0 2024-05-27T10:19:33.687 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:33.687 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":82,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:18:42.838286+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1953650070":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2667703980":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2970531755":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/1310078254":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:19:33.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.689+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b2b90 msgr2=0x7f4f4c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:33.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.689+0000 7f4f6c000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b2b90 0x7f4f4c0b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009990 tx=0x7f4f50005940 comp rx=0 tx=0).stop 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 msgr2=0x7f4f6419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 0x7f4f6419ad10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800d950 tx=0x7f4f5800dd10 comp rx=0 tx=0).stop 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 shutdown_connections 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b2b90 0x7f4f4c0b50a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f64109840 0x7f4f6419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f641045c0 0x7f4f6419ad10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 --2- 172.21.15.42:0/3514866091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f64108ef0 0x7f4f6419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:33.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 >> 172.21.15.42:0/3514866091 conn(0x7f4f640ffe90 msgr2=0x7f4f641022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:33.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 shutdown_connections 2024-05-27T10:19:33.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:33.690+0000 7f4f6c000700 1 -- 172.21.15.42:0/3514866091 wait complete. 2024-05-27T10:19:34.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:33 smithi104 ceph-mon[34812]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:34.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:33 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3514866091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:19:34.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:33 smithi143 ceph-mon[35545]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:34.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:33 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3514866091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:19:34.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:33 smithi042 ceph-mon[31661]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T10:19:34.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:33 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3514866091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:19:34.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-27T10:19:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:36.265 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:35 smithi042 ceph-mon[31661]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-27T10:19:36.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:35 smithi143 ceph-mon[35545]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-27T10:19:36.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:35 smithi104 ceph-mon[34812]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-27T10:19:38.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:37 smithi143 ceph-mon[35545]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:38.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:37 smithi042 ceph-mon[31661]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:38.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:37 smithi104 ceph-mon[34812]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:38.976 INFO:teuthology.orchestra.run.smithi042.stdout:{"run_dir":"/var/run/ceph"} 2024-05-27T10:19:38.976 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-27T10:19:38.977 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 2274 2024-05-27T10:19:38.977 DEBUG:tasks.quiescer:Yielding 2024-05-27T10:19:38.977 INFO:teuthology.run_tasks:Running task workunit... 2024-05-27T10:19:38.992 INFO:tasks.workunit:Pulling workunits from ref a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:19:38.992 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-27T10:19:38.992 INFO:tasks.workunit:timeout=3h 2024-05-27T10:19:38.992 INFO:tasks.workunit:cleanup=True 2024-05-27T10:19:38.993 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-27T10:19:38.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:19:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-27T10:19:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-27T10:19:39.048 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-05-27T10:19:39.048 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T10:19:39.048 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-27T10:19:39.049 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-27 10:14:55.153219992 +0000 2024-05-27T10:19:39.049 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-27 10:14:55.153219992 +0000 2024-05-27T10:19:39.049 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-27 10:18:42.022145687 +0000 2024-05-27T10:19:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-05-27T10:19:39.049 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-27T10:19:39.049 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-27T10:19:39.188 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T10:19:39.245 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-27T10:19:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:40.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:39 smithi143 ceph-mon[35545]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:40.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:39 smithi042 ceph-mon[31661]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:40.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:39 smithi104 ceph-mon[34812]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:41.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:19:41.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 -- 172.21.15.42:0/1647435610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 msgr2=0x7fbf0c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:41.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1647435610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c1049f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf000099c0 tx=0x7fbf00009cd0 comp rx=0 tx=0).stop 2024-05-27T10:19:41.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 -- 172.21.15.42:0/1647435610 shutdown_connections 2024-05-27T10:19:41.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1647435610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c109860 0x7fbf0c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:41.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1647435610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c1049f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:41.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1647435610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c108f10 0x7fbf0c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:41.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 -- 172.21.15.42:0/1647435610 >> 172.21.15.42:0/1647435610 conn(0x7fbf0c100030 msgr2=0x7fbf0c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:41.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 -- 172.21.15.42:0/1647435610 shutdown_connections 2024-05-27T10:19:41.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.955+0000 7fbf13c00700 1 -- 172.21.15.42:0/1647435610 wait complete. 2024-05-27T10:19:41.956 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:41.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.957+0000 7fbf13c00700 1 Processor -- start 2024-05-27T10:19:41.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.957+0000 7fbf13c00700 1 -- start start 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf13c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf13c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c108f10 0x7fbf0c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf13c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c109860 0x7fbf0c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf13c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c114340 con 0x7fbf0c1045e0 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf13c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1141c0 con 0x7fbf0c108f10 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf13c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1144c0 con 0x7fbf0c109860 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf11200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf11200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45800/0 (socket says 172.21.15.42:45800) 2024-05-27T10:19:41.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf11200700 1 -- 172.21.15.42:0/1337463858 learned_addr learned my addr 172.21.15.42:0/1337463858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:41.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf11c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c109860 0x7fbf0c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:41.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.958+0000 7fbf0be00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c108f10 0x7fbf0c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:41.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf11200700 1 -- 172.21.15.42:0/1337463858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c109860 msgr2=0x7fbf0c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf11200700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c109860 0x7fbf0c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf11200700 1 -- 172.21.15.42:0/1337463858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c108f10 msgr2=0x7fbf0c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf11200700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c108f10 0x7fbf0c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf11200700 1 -- 172.21.15.42:0/1337463858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c19fc20 con 0x7fbf0c1045e0 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf0be00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c108f10 0x7fbf0c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf11200700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c19ad50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf00000c00 tx=0x7fbf0000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000f940 con 0x7fbf0c1045e0 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf00004830 con 0x7fbf0c1045e0 2024-05-27T10:19:41.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.959+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c19feb0 con 0x7fbf0c1045e0 2024-05-27T10:19:41.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.960+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000054b0 con 0x7fbf0c1045e0 2024-05-27T10:19:41.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.960+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1a0330 con 0x7fbf0c1045e0 2024-05-27T10:19:41.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.961+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0c0512b0 con 0x7fbf0c1045e0 2024-05-27T10:19:41.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.964+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fbf00029060 con 0x7fbf0c1045e0 2024-05-27T10:19:41.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.965+0000 7fbf09600700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbef40b2b10 0x7fbef40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:41.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.966+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbf000e36f0 con 0x7fbf0c1045e0 2024-05-27T10:19:41.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.966+0000 7fbf0be00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbef40b2b10 0x7fbef40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:41.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.967+0000 7fbf0be00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbef40b2b10 0x7fbef40b5020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005950 tx=0x7fbef80058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:41.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:41.968+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf00013070 con 0x7fbf0c1045e0 2024-05-27T10:19:42.118 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:41 smithi042 ceph-mon[31661]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-27T10:19:42.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.281+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf0c02e5e0 con 0x7fbf0c1045e0 2024-05-27T10:19:42.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.282+0000 7fbf09600700 1 -- 172.21.15.42:0/1337463858 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fbf000aaf00 con 0x7fbf0c1045e0 2024-05-27T10:19:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:42.283 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:19:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbef40b2b10 msgr2=0x7fbef40b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbef40b2b10 0x7fbef40b5020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005950 tx=0x7fbef80058e0 comp rx=0 tx=0).stop 2024-05-27T10:19:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 msgr2=0x7fbf0c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c19ad50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf00000c00 tx=0x7fbf0000f510 comp rx=0 tx=0).stop 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 shutdown_connections 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbef40b2b10 0x7fbef40b5020 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c109860 0x7fbf0c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c1045e0 0x7fbf0c19ad50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 --2- 172.21.15.42:0/1337463858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf0c108f10 0x7fbf0c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.285+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 >> 172.21.15.42:0/1337463858 conn(0x7fbf0c100030 msgr2=0x7fbf0c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.286+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 shutdown_connections 2024-05-27T10:19:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:42.286+0000 7fbf13c00700 1 -- 172.21.15.42:0/1337463858 wait complete. 2024-05-27T10:19:42.287 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:19:42.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:41 smithi143 ceph-mon[35545]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-27T10:19:42.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:41 smithi104 ceph-mon[34812]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-27T10:19:43.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:42 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1337463858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:43.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:42 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1337463858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:43.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:42 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1337463858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:43.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T10:14:18.127208+0000', 'modified': '2024-05-27T10:18:42.569936+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34499, 'mds_1': 34466, 'mds_2': 34442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/4166809554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 4166809554}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 4166809554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6839/517039305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 517039305}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 517039305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34421': {'gid': 34421, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/3150585498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 3150585498}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 3150585498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.104:6835/3501163811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 3501163811}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 3501163811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34466': {'gid': 34466, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.104:6837/219118322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 219118322}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 219118322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34499': {'gid': 34499, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.104:6839/3864142797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3864142797}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3864142797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34499, 'qdb_cluster': [34442, 34466, 34499]} 2024-05-27T10:19:43.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-27T10:19:43.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T10:19:44.027 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:43 smithi042 ceph-mon[31661]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:44.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:43 smithi143 ceph-mon[35545]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:44.433 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:44.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:43 smithi104 ceph-mon[34812]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T10:19:45.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 -- 172.21.15.42:0/2946115285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac105e20 msgr2=0x7f0dac10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 --2- 172.21.15.42:0/2946115285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac105e20 0x7f0dac10e370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009990 tx=0x7f0d98009ca0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 -- 172.21.15.42:0/2946115285 shutdown_connections 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 --2- 172.21.15.42:0/2946115285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dac10e900 0x7f0dac110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 --2- 172.21.15.42:0/2946115285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac1054d0 0x7f0dac1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 --2- 172.21.15.42:0/2946115285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac105e20 0x7f0dac10e370 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.226+0000 7f0db3000700 1 -- 172.21.15.42:0/2946115285 >> 172.21.15.42:0/2946115285 conn(0x7f0dac0fef20 msgr2=0x7f0dac101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.227+0000 7f0db3000700 1 -- 172.21.15.42:0/2946115285 shutdown_connections 2024-05-27T10:19:45.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.227+0000 7f0db3000700 1 -- 172.21.15.42:0/2946115285 wait complete. 2024-05-27T10:19:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 Processor -- start 2024-05-27T10:19:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 -- start start 2024-05-27T10:19:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 0x7f0dac1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dac105e20 0x7f0dac1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac10e900 0x7f0dac19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac114310 con 0x7f0dac1054d0 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac114190 con 0x7f0dac10e900 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.228+0000 7f0db3000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac114490 con 0x7f0dac105e20 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 0x7f0dac1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 0x7f0dac1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60806/0 (socket says 172.21.15.42:60806) 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 -- 172.21.15.42:0/3510977927 learned_addr learned my addr 172.21.15.42:0/3510977927 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dab400700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dac105e20 0x7f0dac1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0db1000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac10e900 0x7f0dac19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 -- 172.21.15.42:0/3510977927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dac105e20 msgr2=0x7f0dac1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dac105e20 0x7f0dac1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 -- 172.21.15.42:0/3510977927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac10e900 msgr2=0x7f0dac19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac10e900 0x7f0dac19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0dabe00700 1 -- 172.21.15.42:0/3510977927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19b260 con 0x7f0dac1054d0 2024-05-27T10:19:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.229+0000 7f0db1000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac10e900 0x7f0dac19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.230+0000 7f0dabe00700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 0x7f0dac1a09a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e9a0 tx=0x7f0d9c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.230+0000 7f0da8c00700 1 -- 172.21.15.42:0/3510977927 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00cbd0 con 0x7f0dac1054d0 2024-05-27T10:19:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.230+0000 7f0da8c00700 1 -- 172.21.15.42:0/3510977927 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d9c00cd70 con 0x7f0dac1054d0 2024-05-27T10:19:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.230+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19b550 con 0x7f0dac1054d0 2024-05-27T10:19:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.230+0000 7f0da8c00700 1 -- 172.21.15.42:0/3510977927 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c018680 con 0x7f0dac1054d0 2024-05-27T10:19:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.230+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a7750 con 0x7f0dac1054d0 2024-05-27T10:19:45.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.234+0000 7f0da8c00700 1 -- 172.21.15.42:0/3510977927 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0d9c010950 con 0x7f0dac1054d0 2024-05-27T10:19:45.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.235+0000 7f0dc1c0fb80 1 Processor -- start 2024-05-27T10:19:45.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.235+0000 7f0dc1c0fb80 1 -- start start 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0da8c00700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d900b2b90 0x7f0d900b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc417bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc4175c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ddc409fe40 con 0x55ddc417c0e0 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ddc408f9e0 con 0x55ddc41761c0 2024-05-27T10:19:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ddc4093a20 con 0x55ddc3ff2e40 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0da8c00700 1 -- 172.21.15.42:0/3510977927 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0d9c0e2b90 con 0x7f0dac1054d0 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dab400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc4175c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0db1000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d900b2b90 0x7f0d900b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dab400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc4175c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60820/0 (socket says 172.21.15.42:60820) 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dab400700 1 -- 172.21.15.42:0/1652572408 learned_addr learned my addr 172.21.15.42:0/1652572408 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.236+0000 7f0dabe00700 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc417bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0db1000700 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dab400700 1 -- 172.21.15.42:0/1652572408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 msgr2=0x55ddc417bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dab400700 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc417bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dab400700 1 -- 172.21.15.42:0/1652572408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 msgr2=0x55ddc41765d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dab400700 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41765d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dab400700 1 -- 172.21.15.42:0/1652572408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddc4058420 con 0x55ddc417c0e0 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dabe00700 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc417bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0dab400700 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc4175c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d980096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f544c583170c377 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.237+0000 7f0db1000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d900b2b90 0x7f0d900b50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009710 tx=0x7f0da4009f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.238+0000 7f0da1600700 1 -- 172.21.15.42:0/1652572408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98005550 con 0x55ddc417c0e0 2024-05-27T10:19:45.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.238+0000 7f0da1600700 1 -- 172.21.15.42:0/1652572408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d98003b10 con 0x55ddc417c0e0 2024-05-27T10:19:45.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.238+0000 7f0da1600700 1 -- 172.21.15.42:0/1652572408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800bc10 con 0x55ddc417c0e0 2024-05-27T10:19:45.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/1652572408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 msgr2=0x55ddc4175c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc4175c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d980096a0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/1652572408 shutdown_connections 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc417bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/1652572408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc4175c80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/1652572408 >> 172.21.15.42:0/1652572408 conn(0x55ddc4052fc0 msgr2=0x55ddc416c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/1652572408 shutdown_connections 2024-05-27T10:19:45.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.239+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/1652572408 wait complete. 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.240+0000 7f0dc1c0fb80 1 -- start start 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.240+0000 7f0dc1c0fb80 1 Processor -- start 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc417f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc41761c0 0x55ddc417fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc417c0e0 0x55ddc4180390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ddc3f95fa0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ddc4030760 con 0x55ddc417c0e0 2024-05-27T10:19:45.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ddc4182bc0 con 0x55ddc41761c0 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dc1c0fb80 10 client.0 _ll_register_callbacks cb 0x55ddc4039c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc417f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc417f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60830/0 (socket says 172.21.15.42:60830) 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dabe00700 1 -- 172.21.15.42:0/4240405577 learned_addr learned my addr 172.21.15.42:0/4240405577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0dab400700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc41761c0 0x55ddc417fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.241+0000 7f0db1000700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc417c0e0 0x55ddc4180390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dabe00700 1 -- 172.21.15.42:0/4240405577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc41761c0 msgr2=0x55ddc417fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dabe00700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc41761c0 0x55ddc417fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dabe00700 1 -- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc417c0e0 msgr2=0x55ddc4180390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dabe00700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc417c0e0 0x55ddc4180390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dabe00700 1 -- 172.21.15.42:0/4240405577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc3fdf8d0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dabe00700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc417f910 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0a9ba0 tx=0x7f0d9c0bd940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf773951faceed00 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0d96000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0d96000700 1 -- 172.21.15.42:0/4240405577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0a53a0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0d96000700 1 -- 172.21.15.42:0/4240405577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0d9c0b27f0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.242+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d98003b10 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0d96000700 1 -- 172.21.15.42:0/4240405577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c0c2020 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 10 client.15441 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddc4060060 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0d96000700 1 -- 172.21.15.42:0/4240405577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d98003b10 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0d96000700 1 -- 172.21.15.42:0/4240405577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0d9c0a8e70 con 0x55ddc3ff2e40 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:45.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 20 client.15441 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:45.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc418fab0 0x55ddc4191f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 4 client.15441 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:45.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.243+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddc40a1d90 con 0x55ddc418fab0 2024-05-27T10:19:45.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.244+0000 7f0dab400700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc418fab0 0x55ddc4191f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.245+0000 7f0dab400700 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc418fab0 0x55ddc4191f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7b0b3506506e8b server_cookie=be1b6e8efd83f287 in_seq=0 out_seq=0 2024-05-27T10:19:45.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.245+0000 7f0d96000700 10 client.15441 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0d96000700 1 -- 172.21.15.42:0/4240405577 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ddc40a1d90 con 0x55ddc418fab0 2024-05-27T10:19:45.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0d96000700 10 client.15441 handle_command_reply: tid=0 2024-05-27T10:19:45.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 client.15441 shutdown 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 msgr2=0x55ddc417f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc417f910 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0a9ba0 tx=0x7f0d9c0bd940 comp rx=0 tx=0).stop 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 shutdown_connections 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc418fab0 0x55ddc4191f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc417c0e0 0x55ddc4180390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc417f910 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0d96000700 0 client.15441 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/4240405577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc41761c0 0x55ddc417fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.247+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 >> 172.21.15.42:0/4240405577 conn(0x55ddc4052fc0 msgr2=0x55ddc416c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.248+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 shutdown_connections 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.248+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/4240405577 wait complete. 2024-05-27T10:19:45.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.248+0000 7f0dc1c0fb80 20 client.15441 trim_cache size 0 max 16384 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.271+0000 7f0dc1c0fb80 1 Processor -- start 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 -- start start 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc4179f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc418bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc417f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1af1e0 con 0x55ddc417c0e0 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1aca70 con 0x55ddc41761c0 2024-05-27T10:19:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.272+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1aa8e0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc4179f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0db1000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc417f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc4179f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:42968/0 (socket says 172.21.15.42:42968) 2024-05-27T10:19:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0dabe00700 1 -- 172.21.15.42:0/2056137268 learned_addr learned my addr 172.21.15.42:0/2056137268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0dab400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc418bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0db1000700 1 -- 172.21.15.42:0/2056137268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 msgr2=0x55ddc4179f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0db1000700 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc4179f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0db1000700 1 -- 172.21.15.42:0/2056137268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 msgr2=0x55ddc418bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0db1000700 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc418bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.273+0000 7f0db1000700 1 -- 172.21.15.42:0/2056137268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddc417de80 con 0x55ddc417c0e0 2024-05-27T10:19:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.274+0000 7f0db1000700 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc417f6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0da4007d30 tx=0x7f0da401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322f36132a521144 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.274+0000 7f0da0c00700 1 -- 172.21.15.42:0/2056137268 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4012030 con 0x55ddc417c0e0 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.274+0000 7f0da0c00700 1 -- 172.21.15.42:0/2056137268 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da4013040 con 0x55ddc417c0e0 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.274+0000 7f0da0c00700 1 -- 172.21.15.42:0/2056137268 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4025760 con 0x55ddc417c0e0 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2056137268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 msgr2=0x55ddc417f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc417f6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0da4007d30 tx=0x7f0da401b000 comp rx=0 tx=0).stop 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2056137268 shutdown_connections 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc417c0e0 0x55ddc417f6f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc3ff2e40 0x55ddc4179f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2056137268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc418bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2056137268 >> 172.21.15.42:0/2056137268 conn(0x55ddc4052fc0 msgr2=0x55ddc4170ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2056137268 shutdown_connections 2024-05-27T10:19:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.275+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2056137268 wait complete. 2024-05-27T10:19:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.276+0000 7f0dc1c0fb80 1 -- start start 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.276+0000 7f0dc1c0fb80 1 Processor -- start 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.276+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc4197390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.276+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dc1c0fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc417c0e0 0x55ddc4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1aaf30 con 0x55ddc3ff2e40 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ac3a0 con 0x55ddc41761c0 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dc1c0fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ab600 con 0x55ddc417c0e0 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dc1c0fb80 10 client.0 _ll_register_callbacks cb 0x55ddc4039c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc4197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc4197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60846/0 (socket says 172.21.15.42:60846) 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 -- 172.21.15.42:0/2809557507 learned_addr learned my addr 172.21.15.42:0/2809557507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dab400700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0db1000700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc417c0e0 0x55ddc4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 -- 172.21.15.42:0/2809557507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc417c0e0 msgr2=0x55ddc4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc417c0e0 0x55ddc4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 -- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 msgr2=0x55ddc41978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.277+0000 7f0dabe00700 1 -- 172.21.15.42:0/2809557507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc41f96e0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0dabe00700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc4197390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0a61b0 tx=0x7f0d9c00efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43e3e38d3d79b7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0d96000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0d96000700 1 -- 172.21.15.42:0/2809557507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b7060 con 0x55ddc3ff2e40 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0d96000700 1 -- 172.21.15.42:0/2809557507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0d9c0bdb80 con 0x55ddc3ff2e40 2024-05-27T10:19:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da4013040 con 0x55ddc3ff2e40 2024-05-27T10:19:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0d96000700 1 -- 172.21.15.42:0/2809557507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c0a5ad0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0dc1c0fb80 10 client.15447 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.278+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddc41fa8f0 con 0x55ddc3ff2e40 2024-05-27T10:19:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0d96000700 1 -- 172.21.15.42:0/2809557507 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da4013040 con 0x55ddc3ff2e40 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0d96000700 1 -- 172.21.15.42:0/2809557507 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0d9c010430 con 0x55ddc3ff2e40 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0dc1c0fb80 10 client.15447 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0dc1c0fb80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0dc1c0fb80 20 client.15447 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc41faea0 0x55ddc41fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0dc1c0fb80 4 client.15447 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.279+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ddc409b570 con 0x55ddc41faea0 2024-05-27T10:19:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.280+0000 7f0dab400700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc41faea0 0x55ddc41fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.281+0000 7f0d96000700 10 client.15447 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.281+0000 7f0dab400700 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc41faea0 0x55ddc41fd2c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebdd9b7e94a67f6 server_cookie=ce45dd72e0874051 in_seq=0 out_seq=0 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.281+0000 7f0d96000700 1 -- 172.21.15.42:0/2809557507 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ddc409b570 con 0x55ddc41faea0 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.281+0000 7f0d96000700 10 client.15447 handle_command_reply: tid=0 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.281+0000 7f0dc1c0fb80 1 client.15447 shutdown 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 msgr2=0x55ddc4197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc4197390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0a61b0 tx=0x7f0d9c00efa0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 shutdown_connections 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ddc41faea0 0x55ddc41fd2c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddc417c0e0 0x55ddc4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddc3ff2e40 0x55ddc4197390 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 --2- 172.21.15.42:0/2809557507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddc41761c0 0x55ddc41978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 >> 172.21.15.42:0/2809557507 conn(0x55ddc4052fc0 msgr2=0x55ddc417e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0d96000700 0 client.15447 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 shutdown_connections 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 1 -- 172.21.15.42:0/2809557507 wait complete. 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.282+0000 7f0dc1c0fb80 20 client.15447 trim_cache size 0 max 16384 2024-05-27T10:19:45.284 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:19:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-05-27T10:19:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-05-27T10:19:45.285 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:19:45.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d900b2b90 msgr2=0x7f0d900b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d900b2b90 0x7f0d900b50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009710 tx=0x7f0da4009f90 comp rx=0 tx=0).stop 2024-05-27T10:19:45.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 msgr2=0x7f0dac1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 0x7f0dac1a09a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e9a0 tx=0x7f0d9c00ed60 comp rx=0 tx=0).stop 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 shutdown_connections 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d900b2b90 0x7f0d900b50a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dac10e900 0x7f0dac19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dac1054d0 0x7f0dac1a09a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 --2- 172.21.15.42:0/3510977927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dac105e20 0x7f0dac1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 >> 172.21.15.42:0/3510977927 conn(0x7f0dac0fef20 msgr2=0x7f0dac0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 shutdown_connections 2024-05-27T10:19:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:45.283+0000 7f0db3000700 1 -- 172.21.15.42:0/3510977927 wait complete. 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.546+0000 7f463b800640 1 -- 172.21.15.42:0/2826950623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463410bea0 msgr2=0x7f4634105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.546+0000 7f463b800640 1 --2- 172.21.15.42:0/2826950623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463410bea0 0x7f4634105a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4628009900 tx=0x7f462802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 -- 172.21.15.42:0/2826950623 shutdown_connections 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 --2- 172.21.15.42:0/2826950623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46341065f0 0x7f46341129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 --2- 172.21.15.42:0/2826950623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463410c340 0x7f4634105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 --2- 172.21.15.42:0/2826950623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463410bea0 0x7f4634105a40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 -- 172.21.15.42:0/2826950623 >> 172.21.15.42:0/2826950623 conn(0x7f46340771f0 msgr2=0x7f4634077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 -- 172.21.15.42:0/2826950623 shutdown_connections 2024-05-27T10:19:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.547+0000 7f463b800640 1 -- 172.21.15.42:0/2826950623 wait complete. 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.548+0000 7f463b800640 1 Processor -- start 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.548+0000 7f463b800640 1 -- start start 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f463b800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 0x7f46341a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f463b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463410bea0 0x7f46341a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f463b800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f463410c340 0x7f463419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f463b800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4634115f80 con 0x7f46341065f0 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f463b800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4634115e00 con 0x7f463410c340 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f463b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4634116100 con 0x7f463410bea0 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f4638e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 0x7f46341a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f4638e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 0x7f46341a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60872/0 (socket says 172.21.15.42:60872) 2024-05-27T10:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f4638e00640 1 -- 172.21.15.42:0/1595344996 learned_addr learned my addr 172.21.15.42:0/1595344996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f4633e00640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463410bea0 0x7f46341a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f4639800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f463410c340 0x7f463419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.549+0000 7f4638e00640 1 -- 172.21.15.42:0/1595344996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463410bea0 msgr2=0x7f46341a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4638e00640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463410bea0 0x7f46341a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4638e00640 1 -- 172.21.15.42:0/1595344996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f463410c340 msgr2=0x7f463419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4638e00640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f463410c340 0x7f463419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4638e00640 1 -- 172.21.15.42:0/1595344996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463419ce70 con 0x7f46341065f0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4638e00640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 0x7f46341a25d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462802a910 tx=0x7f4628003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4631600640 1 -- 172.21.15.42:0/1595344996 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462803f070 con 0x7f46341065f0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463419d100 con 0x7f46341065f0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4631600640 1 -- 172.21.15.42:0/1595344996 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4628003e10 con 0x7f46341065f0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.550+0000 7f4631600640 1 -- 172.21.15.42:0/1595344996 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628035030 con 0x7f46341065f0 2024-05-27T10:19:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.551+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463419d5f0 con 0x7f46341065f0 2024-05-27T10:19:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.554+0000 7f46427e5740 1 Processor -- start 2024-05-27T10:19:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.554+0000 7f46427e5740 1 -- start start 2024-05-27T10:19:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f46427e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389594860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f46427e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x565389594da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4631600640 1 -- 172.21.15.42:0/1595344996 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f4628042070 con 0x7f46341065f0 2024-05-27T10:19:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f46427e5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f46427e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565389505b90 con 0x5653891f21b0 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f46427e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5653894f5730 con 0x5653891c92f0 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f46427e5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5653894f9770 con 0x5653891d14d0 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389594860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389594860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60880/0 (socket says 172.21.15.42:60880) 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 -- 172.21.15.42:0/734596750 learned_addr learned my addr 172.21.15.42:0/734596750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4633e00640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4638e00640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x565389594da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 -- 172.21.15.42:0/734596750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 msgr2=0x5653895952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 -- 172.21.15.42:0/734596750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 msgr2=0x565389594da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x565389594da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4639800640 1 -- 172.21.15.42:0/734596750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538948e8b0 con 0x5653891f21b0 2024-05-27T10:19:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4633e00640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.555+0000 7f4638e00640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x565389594da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.556+0000 7f4639800640 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389594860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f462c009df0 tx=0x7f462c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a09ed33fb4d8d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.556+0000 7f4626000640 1 -- 172.21.15.42:0/734596750 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c003970 con 0x5653891f21b0 2024-05-27T10:19:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.556+0000 7f4626000640 1 -- 172.21.15.42:0/734596750 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f462c004030 con 0x5653891f21b0 2024-05-27T10:19:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.556+0000 7f4626000640 1 -- 172.21.15.42:0/734596750 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c011680 con 0x5653891f21b0 2024-05-27T10:19:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f4631600640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f461c0b2a70 0x7f461c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f4631600640 1 -- 172.21.15.42:0/1595344996 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f46281030e0 con 0x7f46341065f0 2024-05-27T10:19:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f4638e00640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f461c0b2a70 0x7f461c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 -- 172.21.15.42:0/734596750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 msgr2=0x565389594860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389594860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f462c009df0 tx=0x7f462c00f730 comp rx=0 tx=0).stop 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 -- 172.21.15.42:0/734596750 shutdown_connections 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x565389594da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 --2- 172.21.15.42:0/734596750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389594860 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 -- 172.21.15.42:0/734596750 >> 172.21.15.42:0/734596750 conn(0x5653891e6300 msgr2=0x565389591010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 -- 172.21.15.42:0/734596750 shutdown_connections 2024-05-27T10:19:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.557+0000 7f46427e5740 1 -- 172.21.15.42:0/734596750 wait complete. 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.558+0000 7f4638e00640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f461c0b2a70 0x7f461c0b4f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4628100700 tx=0x7f4628005de0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.558+0000 7f46427e5740 1 -- start start 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.558+0000 7f46427e5740 1 Processor -- start 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891c92f0 0x56538959e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891d14d0 0x56538959eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x5653895aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5653891db790 con 0x5653891f21b0 2024-05-27T10:19:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56538949dbc0 con 0x5653891d14d0 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5653895a1990 con 0x5653891c92f0 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f46427e5740 10 client.0 _ll_register_callbacks cb 0x56538947b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f4639800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x5653895aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f4639800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x5653895aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60888/0 (socket says 172.21.15.42:60888) 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.559+0000 7f4639800640 1 -- 172.21.15.42:0/882875740 learned_addr learned my addr 172.21.15.42:0/882875740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891c92f0 0x56538959e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4633e00640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891d14d0 0x56538959eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 -- 172.21.15.42:0/882875740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891c92f0 msgr2=0x56538959e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891c92f0 0x56538959e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 -- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891d14d0 msgr2=0x56538959eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891d14d0 0x56538959eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 -- 172.21.15.42:0/882875740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56538920a210 con 0x5653891f21b0 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4633e00640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891d14d0 0x56538959eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f4639800640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x5653895aacc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f462c009b80 tx=0x7f462c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e80e9934904130 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.560+0000 7f461aa00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f461aa00640 1 -- 172.21.15.42:0/882875740 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c023040 con 0x5653891f21b0 2024-05-27T10:19:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f461aa00640 1 -- 172.21.15.42:0/882875740 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f462c011440 con 0x5653891f21b0 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462c004030 con 0x5653891f21b0 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f461aa00640 1 -- 172.21.15.42:0/882875740 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f462c011640 con 0x5653891f21b0 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f46427e5740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653892d6620 con 0x5653891f21b0 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.561+0000 7f461aa00640 1 -- 172.21.15.42:0/882875740 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f462c004030 con 0x5653891f21b0 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f461aa00640 1 -- 172.21.15.42:0/882875740 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f462c01a5a0 con 0x5653891f21b0 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f46427e5740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f46427e5740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f46427e5740 20 client.15459 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f46427e5740 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891b0d80 0x5653895b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f46427e5740 4 client.15459 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565389507ae0 con 0x5653891b0d80 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.562+0000 7f4633e00640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891b0d80 0x5653895b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.563+0000 7f4633e00640 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891b0d80 0x5653895b0a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca75e436d9e290e server_cookie=36a9bf3fa9efc3df in_seq=0 out_seq=0 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.564+0000 7f461aa00640 10 client.15459 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.565+0000 7f461aa00640 1 -- 172.21.15.42:0/882875740 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565389507ae0 con 0x5653891b0d80 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.565+0000 7f461aa00640 10 client.15459 handle_command_reply: tid=0 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.565+0000 7f46427e5740 1 client.15459 shutdown 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 msgr2=0x5653895aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x5653895aacc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f462c009b80 tx=0x7f462c009900 comp rx=0 tx=0).stop 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 shutdown_connections 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x5653895aacc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891d14d0 0x56538959eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891b0d80 0x5653895b0a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 --2- 172.21.15.42:0/882875740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891c92f0 0x56538959e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 >> 172.21.15.42:0/882875740 conn(0x5653891e6300 msgr2=0x565389594690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f461aa00640 0 client.15459 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 shutdown_connections 2024-05-27T10:19:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 1 -- 172.21.15.42:0/882875740 wait complete. 2024-05-27T10:19:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.566+0000 7f46427e5740 20 client.15459 trim_cache size 0 max 16384 2024-05-27T10:19:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 Processor -- start 2024-05-27T10:19:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 -- start start 2024-05-27T10:19:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x56538959e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x56538959ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b43d0 con 0x5653891f21b0 2024-05-27T10:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b1ca0 con 0x5653891c92f0 2024-05-27T10:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.583+0000 7f46427e5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46341afaf0 con 0x5653891d14d0 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x56538959ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x56538959ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60898/0 (socket says 172.21.15.42:60898) 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 -- 172.21.15.42:0/1008603798 learned_addr learned my addr 172.21.15.42:0/1008603798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4633e00640 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x56538959e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 -- 172.21.15.42:0/1008603798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 msgr2=0x56538959e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x56538959e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 -- 172.21.15.42:0/1008603798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 msgr2=0x5653895a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.584+0000 7f4639800640 1 -- 172.21.15.42:0/1008603798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565389591c00 con 0x5653891f21b0 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.585+0000 7f4639800640 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x56538959ee60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f462c009870 tx=0x7f462c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cc77dd96715aa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.585+0000 7f4625600640 1 -- 172.21.15.42:0/1008603798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c00fe40 con 0x5653891f21b0 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.585+0000 7f4625600640 1 -- 172.21.15.42:0/1008603798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f462c005b30 con 0x5653891f21b0 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.585+0000 7f4625600640 1 -- 172.21.15.42:0/1008603798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c011a40 con 0x5653891f21b0 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 -- 172.21.15.42:0/1008603798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 msgr2=0x56538959ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x56538959ee60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f462c009870 tx=0x7f462c01bf90 comp rx=0 tx=0).stop 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 -- 172.21.15.42:0/1008603798 shutdown_connections 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x56538959ee60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x56538959e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 --2- 172.21.15.42:0/1008603798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 -- 172.21.15.42:0/1008603798 >> 172.21.15.42:0/1008603798 conn(0x5653891e6300 msgr2=0x56538959e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 -- 172.21.15.42:0/1008603798 shutdown_connections 2024-05-27T10:19:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 -- 172.21.15.42:0/1008603798 wait complete. 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.586+0000 7f46427e5740 1 -- start start 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 Processor -- start 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389616fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b0160 con 0x5653891f21b0 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b15d0 con 0x5653891c92f0 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b0830 con 0x5653891d14d0 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.587+0000 7f46427e5740 10 client.0 _ll_register_callbacks cb 0x7f462c009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.588+0000 7f4633e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.588+0000 7f4633e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:43064/0 (socket says 172.21.15.42:43064) 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.588+0000 7f4633e00640 1 -- 172.21.15.42:0/2622705194 learned_addr learned my addr 172.21.15.42:0/2622705194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.588+0000 7f4639800640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389616fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.588+0000 7f4639800640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4633e00640 1 -- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 msgr2=0x5653895a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4633e00640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4633e00640 1 -- 172.21.15.42:0/2622705194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 msgr2=0x565389616fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4633e00640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389616fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4633e00640 1 -- 172.21.15.42:0/2622705194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565389617500 con 0x5653891d14d0 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4639800640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f4633e00640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895a4520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f463410c770 tx=0x7f462000de70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9628bb79bfb804d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.589+0000 7f461aa00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.590+0000 7f461aa00640 1 -- 172.21.15.42:0/2622705194 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620005220 con 0x5653891d14d0 2024-05-27T10:19:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.590+0000 7f461aa00640 1 -- 172.21.15.42:0/2622705194 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f46200053c0 con 0x5653891d14d0 2024-05-27T10:19:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.590+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462c005b30 con 0x5653891d14d0 2024-05-27T10:19:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.590+0000 7f461aa00640 1 -- 172.21.15.42:0/2622705194 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f462002a050 con 0x5653891d14d0 2024-05-27T10:19:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.590+0000 7f46427e5740 10 client.25189 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.590+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653896185f0 con 0x5653891d14d0 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f461aa00640 1 -- 172.21.15.42:0/2622705194 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f462c005b30 con 0x5653891d14d0 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f461aa00640 1 -- 172.21.15.42:0/2622705194 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f462001b4c0 con 0x5653891d14d0 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f46427e5740 10 client.25189 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f46427e5740 10 client.25189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f46427e5740 20 client.25189 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f46427e5740 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891861c0 0x56538961ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f46427e5740 4 client.25189 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:19:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.591+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653895012c0 con 0x5653891861c0 2024-05-27T10:19:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.592+0000 7f4639800640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891861c0 0x56538961ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.593+0000 7f461aa00640 10 client.25189 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.593+0000 7f4639800640 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891861c0 0x56538961ad30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ea4cbebb014af0 server_cookie=a3278a05651128fd in_seq=0 out_seq=0 2024-05-27T10:19:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f461aa00640 1 -- 172.21.15.42:0/2622705194 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5653895012c0 con 0x5653891861c0 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f461aa00640 10 client.25189 handle_command_reply: tid=0 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 client.25189 shutdown 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 msgr2=0x5653895a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895a4520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f463410c770 tx=0x7f462000de70 comp rx=0 tx=0).stop 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 shutdown_connections 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653891f21b0 0x565389616fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5653891861c0 0x56538961ad30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653891d14d0 0x5653895a4520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f46427e5740 1 --2- 172.21.15.42:0/2622705194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653891c92f0 0x5653895a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.615+0000 7f461aa00640 0 client.25189 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.616+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 >> 172.21.15.42:0/2622705194 conn(0x5653891e6300 msgr2=0x56538959ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.616+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 shutdown_connections 2024-05-27T10:19:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.616+0000 7f46427e5740 1 -- 172.21.15.42:0/2622705194 wait complete. 2024-05-27T10:19:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.616+0000 7f46427e5740 20 client.25189 trim_cache size 0 max 16384 2024-05-27T10:19:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:19:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:19:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f461c0b2a70 msgr2=0x7f461c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f461c0b2a70 0x7f461c0b4f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4628100700 tx=0x7f4628005de0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 msgr2=0x7f46341a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 0x7f46341a25d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462802a910 tx=0x7f4628003c20 comp rx=0 tx=0).stop 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 shutdown_connections 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f461c0b2a70 0x7f461c0b4f80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f463410c340 0x7f463419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463410bea0 0x7f46341a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 --2- 172.21.15.42:0/1595344996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46341065f0 0x7f46341a25d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 >> 172.21.15.42:0/1595344996 conn(0x7f46340771f0 msgr2=0x7f4634107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 shutdown_connections 2024-05-27T10:19:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:45.617+0000 7f463b800640 1 -- 172.21.15.42:0/1595344996 wait complete. 2024-05-27T10:19:46.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:45 smithi042 ceph-mon[31661]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-27T10:19:46.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:45 smithi143 ceph-mon[35545]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-27T10:19:46.318 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T10:19:46.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:19:46.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:45 smithi104 ceph-mon[34812]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-27T10:19:47.593 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:48.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:48 smithi143 ceph-mon[35545]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-05-27T10:19:48.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:48 smithi042 ceph-mon[31661]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-05-27T10:19:48.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:48 smithi104 ceph-mon[34812]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-05-27T10:19:49.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.364+0000 7feb42e00700 1 -- 172.21.15.42:0/1491457830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c1045e0 msgr2=0x7feb3c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.364+0000 7feb42e00700 1 --2- 172.21.15.42:0/1491457830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c1045e0 0x7feb3c1049f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0099c0 tx=0x7feb2c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 -- 172.21.15.42:0/1491457830 shutdown_connections 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 --2- 172.21.15.42:0/1491457830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb3c109860 0x7feb3c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 --2- 172.21.15.42:0/1491457830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c1045e0 0x7feb3c1049f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 --2- 172.21.15.42:0/1491457830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb3c108f10 0x7feb3c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 -- 172.21.15.42:0/1491457830 >> 172.21.15.42:0/1491457830 conn(0x7feb3c100030 msgr2=0x7feb3c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 -- 172.21.15.42:0/1491457830 shutdown_connections 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.365+0000 7feb42e00700 1 -- 172.21.15.42:0/1491457830 wait complete. 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.366+0000 7feb42e00700 1 Processor -- start 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.366+0000 7feb42e00700 1 -- start start 2024-05-27T10:19:49.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.366+0000 7feb42e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb3c1045e0 0x7feb3c075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb42e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 0x7feb3c075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb42e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb3c109860 0x7feb3c076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb42e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c114390 con 0x7feb3c108f10 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb42e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c114210 con 0x7feb3c109860 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb42e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c114510 con 0x7feb3c1045e0 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 0x7feb3c075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 0x7feb3c075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60932/0 (socket says 172.21.15.42:60932) 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 -- 172.21.15.42:0/2908951090 learned_addr learned my addr 172.21.15.42:0/2908951090 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb3be00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb3c1045e0 0x7feb3c075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb40e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb3c109860 0x7feb3c076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 -- 172.21.15.42:0/2908951090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb3c1045e0 msgr2=0x7feb3c075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb3c1045e0 0x7feb3c075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 -- 172.21.15.42:0/2908951090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb3c109860 msgr2=0x7feb3c076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb3c109860 0x7feb3c076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.367+0000 7feb33400700 1 -- 172.21.15.42:0/2908951090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c1a9100 con 0x7feb3c108f10 2024-05-27T10:19:49.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.368+0000 7feb33400700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 0x7feb3c075ff0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c8f0 tx=0x7feb2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.368+0000 7feb39600700 1 -- 172.21.15.42:0/2908951090 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24007b00 con 0x7feb3c108f10 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.368+0000 7feb39600700 1 -- 172.21.15.42:0/2908951090 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb24007ca0 con 0x7feb3c108f10 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.368+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c1a93f0 con 0x7feb3c108f10 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.368+0000 7feb39600700 1 -- 172.21.15.42:0/2908951090 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24018660 con 0x7feb3c108f10 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.368+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1a9930 con 0x7feb3c108f10 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.372+0000 7feb39600700 1 -- 172.21.15.42:0/2908951090 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7feb2400f450 con 0x7feb3c108f10 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.373+0000 7feb5190fb80 1 Processor -- start 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.373+0000 7feb5190fb80 1 -- start start 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb39600700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feb200b2b10 0x7feb200b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a6512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a651820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb5190fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e11a57fdf0 con 0x55e11a4d2df0 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb5190fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e11a56f990 con 0x55e11a5be9a0 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb5190fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e11a5739d0 con 0x55e11a651e20 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb39600700 1 -- 172.21.15.42:0/2908951090 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7feb240e28f0 con 0x7feb3c108f10 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb40e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feb200b2b10 0x7feb200b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb33400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a651820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb33400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a651820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60934/0 (socket says 172.21.15.42:60934) 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.374+0000 7feb33400700 1 -- 172.21.15.42:0/1836857039 learned_addr learned my addr 172.21.15.42:0/1836857039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb3be00700 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a6512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb40e00700 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb33400700 1 -- 172.21.15.42:0/1836857039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 msgr2=0x55e11a65e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb33400700 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb33400700 1 -- 172.21.15.42:0/1836857039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 msgr2=0x55e11a6512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb33400700 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a6512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb33400700 1 -- 172.21.15.42:0/1836857039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e11a5383d0 con 0x55e11a4d2df0 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.375+0000 7feb33400700 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a651820 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feb240a9900 tx=0x7feb240a9160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c6eb625ff5d82d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.376+0000 7feb40e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feb200b2b10 0x7feb200b5020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feb340060b0 tx=0x7feb34007890 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.376+0000 7feb31600700 1 -- 172.21.15.42:0/1836857039 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2407f5c0 con 0x55e11a4d2df0 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.376+0000 7feb31600700 1 -- 172.21.15.42:0/1836857039 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb240a9480 con 0x55e11a4d2df0 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.376+0000 7feb31600700 1 -- 172.21.15.42:0/1836857039 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240cc2b0 con 0x55e11a4d2df0 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 -- 172.21.15.42:0/1836857039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 msgr2=0x55e11a651820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a651820 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feb240a9900 tx=0x7feb240a9160 comp rx=0 tx=0).stop 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 -- 172.21.15.42:0/1836857039 shutdown_connections 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a6512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1836857039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a651820 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 -- 172.21.15.42:0/1836857039 >> 172.21.15.42:0/1836857039 conn(0x55e11a56c340 msgr2=0x55e11a64d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 -- 172.21.15.42:0/1836857039 shutdown_connections 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.377+0000 7feb5190fb80 1 -- 172.21.15.42:0/1836857039 wait complete. 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.378+0000 7feb5190fb80 1 -- start start 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.378+0000 7feb5190fb80 1 Processor -- start 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a65b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a65bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e11a475f50 con 0x55e11a4d2df0 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e11a510710 con 0x55e11a5be9a0 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e11a65eb30 con 0x55e11a651e20 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb5190fb80 10 client.0 _ll_register_callbacks cb 0x55e11a519c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:49.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb3be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a65b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb3be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a65b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60944/0 (socket says 172.21.15.42:60944) 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb3be00700 1 -- 172.21.15.42:0/3200979987 learned_addr learned my addr 172.21.15.42:0/3200979987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb33400700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a65bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.379+0000 7feb40e00700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb3be00700 1 -- 172.21.15.42:0/3200979987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 msgr2=0x55e11a65c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb3be00700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb3be00700 1 -- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 msgr2=0x55e11a65bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb3be00700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a65bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb3be00700 1 -- 172.21.15.42:0/3200979987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e11a4bf880 con 0x55e11a4d2df0 2024-05-27T10:19:49.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb33400700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a65bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb3be00700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a65b650 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0099c0 tx=0x7feb2c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dbecd81dc6ac16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb2a000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb2a000700 1 -- 172.21.15.42:0/3200979987 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00bd90 con 0x55e11a4d2df0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb2a000700 1 -- 172.21.15.42:0/3200979987 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7feb2c003680 con 0x55e11a4d2df0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.380+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb240a9480 con 0x55e11a4d2df0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.381+0000 7feb2a000700 1 -- 172.21.15.42:0/3200979987 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb2c030050 con 0x55e11a4d2df0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.381+0000 7feb5190fb80 10 client.15477 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.381+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e11a540010 con 0x55e11a4d2df0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.381+0000 7feb2a000700 1 -- 172.21.15.42:0/3200979987 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb240a9480 con 0x55e11a4d2df0 2024-05-27T10:19:49.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.381+0000 7feb2a000700 1 -- 172.21.15.42:0/3200979987 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7feb2c00f7e0 con 0x55e11a4d2df0 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.381+0000 7feb5190fb80 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.382+0000 7feb5190fb80 10 client.15477 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.382+0000 7feb5190fb80 20 client.15477 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.382+0000 7feb5190fb80 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a66bb00 0x55e11a66cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.382+0000 7feb5190fb80 4 client.15477 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.382+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e11a581d40 con 0x55e11a66bb00 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.382+0000 7feb33400700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a66bb00 0x55e11a66cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.383+0000 7feb33400700 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a66bb00 0x55e11a66cf70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba23b12511f9efd1 server_cookie=ee80e284e52f503a in_seq=0 out_seq=0 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.383+0000 7feb2a000700 10 client.15477 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.385+0000 7feb2a000700 1 -- 172.21.15.42:0/3200979987 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e11a581d40 con 0x55e11a66bb00 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.385+0000 7feb2a000700 10 client.15477 handle_command_reply: tid=0 2024-05-27T10:19:49.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.385+0000 7feb5190fb80 1 client.15477 shutdown 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 msgr2=0x55e11a65b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a65b650 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0099c0 tx=0x7feb2c0096a0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 shutdown_connections 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a66bb00 0x55e11a66cf70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a651e20 0x55e11a65c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a4d2df0 0x55e11a65b650 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 --2- 172.21.15.42:0/3200979987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a65bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb2a000700 0 client.15477 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 >> 172.21.15.42:0/3200979987 conn(0x55e11a56c340 msgr2=0x55e11a6512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 shutdown_connections 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 1 -- 172.21.15.42:0/3200979987 wait complete. 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.386+0000 7feb5190fb80 20 client.15477 trim_cache size 0 max 16384 2024-05-27T10:19:49.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.410+0000 7feb5190fb80 1 Processor -- start 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 -- start start 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a659f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a64d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a651060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e11a66b2b0 con 0x55e11a651e20 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e11a66b980 con 0x55e11a5be9a0 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb5190fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e11a66c050 con 0x55e11a4d2df0 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb40e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a651060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.411+0000 7feb40e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a651060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60958/0 (socket says 172.21.15.42:60958) 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 -- 172.21.15.42:0/2421826878 learned_addr learned my addr 172.21.15.42:0/2421826878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb33400700 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a64d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb3be00700 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a659f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 -- 172.21.15.42:0/2421826878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 msgr2=0x55e11a659f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a659f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 -- 172.21.15.42:0/2421826878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 msgr2=0x55e11a64d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a64d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 -- 172.21.15.42:0/2421826878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e11a64e800 con 0x55e11a651e20 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.412+0000 7feb40e00700 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a651060 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feb34019660 tx=0x7feb34009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84694800267caec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.413+0000 7feb30c00700 1 -- 172.21.15.42:0/2421826878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34012040 con 0x55e11a651e20 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.413+0000 7feb30c00700 1 -- 172.21.15.42:0/2421826878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb34024a90 con 0x55e11a651e20 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.413+0000 7feb30c00700 1 -- 172.21.15.42:0/2421826878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340238e0 con 0x55e11a651e20 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.413+0000 7feb5190fb80 1 -- 172.21.15.42:0/2421826878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 msgr2=0x55e11a651060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.413+0000 7feb5190fb80 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a651060 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feb34019660 tx=0x7feb34009510 comp rx=0 tx=0).stop 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 -- 172.21.15.42:0/2421826878 shutdown_connections 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a651060 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a659f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 --2- 172.21.15.42:0/2421826878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a64d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 -- 172.21.15.42:0/2421826878 >> 172.21.15.42:0/2421826878 conn(0x55e11a56c340 msgr2=0x55e11a64ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 -- 172.21.15.42:0/2421826878 shutdown_connections 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 -- 172.21.15.42:0/2421826878 wait complete. 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 -- start start 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.414+0000 7feb5190fb80 1 Processor -- start 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a6763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a676930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a66f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e11a66cdf0 con 0x55e11a651e20 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e11a66c720 con 0x55e11a5be9a0 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e11a66d4c0 con 0x55e11a4d2df0 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb5190fb80 10 client.0 _ll_register_callbacks cb 0x55e11a519c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:49.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb40e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a66f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb40e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a66f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60968/0 (socket says 172.21.15.42:60968) 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.415+0000 7feb40e00700 1 -- 172.21.15.42:0/1875412333 learned_addr learned my addr 172.21.15.42:0/1875412333 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb3be00700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a6763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb33400700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a676930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb40e00700 1 -- 172.21.15.42:0/1875412333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 msgr2=0x55e11a6763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb40e00700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a6763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb40e00700 1 -- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 msgr2=0x55e11a676930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb40e00700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a676930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb40e00700 1 -- 172.21.15.42:0/1875412333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e11a65c440 con 0x55e11a651e20 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb40e00700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a66f390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb34026040 tx=0x7feb34004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449574034d925ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.416+0000 7feb2a000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb2a000700 1 -- 172.21.15.42:0/1875412333 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34031020 con 0x55e11a651e20 2024-05-27T10:19:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb2a000700 1 -- 172.21.15.42:0/1875412333 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7feb34023520 con 0x55e11a651e20 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb34024a90 con 0x55e11a651e20 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb2a000700 1 -- 172.21.15.42:0/1875412333 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb3403c050 con 0x55e11a651e20 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb5190fb80 10 client.15483 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e11a6d7650 con 0x55e11a651e20 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.417+0000 7feb2a000700 1 -- 172.21.15.42:0/1875412333 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb34024a90 con 0x55e11a651e20 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb2a000700 1 -- 172.21.15.42:0/1875412333 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7feb34018040 con 0x55e11a651e20 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:19:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34442, 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 311.830276895, 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 82, 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 82, 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 386.248388035 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb5190fb80 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb5190fb80 10 client.15483 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb5190fb80 20 client.15483 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a6d9730 0x55e11a6daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb5190fb80 4 client.15483 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.418+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "status"}) -- 0x55e11a57b520 con 0x55e11a6d9730 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.419+0000 7feb3be00700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a6d9730 0x55e11a6daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.419+0000 7feb3be00700 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a6d9730 0x55e11a6daba0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a72e0c3ee3aa808 server_cookie=98ad7da06971da0c in_seq=0 out_seq=0 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.419+0000 7feb2a000700 10 client.15483 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb2a000700 1 -- 172.21.15.42:0/1875412333 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e11a57b520 con 0x55e11a6d9730 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb2a000700 10 client.15483 handle_command_reply: tid=0 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 client.15483 shutdown 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 msgr2=0x55e11a66f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a66f390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb34026040 tx=0x7feb34004e20 comp rx=0 tx=0).stop 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 shutdown_connections 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55e11a6d9730 0x55e11a6daba0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e11a651e20 0x55e11a66f390 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e11a4d2df0 0x55e11a6763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 --2- 172.21.15.42:0/1875412333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e11a5be9a0 0x55e11a676930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb2a000700 0 client.15483 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 >> 172.21.15.42:0/1875412333 conn(0x55e11a56c340 msgr2=0x55e11a64ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 shutdown_connections 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.420+0000 7feb5190fb80 1 -- 172.21.15.42:0/1875412333 wait complete. 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb5190fb80 20 client.15483 trim_cache size 0 max 16384 2024-05-27T10:19:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feb200b2b10 msgr2=0x7feb200b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feb200b2b10 0x7feb200b5020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feb340060b0 tx=0x7feb34007890 comp rx=0 tx=0).stop 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 msgr2=0x7feb3c075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 0x7feb3c075ff0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c8f0 tx=0x7feb2400ccb0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 shutdown_connections 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feb200b2b10 0x7feb200b5020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb3c109860 0x7feb3c076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb3c1045e0 0x7feb3c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 --2- 172.21.15.42:0/2908951090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb3c108f10 0x7feb3c075ff0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.421+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 >> 172.21.15.42:0/2908951090 conn(0x7feb3c100030 msgr2=0x7feb3c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.422+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 shutdown_connections 2024-05-27T10:19:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:49.422+0000 7feb42e00700 1 -- 172.21.15.42:0/2908951090 wait complete. 2024-05-27T10:19:50.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:50 smithi042 ceph-mon[31661]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-27T10:19:50.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:50 smithi143 ceph-mon[35545]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-27T10:19:50.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:50 smithi104 ceph-mon[34812]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-27T10:19:50.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T10:19:51.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 -- 172.21.15.42:0/2530469300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc784105510 msgr2=0x7fc784105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 --2- 172.21.15.42:0/2530469300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc784105510 0x7fc784105920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc774009990 tx=0x7fc774009ca0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 -- 172.21.15.42:0/2530469300 shutdown_connections 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 --2- 172.21.15.42:0/2530469300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc78410e8f0 0x7fc784110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 --2- 172.21.15.42:0/2530469300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc784105510 0x7fc784105920 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 --2- 172.21.15.42:0/2530469300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 0x7fc78410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 -- 172.21.15.42:0/2530469300 >> 172.21.15.42:0/2530469300 conn(0x7fc7840fedb0 msgr2=0x7fc7841011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 -- 172.21.15.42:0/2530469300 shutdown_connections 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.583+0000 7fc78a000700 1 -- 172.21.15.42:0/2530469300 wait complete. 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 Processor -- start 2024-05-27T10:19:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 -- start start 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc784105510 0x7fc784103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 0x7fc784103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc78410e8f0 0x7fc784103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841142d0 con 0x7fc78410e8f0 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc784114150 con 0x7fc784105510 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.585+0000 7fc78a000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc784114450 con 0x7fc784105e60 2024-05-27T10:19:51.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.586+0000 7fc783400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc784105510 0x7fc784103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.586+0000 7fc782a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 0x7fc784103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.586+0000 7fc782a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 0x7fc784103740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:43138/0 (socket says 172.21.15.42:43138) 2024-05-27T10:19:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.586+0000 7fc782a00700 1 -- 172.21.15.42:0/997128695 learned_addr learned my addr 172.21.15.42:0/997128695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783e00700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc78410e8f0 0x7fc784103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783400700 1 -- 172.21.15.42:0/997128695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 msgr2=0x7fc784103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783400700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 0x7fc784103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783400700 1 -- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc78410e8f0 msgr2=0x7fc784103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783400700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc78410e8f0 0x7fc784103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783400700 1 -- 172.21.15.42:0/997128695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7841a9010 con 0x7fc784105510 2024-05-27T10:19:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.587+0000 7fc783e00700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc78410e8f0 0x7fc784103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:19:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.588+0000 7fc783400700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc784105510 0x7fc784103200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc7740057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.588+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774027070 con 0x7fc784105510 2024-05-27T10:19:51.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.588+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7841a92a0 con 0x7fc784105510 2024-05-27T10:19:51.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.589+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7841a9750 con 0x7fc784105510 2024-05-27T10:19:51.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.589+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc77400fb00 con 0x7fc784105510 2024-05-27T10:19:51.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.589+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774022a60 con 0x7fc784105510 2024-05-27T10:19:51.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.590+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc764005330 con 0x7fc784105510 2024-05-27T10:19:51.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.596+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fc77402a070 con 0x7fc784105510 2024-05-27T10:19:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.597+0000 7fc77be00700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc7680b2b90 0x7fc7680b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.598+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc7740e3910 con 0x7fc784105510 2024-05-27T10:19:51.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.598+0000 7fc782a00700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc7680b2b90 0x7fc7680b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:51.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.599+0000 7fc782a00700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc7680b2b90 0x7fc7680b50a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc7841024a0 tx=0x7fc770005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.610+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc774013070 con 0x7fc784105510 2024-05-27T10:19:51.673 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:51.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.929+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc764005620 con 0x7fc784105510 2024-05-27T10:19:51.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.932+0000 7fc77be00700 1 -- 172.21.15.42:0/997128695 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fc7740ab120 con 0x7fc784105510 2024-05-27T10:19:51.932 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:19:51.932 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:19:51.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc7680b2b90 msgr2=0x7fc7680b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:51.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc7680b2b90 0x7fc7680b50a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc7841024a0 tx=0x7fc770005f90 comp rx=0 tx=0).stop 2024-05-27T10:19:51.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc784105510 msgr2=0x7fc784103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:51.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc784105510 0x7fc784103200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc7740057a0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 shutdown_connections 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc7680b2b90 0x7fc7680b50a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc78410e8f0 0x7fc784103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc784105510 0x7fc784103200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 --2- 172.21.15.42:0/997128695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc784105e60 0x7fc784103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 >> 172.21.15.42:0/997128695 conn(0x7fc7840fedb0 msgr2=0x7fc7840ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 shutdown_connections 2024-05-27T10:19:51.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:51.937+0000 7fc78a000700 1 -- 172.21.15.42:0/997128695 wait complete. 2024-05-27T10:19:51.939 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:19:52.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:52 smithi143 ceph-mon[35545]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-05-27T10:19:52.308 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:52 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/997128695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:52 smithi042 ceph-mon[31661]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-05-27T10:19:52.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:52 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/997128695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:52.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:52 smithi104 ceph-mon[34812]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-05-27T10:19:52.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:52 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/997128695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:19:54.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:54 smithi143 ceph-mon[35545]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-27T10:19:54.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:54 smithi042 ceph-mon[31661]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-27T10:19:54.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T10:19:54.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:54 smithi104 ceph-mon[34812]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-27T10:19:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:55.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.159+0000 7f0da3200700 1 -- 172.21.15.42:0/4082353684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 msgr2=0x7f0d9c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.159+0000 7f0da3200700 1 --2- 172.21.15.42:0/4082353684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c079200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009990 tx=0x7f0d8c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 -- 172.21.15.42:0/4082353684 shutdown_connections 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 --2- 172.21.15.42:0/4082353684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c115210 0x7f0d9c117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 --2- 172.21.15.42:0/4082353684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c079200 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 --2- 172.21.15.42:0/4082353684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c079740 0x7f0d9c114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 -- 172.21.15.42:0/4082353684 >> 172.21.15.42:0/4082353684 conn(0x7f0d9c1020b0 msgr2=0x7f0d9c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 -- 172.21.15.42:0/4082353684 shutdown_connections 2024-05-27T10:19:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.160+0000 7f0da3200700 1 -- 172.21.15.42:0/4082353684 wait complete. 2024-05-27T10:19:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.161+0000 7f0da3200700 1 Processor -- start 2024-05-27T10:19:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.161+0000 7f0da3200700 1 -- start start 2024-05-27T10:19:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0da3200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0da3200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c079740 0x7f0d9c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0da3200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c115210 0x7f0d9c1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0da3200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c11a960 con 0x7f0d9c078df0 2024-05-27T10:19:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0da3200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c11a7e0 con 0x7f0d9c079740 2024-05-27T10:19:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0da3200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c11aae0 con 0x7f0d9c115210 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0d9be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0d9be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c1a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37246/0 (socket says 172.21.15.42:37246) 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0d9be00700 1 -- 172.21.15.42:0/1482793597 learned_addr learned my addr 172.21.15.42:0/1482793597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.162+0000 7f0d9b400700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c079740 0x7f0d9c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0da1200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c115210 0x7f0d9c1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9be00700 1 -- 172.21.15.42:0/1482793597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c115210 msgr2=0x7f0d9c1a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9be00700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c115210 0x7f0d9c1a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9be00700 1 -- 172.21.15.42:0/1482793597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c079740 msgr2=0x7f0d9c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9be00700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c079740 0x7f0d9c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9be00700 1 -- 172.21.15.42:0/1482793597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c1a6170 con 0x7f0d9c078df0 2024-05-27T10:19:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9b400700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c079740 0x7f0d9c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.163+0000 7f0d9be00700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c1a1350 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.164+0000 7f0d98c00700 1 -- 172.21.15.42:0/1482793597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c027070 con 0x7f0d9c078df0 2024-05-27T10:19:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.164+0000 7f0d98c00700 1 -- 172.21.15.42:0/1482793597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d8c00fdb0 con 0x7f0d9c078df0 2024-05-27T10:19:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.164+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1a6400 con 0x7f0d9c078df0 2024-05-27T10:19:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.164+0000 7f0d98c00700 1 -- 172.21.15.42:0/1482793597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0173f0 con 0x7f0d9c078df0 2024-05-27T10:19:55.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.164+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a6930 con 0x7f0d9c078df0 2024-05-27T10:19:55.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.168+0000 7f0d98c00700 1 -- 172.21.15.42:0/1482793597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0d8c02a070 con 0x7f0d9c078df0 2024-05-27T10:19:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.169+0000 7f0db1ca5b80 1 Processor -- start 2024-05-27T10:19:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.169+0000 7f0db1ca5b80 1 -- start start 2024-05-27T10:19:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c45267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c4520820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4521170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5620c444edf0 con 0x5620c4520d60 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5620c443e990 con 0x5620c4526ce0 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5620c44429d0 con 0x5620c43a1df0 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0d9b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4521170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0d9b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4521170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37254/0 (socket says 172.21.15.42:37254) 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.170+0000 7f0d9b400700 1 -- 172.21.15.42:0/411240563 learned_addr learned my addr 172.21.15.42:0/411240563 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9be00700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c4520820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0da1200700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c45267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9b400700 1 -- 172.21.15.42:0/411240563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 msgr2=0x5620c45267a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9b400700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c45267a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9b400700 1 -- 172.21.15.42:0/411240563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 msgr2=0x5620c4520820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9b400700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c4520820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9b400700 1 -- 172.21.15.42:0/411240563 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c44073d0 con 0x5620c4520d60 2024-05-27T10:19:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0da1200700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c45267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d98c00700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d800b2b90 0x7f0d800b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.171+0000 7f0d9be00700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c4520820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.172+0000 7f0d9b400700 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4521170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d88008c40 tx=0x7f0d88008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabb56ddb79d30d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.172+0000 7f0d98c00700 1 -- 172.21.15.42:0/1482793597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0d8c0e3a90 con 0x7f0d9c078df0 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.172+0000 7f0da1200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d800b2b90 0x7f0d800b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.172+0000 7f0d91600700 1 -- 172.21.15.42:0/411240563 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88014070 con 0x5620c4520d60 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.172+0000 7f0d91600700 1 -- 172.21.15.42:0/411240563 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d8800ee20 con 0x5620c4520d60 2024-05-27T10:19:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.172+0000 7f0d91600700 1 -- 172.21.15.42:0/411240563 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88010430 con 0x5620c4520d60 2024-05-27T10:19:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/411240563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 msgr2=0x5620c4521170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0da1200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d800b2b90 0x7f0d800b50a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5620c4521ae0 tx=0x7f0d94009040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4521170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d88008c40 tx=0x7f0d88008f50 comp rx=0 tx=0).stop 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/411240563 shutdown_connections 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4521170 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c45267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/411240563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c4520820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.173+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/411240563 >> 172.21.15.42:0/411240563 conn(0x5620c4401f70 msgr2=0x5620c451b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.174+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/411240563 shutdown_connections 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.174+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/411240563 wait complete. 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.174+0000 7f0db1ca5b80 1 -- start start 2024-05-27T10:19:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 Processor -- start 2024-05-27T10:19:55.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c43a1df0 0x5620c452a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c452a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 0x5620c452aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5620c4344f50 con 0x5620c4526ce0 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5620c43df710 con 0x5620c43a1df0 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.175+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5620c452d7b0 con 0x5620c4520d60 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.176+0000 7f0db1ca5b80 10 client.0 _ll_register_callbacks cb 0x5620c43e8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.176+0000 7f0da1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 0x5620c452aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.176+0000 7f0da1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 0x5620c452aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37256/0 (socket says 172.21.15.42:37256) 2024-05-27T10:19:55.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.176+0000 7f0da1200700 1 -- 172.21.15.42:0/606476603 learned_addr learned my addr 172.21.15.42:0/606476603 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.176+0000 7f0d9be00700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c43a1df0 0x5620c452a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.176+0000 7f0d9b400700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c452a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0da1200700 1 -- 172.21.15.42:0/606476603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 msgr2=0x5620c452a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0da1200700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c452a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0da1200700 1 -- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c43a1df0 msgr2=0x5620c452a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0da1200700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c43a1df0 0x5620c452a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0da1200700 1 -- 172.21.15.42:0/606476603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c438e880 con 0x5620c4526ce0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0da1200700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 0x5620c452aeb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0d94015a40 tx=0x7f0d94004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48216d0a7477351f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0d86000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0d86000700 1 -- 172.21.15.42:0/606476603 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9401c050 con 0x5620c4526ce0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0d86000700 1 -- 172.21.15.42:0/606476603 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0d94016ab0 con 0x5620c4526ce0 2024-05-27T10:19:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.177+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8800ee20 con 0x5620c4526ce0 2024-05-27T10:19:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.178+0000 7f0d86000700 1 -- 172.21.15.42:0/606476603 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d9402f050 con 0x5620c4526ce0 2024-05-27T10:19:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.178+0000 7f0db1ca5b80 10 client.15501 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.178+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5620c440f010 con 0x5620c4526ce0 2024-05-27T10:19:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.178+0000 7f0d86000700 1 -- 172.21.15.42:0/606476603 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d8800ee20 con 0x5620c4526ce0 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.178+0000 7f0d86000700 1 -- 172.21.15.42:0/606476603 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0d940143c0 con 0x5620c4526ce0 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0db1ca5b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0db1ca5b80 10 client.15501 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0db1ca5b80 20 client.15501 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c453a7d0 0x5620c453bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0db1ca5b80 4 client.15501 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620c4450d40 con 0x5620c453a7d0 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.179+0000 7f0d9be00700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c453a7d0 0x5620c453bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.181+0000 7f0d9be00700 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c453a7d0 0x5620c453bc40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce8957487b60b8cd server_cookie=6cfcbee1d6be8b5e in_seq=0 out_seq=0 2024-05-27T10:19:55.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.181+0000 7f0d86000700 10 client.15501 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:19:55.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0d86000700 1 -- 172.21.15.42:0/606476603 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5620c4450d40 con 0x5620c453a7d0 2024-05-27T10:19:55.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0d86000700 10 client.15501 handle_command_reply: tid=0 2024-05-27T10:19:55.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 client.15501 shutdown 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 msgr2=0x5620c452aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 0x5620c452aeb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0d94015a40 tx=0x7f0d94004b40 comp rx=0 tx=0).stop 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 shutdown_connections 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c453a7d0 0x5620c453bc40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4526ce0 0x5620c452aeb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c43a1df0 0x5620c452a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/606476603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c452a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 >> 172.21.15.42:0/606476603 conn(0x5620c4401f70 msgr2=0x5620c451b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:55.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0d86000700 0 client.15501 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:19:55.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 shutdown_connections 2024-05-27T10:19:55.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.183+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/606476603 wait complete. 2024-05-27T10:19:55.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.184+0000 7f0db1ca5b80 20 client.15501 trim_cache size 0 max 16384 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.208+0000 7f0db1ca5b80 1 Processor -- start 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 -- start start 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c4520820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4536ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5620c4539f80 con 0x5620c4520d60 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5620c453a650 con 0x5620c4526ce0 2024-05-27T10:19:55.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5620c453ad20 con 0x5620c43a1df0 2024-05-27T10:19:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0d9b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4536ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0d9b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4536ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37272/0 (socket says 172.21.15.42:37272) 2024-05-27T10:19:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.209+0000 7f0d9b400700 1 -- 172.21.15.42:0/457663898 learned_addr learned my addr 172.21.15.42:0/457663898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0da1200700 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0d9be00700 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c4520820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0d9b400700 1 -- 172.21.15.42:0/457663898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 msgr2=0x5620c4520820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0d9b400700 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c4520820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0d9b400700 1 -- 172.21.15.42:0/457663898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 msgr2=0x5620c45293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0d9b400700 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45293b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.210+0000 7f0d9b400700 1 -- 172.21.15.42:0/457663898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c452b220 con 0x5620c4520d60 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.211+0000 7f0d9b400700 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4536ae0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0d88008c40 tx=0x7f0d88000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75a2719c214b6d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.211+0000 7f0d90c00700 1 -- 172.21.15.42:0/457663898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88014070 con 0x5620c4520d60 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.211+0000 7f0d90c00700 1 -- 172.21.15.42:0/457663898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d88005030 con 0x5620c4520d60 2024-05-27T10:19:55.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.211+0000 7f0d90c00700 1 -- 172.21.15.42:0/457663898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88004930 con 0x5620c4520d60 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.212+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/457663898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 msgr2=0x5620c4536ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.212+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4536ae0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0d88008c40 tx=0x7f0d88000bc0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.212+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/457663898 shutdown_connections 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.213+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.213+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c43a1df0 0x5620c4520820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.213+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/457663898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c4520d60 0x5620c4536ae0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.213+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/457663898 >> 172.21.15.42:0/457663898 conn(0x5620c4401f70 msgr2=0x5620c45240f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.213+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/457663898 shutdown_connections 2024-05-27T10:19:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.213+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/457663898 wait complete. 2024-05-27T10:19:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.214+0000 7f0db1ca5b80 1 -- start start 2024-05-27T10:19:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.214+0000 7f0db1ca5b80 1 Processor -- start 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 0x5620c4545330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c4545870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0db1ca5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0d9be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 0x5620c4545330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0d9be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 0x5620c4545330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37278/0 (socket says 172.21.15.42:37278) 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0d9be00700 1 -- 172.21.15.42:0/1179551784 learned_addr learned my addr 172.21.15.42:0/1179551784 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0db1ca5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5620c453bac0 con 0x5620c43a1df0 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5620c453b3f0 con 0x5620c4526ce0 2024-05-27T10:19:55.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.216+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5620c453c190 con 0x5620c4520d60 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0db1ca5b80 10 client.0 _ll_register_callbacks cb 0x5620c43e8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9b400700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c4545870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0da1200700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9be00700 1 -- 172.21.15.42:0/1179551784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 msgr2=0x5620c4545870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9be00700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c4545870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9be00700 1 -- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 msgr2=0x5620c45400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9be00700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9be00700 1 -- 172.21.15.42:0/1179551784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c45408a0 con 0x5620c43a1df0 2024-05-27T10:19:55.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9b400700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c4545870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:19:55.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d9be00700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 0x5620c4545330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0aa8a0 tx=0x7f0d8c0e5ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94756b287fc91708 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:55.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.217+0000 7f0d86000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:55.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0d86000700 1 -- 172.21.15.42:0/1179551784 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0b32b0 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0d86000700 1 -- 172.21.15.42:0/1179551784 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0d8c022470 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d88005030 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0d86000700 1 -- 172.21.15.42:0/1179551784 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d8c0aa6b0 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0db1ca5b80 10 client.15507 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5620c45a69e0 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.218+0000 7f0d86000700 1 -- 172.21.15.42:0/1179551784 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d88005030 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0d86000700 1 -- 172.21.15.42:0/1179551784 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0d8c0e1330 con 0x5620c43a1df0 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0db1ca5b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0db1ca5b80 10 client.15507 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0db1ca5b80 20 client.15507 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c45a8f10 0x5620c45aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0db1ca5b80 4 client.15507 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:19:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.219+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "status"}) -- 0x5620c444a520 con 0x5620c45a8f10 2024-05-27T10:19:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.220+0000 7f0d9b400700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c45a8f10 0x5620c45aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.221+0000 7f0d9b400700 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c45a8f10 0x5620c45aa380 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b22d4470ab30531 server_cookie=2a30c960a92745f8 in_seq=0 out_seq=0 2024-05-27T10:19:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.221+0000 7f0d86000700 10 client.15507 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.221+0000 7f0d86000700 1 -- 172.21.15.42:0/1179551784 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5620c444a520 con 0x5620c45a8f10 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.221+0000 7f0d86000700 10 client.15507 handle_command_reply: tid=0 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.221+0000 7f0db1ca5b80 1 client.15507 shutdown 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 msgr2=0x5620c4545330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 0x5620c4545330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0aa8a0 tx=0x7f0d8c0e5ee0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 shutdown_connections 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5620c45a8f10 0x5620c45aa380 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620c4526ce0 0x5620c45400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620c43a1df0 0x5620c4545330 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 --2- 172.21.15.42:0/1179551784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620c4520d60 0x5620c4545870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 >> 172.21.15.42:0/1179551784 conn(0x5620c4401f70 msgr2=0x5620c4525d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0d86000700 0 client.15507 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:19:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 shutdown_connections 2024-05-27T10:19:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 1 -- 172.21.15.42:0/1179551784 wait complete. 2024-05-27T10:19:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.222+0000 7f0db1ca5b80 20 client.15507 trim_cache size 0 max 16384 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34466, 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 318.646252312, 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 82, 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 82, 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 379.76073310999999 2024-05-27T10:19:55.224 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:19:55.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d800b2b90 msgr2=0x7f0d800b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d800b2b90 0x7f0d800b50a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5620c4521ae0 tx=0x7f0d94009040 comp rx=0 tx=0).stop 2024-05-27T10:19:55.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 msgr2=0x7f0d9c1a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:55.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c1a1350 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c0047b0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 shutdown_connections 2024-05-27T10:19:55.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0d800b2b90 0x7f0d800b50a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c115210 0x7f0d9c1a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d9c078df0 0x7f0d9c1a1350 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 --2- 172.21.15.42:0/1482793597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d9c079740 0x7f0d9c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:55.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.223+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 >> 172.21.15.42:0/1482793597 conn(0x7f0d9c1020b0 msgr2=0x7f0d9c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:55.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.224+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 shutdown_connections 2024-05-27T10:19:55.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:19:55.224+0000 7f0da3200700 1 -- 172.21.15.42:0/1482793597 wait complete. 2024-05-27T10:19:56.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:56 smithi042 ceph-mon[31661]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-27T10:19:56.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:56 smithi143 ceph-mon[35545]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-27T10:19:56.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:56 smithi104 ceph-mon[34812]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-27T10:19:56.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T10:19:57.771 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:19:58.190 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:19:58 smithi042 ceph-mon[31661]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-27T10:19:58.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:19:58 smithi143 ceph-mon[35545]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-27T10:19:58.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:19:58 smithi104 ceph-mon[34812]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-27T10:19:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 -- 172.21.15.42:0/1497639410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd90010d9c0 msgr2=0x7fd90010fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 --2- 172.21.15.42:0/1497639410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd90010d9c0 0x7fd90010fde0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009900 tx=0x7fd8f002a480 comp rx=0 tx=0).stop 2024-05-27T10:19:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 -- 172.21.15.42:0/1497639410 shutdown_connections 2024-05-27T10:19:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 --2- 172.21.15.42:0/1497639410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd900110320 0x7fd900112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 --2- 172.21.15.42:0/1497639410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd90010d9c0 0x7fd90010fde0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 --2- 172.21.15.42:0/1497639410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd900075420 0x7fd900075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 -- 172.21.15.42:0/1497639410 >> 172.21.15.42:0/1497639410 conn(0x7fd90007a1b0 msgr2=0x7fd90007a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 -- 172.21.15.42:0/1497639410 shutdown_connections 2024-05-27T10:19:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.855+0000 7fd907200640 1 -- 172.21.15.42:0/1497639410 wait complete. 2024-05-27T10:19:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 Processor -- start 2024-05-27T10:19:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 -- start start 2024-05-27T10:19:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 0x7fd9001a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90010d9c0 0x7fd9001a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd900110320 0x7fd90019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd900115d90 con 0x7fd900075420 2024-05-27T10:19:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd900115c10 con 0x7fd900110320 2024-05-27T10:19:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.857+0000 7fd907200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd900115f10 con 0x7fd90010d9c0 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 0x7fd9001a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 0x7fd9001a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37292/0 (socket says 172.21.15.42:37292) 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 -- 172.21.15.42:0/3535938590 learned_addr learned my addr 172.21.15.42:0/3535938590 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905400640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90010d9c0 0x7fd9001a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd906800640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd900110320 0x7fd90019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 -- 172.21.15.42:0/3535938590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90010d9c0 msgr2=0x7fd9001a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90010d9c0 0x7fd9001a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 -- 172.21.15.42:0/3535938590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd900110320 msgr2=0x7fd90019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd900110320 0x7fd90019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.858+0000 7fd905e00640 1 -- 172.21.15.42:0/3535938590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90019ce40 con 0x7fd900075420 2024-05-27T10:19:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.859+0000 7fd905e00640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 0x7fd9001a2590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800eab0 tx=0x7fd8f800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.859+0000 7fd8f6a00640 1 -- 172.21.15.42:0/3535938590 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800ce40 con 0x7fd900075420 2024-05-27T10:19:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.859+0000 7fd8f6a00640 1 -- 172.21.15.42:0/3535938590 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8f8011900 con 0x7fd900075420 2024-05-27T10:19:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.859+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90019d0d0 con 0x7fd900075420 2024-05-27T10:19:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.860+0000 7fd8f6a00640 1 -- 172.21.15.42:0/3535938590 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8010990 con 0x7fd900075420 2024-05-27T10:19:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.860+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9001a9330 con 0x7fd900075420 2024-05-27T10:19:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.863+0000 7fd90f7c0740 1 Processor -- start 2024-05-27T10:19:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 -- start start 2024-05-27T10:19:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd8f6a00640 1 -- 172.21.15.42:0/3535938590 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fd8f8011470 con 0x7fd900075420 2024-05-27T10:19:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 0x563a9c6cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3002f0 0x563a9c6cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3084d0 0x563a9c6cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563a9c63cb90 con 0x563a9c3291b0 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563a9c62c730 con 0x563a9c3084d0 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563a9c630770 con 0x563a9c3002f0 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.864+0000 7fd906800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 0x563a9c6cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 0x563a9c6cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37298/0 (socket says 172.21.15.42:37298) 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 -- 172.21.15.42:0/3961546509 learned_addr learned my addr 172.21.15.42:0/3961546509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd905400640 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3084d0 0x563a9c6cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd905e00640 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3002f0 0x563a9c6cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 -- 172.21.15.42:0/3961546509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3002f0 msgr2=0x563a9c6cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3002f0 0x563a9c6cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 -- 172.21.15.42:0/3961546509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3084d0 msgr2=0x563a9c6cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3084d0 0x563a9c6cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.865+0000 7fd906800640 1 -- 172.21.15.42:0/3961546509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9c5c58b0 con 0x563a9c3291b0 2024-05-27T10:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.866+0000 7fd906800640 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 0x563a9c6cba60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00bba0 tx=0x7fd8fc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0306c75159d191 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.866+0000 7fd8eb000640 1 -- 172.21.15.42:0/3961546509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc015070 con 0x563a9c3291b0 2024-05-27T10:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.866+0000 7fd8eb000640 1 -- 172.21.15.42:0/3961546509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8fc0054b0 con 0x563a9c3291b0 2024-05-27T10:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.866+0000 7fd8eb000640 1 -- 172.21.15.42:0/3961546509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc011440 con 0x563a9c3291b0 2024-05-27T10:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd8f6a00640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd8ec0b2a70 0x7fd8ec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd8f6a00640 1 -- 172.21.15.42:0/3535938590 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fd8f80e2f00 con 0x7fd900075420 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd905e00640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd8ec0b2a70 0x7fd8ec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 -- 172.21.15.42:0/3961546509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 msgr2=0x563a9c6cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 0x563a9c6cba60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00bba0 tx=0x7fd8fc00f730 comp rx=0 tx=0).stop 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 -- 172.21.15.42:0/3961546509 shutdown_connections 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3084d0 0x563a9c6cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3002f0 0x563a9c6cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/3961546509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3291b0 0x563a9c6cba60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 -- 172.21.15.42:0/3961546509 >> 172.21.15.42:0/3961546509 conn(0x563a9c31d300 msgr2=0x563a9c6c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 -- 172.21.15.42:0/3961546509 shutdown_connections 2024-05-27T10:19:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.867+0000 7fd90f7c0740 1 -- 172.21.15.42:0/3961546509 wait complete. 2024-05-27T10:19:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.868+0000 7fd905e00640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd8ec0b2a70 0x7fd8ec0b4f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80e0520 tx=0x7fd8f80094a0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.868+0000 7fd90f7c0740 1 -- start start 2024-05-27T10:19:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.868+0000 7fd90f7c0740 1 Processor -- start 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563a9c312790 con 0x563a9c3002f0 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563a9c5d4bc0 con 0x563a9c3291b0 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563a9c6d8b90 con 0x563a9c3084d0 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd90f7c0740 10 client.0 _ll_register_callbacks cb 0x563a9c5b2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd906800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd906800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37312/0 (socket says 172.21.15.42:37312) 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.869+0000 7fd906800640 1 -- 172.21.15.42:0/121844211 learned_addr learned my addr 172.21.15.42:0/121844211 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd905400640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 -- 172.21.15.42:0/121844211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 msgr2=0x563a9c6d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 -- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 msgr2=0x563a9c6e1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 -- 172.21.15.42:0/121844211 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9c341210 con 0x563a9c3002f0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd906800640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d5c00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc018000 tx=0x7fd8fc004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419883fa01a05e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.870+0000 7fd8df400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd8df400640 1 -- 172.21.15.42:0/121844211 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc015050 con 0x563a9c3002f0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd8df400640 1 -- 172.21.15.42:0/121844211 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fd8fc004c90 con 0x563a9c3002f0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8fc0054b0 con 0x563a9c3002f0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd8df400640 1 -- 172.21.15.42:0/121844211 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd8fc005850 con 0x563a9c3002f0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd90f7c0740 10 client.15519 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a9c40d620 con 0x563a9c3002f0 2024-05-27T10:19:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.871+0000 7fd8df400640 1 -- 172.21.15.42:0/121844211 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8fc0054b0 con 0x563a9c3002f0 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd8df400640 1 -- 172.21.15.42:0/121844211 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd8fc032ab0 con 0x563a9c3002f0 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd90f7c0740 10 client.15519 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd90f7c0740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd90f7c0740 20 client.15519 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2e7d80 0x563a9c6e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd90f7c0740 4 client.15519 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.872+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a9c63eae0 con 0x563a9c2e7d80 2024-05-27T10:19:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.873+0000 7fd905400640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2e7d80 0x563a9c6e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.874+0000 7fd905400640 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2e7d80 0x563a9c6e7d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2c5053fbec90b server_cookie=64f8d3fe846a32e2 in_seq=0 out_seq=0 2024-05-27T10:19:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.874+0000 7fd8df400640 10 client.15519 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd8df400640 1 -- 172.21.15.42:0/121844211 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a9c63eae0 con 0x563a9c2e7d80 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd8df400640 10 client.15519 handle_command_reply: tid=0 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 client.15519 shutdown 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 msgr2=0x563a9c6d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d5c00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc018000 tx=0x7fd8fc004bb0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 shutdown_connections 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2e7d80 0x563a9c6e7d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/121844211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d5c00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd8df400640 0 client.15519 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 >> 172.21.15.42:0/121844211 conn(0x563a9c31d300 msgr2=0x563a9c6cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 shutdown_connections 2024-05-27T10:19:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 1 -- 172.21.15.42:0/121844211 wait complete. 2024-05-27T10:19:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.876+0000 7fd90f7c0740 20 client.15519 trim_cache size 0 max 16384 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 Processor -- start 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 -- start start 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001b4330 con 0x563a9c3002f0 2024-05-27T10:19:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001b1c00 con 0x563a9c3291b0 2024-05-27T10:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.894+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001afa50 con 0x563a9c3084d0 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37328/0 (socket says 172.21.15.42:37328) 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 -- 172.21.15.42:0/2483281816 learned_addr learned my addr 172.21.15.42:0/2483281816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd905400640 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 -- 172.21.15.42:0/2483281816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 msgr2=0x563a9c6d5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 -- 172.21.15.42:0/2483281816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 msgr2=0x563a9c6e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.895+0000 7fd906800640 1 -- 172.21.15.42:0/2483281816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9c6d8a70 con 0x563a9c3002f0 2024-05-27T10:19:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.896+0000 7fd906800640 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d8330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc032fd0 tx=0x7fd8fc01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d094fd0a8bf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.896+0000 7fd8ea600640 1 -- 172.21.15.42:0/2483281816 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc015070 con 0x563a9c3002f0 2024-05-27T10:19:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.896+0000 7fd8ea600640 1 -- 172.21.15.42:0/2483281816 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8fc00b9f0 con 0x563a9c3002f0 2024-05-27T10:19:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.896+0000 7fd8ea600640 1 -- 172.21.15.42:0/2483281816 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0326c0 con 0x563a9c3002f0 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 -- 172.21.15.42:0/2483281816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 msgr2=0x563a9c6d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d8330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc032fd0 tx=0x7fd8fc01b040 comp rx=0 tx=0).stop 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 -- 172.21.15.42:0/2483281816 shutdown_connections 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3291b0 0x563a9c6e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3084d0 0x563a9c6d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/2483281816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3002f0 0x563a9c6d8330 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 -- 172.21.15.42:0/2483281816 >> 172.21.15.42:0/2483281816 conn(0x563a9c31d300 msgr2=0x563a9c6cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 -- 172.21.15.42:0/2483281816 shutdown_connections 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.897+0000 7fd90f7c0740 1 -- 172.21.15.42:0/2483281816 wait complete. 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 -- start start 2024-05-27T10:19:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 Processor -- start 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3002f0 0x563a9c6de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 0x563a9c6da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3291b0 0x563a9c6dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001b00c0 con 0x563a9c3084d0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001b1530 con 0x563a9c3002f0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.898+0000 7fd90f7c0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001b0790 con 0x563a9c3291b0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd90f7c0740 10 client.0 _ll_register_callbacks cb 0x7fd8f0005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 0x563a9c6da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 0x563a9c6da910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37334/0 (socket says 172.21.15.42:37334) 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 -- 172.21.15.42:0/265630848 learned_addr learned my addr 172.21.15.42:0/265630848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd906800640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3002f0 0x563a9c6de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd906800640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3291b0 0x563a9c6dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 -- 172.21.15.42:0/265630848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3291b0 msgr2=0x563a9c6dae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3291b0 0x563a9c6dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 -- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3002f0 msgr2=0x563a9c6de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:19:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3002f0 0x563a9c6de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.899+0000 7fd905400640 1 -- 172.21.15.42:0/265630848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9c6db600 con 0x563a9c3084d0 2024-05-27T10:19:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd905400640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 0x563a9c6da910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd90006c5b0 tx=0x7fd8f0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399ff8838469deeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:19:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd8df400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:19:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd8df400640 1 -- 172.21.15.42:0/265630848 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0049070 con 0x563a9c3084d0 2024-05-27T10:19:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd8df400640 1 -- 172.21.15.42:0/265630848 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fd8f00046c0 con 0x563a9c3084d0 2024-05-27T10:19:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8fc00b9f0 con 0x563a9c3084d0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd8df400640 1 -- 172.21.15.42:0/265630848 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd8f0055050 con 0x563a9c3084d0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd90f7c0740 10 client.15525 fetch_fsmap learned FSMap version 26 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.900+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a9c74f2f0 con 0x563a9c3084d0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd8df400640 1 -- 172.21.15.42:0/265630848 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8fc00b9f0 con 0x563a9c3084d0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd8df400640 1 -- 172.21.15.42:0/265630848 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd8f0003b20 con 0x563a9c3084d0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd90f7c0740 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd90f7c0740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd90f7c0740 20 client.15525 populate_metadata read hostname 'smithi042' 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2bd1c0 0x563a9c751a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd90f7c0740 4 client.15525 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.901+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a9c6382c0 con 0x563a9c2bd1c0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.902+0000 7fd906800640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2bd1c0 0x563a9c751a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:19:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.903+0000 7fd906800640 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2bd1c0 0x563a9c751a90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e8e761f7bb58b0 server_cookie=3e5ee3f116133b96 in_seq=0 out_seq=0 2024-05-27T10:19:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.903+0000 7fd8df400640 10 client.15525 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd8df400640 1 -- 172.21.15.42:0/265630848 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563a9c6382c0 con 0x563a9c2bd1c0 2024-05-27T10:19:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd8df400640 10 client.15525 handle_command_reply: tid=0 2024-05-27T10:19:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 client.15525 shutdown 2024-05-27T10:19:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 msgr2=0x563a9c6da910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 0x563a9c6da910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd90006c5b0 tx=0x7fd8f0005fd0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 shutdown_connections 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x563a9c2bd1c0 0x563a9c751a90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563a9c3291b0 0x563a9c6dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563a9c3084d0 0x563a9c6da910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 --2- 172.21.15.42:0/265630848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563a9c3002f0 0x563a9c6de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd8df400640 0 client.15525 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 >> 172.21.15.42:0/265630848 conn(0x563a9c31d300 msgr2=0x563a9c6cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.912+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 shutdown_connections 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.913+0000 7fd90f7c0740 1 -- 172.21.15.42:0/265630848 wait complete. 2024-05-27T10:19:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.913+0000 7fd90f7c0740 20 client.15525 trim_cache size 0 max 16384 2024-05-27T10:19:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:19:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:19:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:19:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.913+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd8ec0b2a70 msgr2=0x7fd8ec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.913+0000 7fd907200640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd8ec0b2a70 0x7fd8ec0b4f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80e0520 tx=0x7fd8f80094a0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.913+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 msgr2=0x7fd9001a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.913+0000 7fd907200640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 0x7fd9001a2590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800eab0 tx=0x7fd8f800ef70 comp rx=0 tx=0).stop 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 shutdown_connections 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd8ec0b2a70 0x7fd8ec0b4f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd900110320 0x7fd90019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90010d9c0 0x7fd9001a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 --2- 172.21.15.42:0/3535938590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd900075420 0x7fd9001a2590 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 >> 172.21.15.42:0/3535938590 conn(0x7fd90007a1b0 msgr2=0x7fd9001021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 shutdown_connections 2024-05-27T10:19:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:19:58.914+0000 7fd907200640 1 -- 172.21.15.42:0/3535938590 wait complete. 2024-05-27T10:20:00.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 -- 172.21.15.42:0/3904636979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a010e8e0 msgr2=0x7f66a0110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 --2- 172.21.15.42:0/3904636979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a010e8e0 0x7f66a0110dc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f669800b0a0 tx=0x7f669800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 -- 172.21.15.42:0/3904636979 shutdown_connections 2024-05-27T10:20:00.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 --2- 172.21.15.42:0/3904636979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a010e8e0 0x7f66a0110dc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 --2- 172.21.15.42:0/3904636979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a0105470 0x7f66a0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 --2- 172.21.15.42:0/3904636979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a0105dc0 0x7f66a010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 -- 172.21.15.42:0/3904636979 >> 172.21.15.42:0/3904636979 conn(0x7f66a00fef20 msgr2=0x7f66a0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:00.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 -- 172.21.15.42:0/3904636979 shutdown_connections 2024-05-27T10:20:00.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.132+0000 7f66a8000700 1 -- 172.21.15.42:0/3904636979 wait complete. 2024-05-27T10:20:00.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:00 smithi042 ceph-mon[31661]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-27T10:20:00.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:00 smithi042 ceph-mon[31661]: overall HEALTH_OK 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 Processor -- start 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 -- start start 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 0x7f66a019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a0105dc0 0x7f66a019d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a010e8e0 0x7f66a01a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01142f0 con 0x7f66a0105470 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0114170 con 0x7f66a010e8e0 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.134+0000 7f66a8000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0114470 con 0x7f66a0105dc0 2024-05-27T10:20:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 0x7f66a019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 0x7f66a019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37346/0 (socket says 172.21.15.42:37346) 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 -- 172.21.15.42:0/2301148627 learned_addr learned my addr 172.21.15.42:0/2301148627 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a4c00700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a0105dc0 0x7f66a019d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a6000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a010e8e0 0x7f66a01a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 -- 172.21.15.42:0/2301148627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a0105dc0 msgr2=0x7f66a019d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a0105dc0 0x7f66a019d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 -- 172.21.15.42:0/2301148627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a010e8e0 msgr2=0x7f66a01a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a010e8e0 0x7f66a01a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a5600700 1 -- 172.21.15.42:0/2301148627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a01a1c20 con 0x7f66a0105470 2024-05-27T10:20:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.135+0000 7f66a6000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a010e8e0 0x7f66a01a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.136+0000 7f66a5600700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 0x7f66a019ce00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f669400c9b0 tx=0x7f669400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.136+0000 7f669e000700 1 -- 172.21.15.42:0/2301148627 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694007b00 con 0x7f66a0105470 2024-05-27T10:20:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.136+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a01a1f10 con 0x7f66a0105470 2024-05-27T10:20:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.136+0000 7f669e000700 1 -- 172.21.15.42:0/2301148627 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6694007ca0 con 0x7f66a0105470 2024-05-27T10:20:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.136+0000 7f669e000700 1 -- 172.21.15.42:0/2301148627 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694018660 con 0x7f66a0105470 2024-05-27T10:20:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.136+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a2450 con 0x7f66a0105470 2024-05-27T10:20:00.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.140+0000 7f669e000700 1 -- 172.21.15.42:0/2301148627 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f669400f450 con 0x7f66a0105470 2024-05-27T10:20:00.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.142+0000 7f66b6bd0b80 1 Processor -- start 2024-05-27T10:20:00.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.142+0000 7f66b6bd0b80 1 -- start start 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e995b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 0x55c7e995b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e9968110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c7e9889e40 con 0x55c7e97dce40 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c7e98799e0 con 0x55c7e995be70 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c7e987da20 con 0x55c7e98c89f0 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f669e000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66880b2b10 0x7f66880b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66a5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 0x55c7e995b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66a5600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 0x55c7e995b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37360/0 (socket says 172.21.15.42:37360) 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66a5600700 1 -- 172.21.15.42:0/3680569661 learned_addr learned my addr 172.21.15.42:0/3680569661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66a6000700 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e995b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f66a6000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66880b2b10 0x7f66880b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.143+0000 7f669e000700 1 -- 172.21.15.42:0/2301148627 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f66940e2990 con 0x7f66a0105470 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a4c00700 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e9968110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a5600700 1 -- 172.21.15.42:0/3680569661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 msgr2=0x55c7e995b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a5600700 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e995b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a5600700 1 -- 172.21.15.42:0/3680569661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 msgr2=0x55c7e9968110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a5600700 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e9968110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a5600700 1 -- 172.21.15.42:0/3680569661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e9842420 con 0x55c7e97dce40 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a5600700 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 0x55c7e995b870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f66940a99a0 tx=0x7f66940775e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1631ff93003c71e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.144+0000 7f66a6000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66880b2b10 0x7f66880b5020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55c7e995dff0 tx=0x7f669800b7b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.145+0000 7f6692a00700 1 -- 172.21.15.42:0/3680569661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940b21b0 con 0x55c7e97dce40 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.145+0000 7f6692a00700 1 -- 172.21.15.42:0/3680569661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66940a9520 con 0x55c7e97dce40 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.145+0000 7f6692a00700 1 -- 172.21.15.42:0/3680569661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940a9e70 con 0x55c7e97dce40 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/3680569661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 msgr2=0x55c7e995b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 0x55c7e995b870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f66940a99a0 tx=0x7f66940775e0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/3680569661 shutdown_connections 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e9968110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e995b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/3680569661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e97dce40 0x55c7e995b870 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/3680569661 >> 172.21.15.42:0/3680569661 conn(0x55c7e9876390 msgr2=0x55c7e99574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/3680569661 shutdown_connections 2024-05-27T10:20:00.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.146+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/3680569661 wait complete. 2024-05-27T10:20:00.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.147+0000 7f66b6bd0b80 1 -- start start 2024-05-27T10:20:00.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.147+0000 7f66b6bd0b80 1 Processor -- start 2024-05-27T10:20:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e99741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e98c89f0 0x55c7e99746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e996e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c7e977ffa0 con 0x55c7e995be70 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c7e981a760 con 0x55c7e98c89f0 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c7e9968b20 con 0x55c7e97dce40 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66b6bd0b80 10 client.0 _ll_register_callbacks cb 0x55c7e9823c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66a6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e996e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66a6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e996e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37366/0 (socket says 172.21.15.42:37366) 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66a6000700 1 -- 172.21.15.42:0/24250373 learned_addr learned my addr 172.21.15.42:0/24250373 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66a4c00700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e98c89f0 0x55c7e99746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.148+0000 7f66a5600700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e99741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f66a6000700 1 -- 172.21.15.42:0/24250373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 msgr2=0x55c7e99741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f66a6000700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e99741b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f66a6000700 1 -- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e98c89f0 msgr2=0x55c7e99746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f66a6000700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e98c89f0 0x55c7e99746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f66a6000700 1 -- 172.21.15.42:0/24250373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e97c98d0 con 0x55c7e995be70 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f66a6000700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e996e5b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6698015000 tx=0x7f66980099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ceed93a963cd1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f6687400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.149+0000 7f6687400700 1 -- 172.21.15.42:0/24250373 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669802f070 con 0x55c7e995be70 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66940a9520 con 0x55c7e995be70 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f6687400700 1 -- 172.21.15.42:0/24250373 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f669802a670 con 0x55c7e995be70 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f6687400700 1 -- 172.21.15.42:0/24250373 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6698024040 con 0x55c7e995be70 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f66b6bd0b80 10 client.15537 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7e984a060 con 0x55c7e995be70 2024-05-27T10:20:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f6687400700 1 -- 172.21.15.42:0/24250373 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66940a9520 con 0x55c7e995be70 2024-05-27T10:20:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.150+0000 7f6687400700 1 -- 172.21.15.42:0/24250373 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6698032020 con 0x55c7e995be70 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66b6bd0b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66b6bd0b80 10 client.15537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66b6bd0b80 20 client.15537 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e997e230 0x55c7e99806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66b6bd0b80 4 client.15537 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7e988bd90 con 0x55c7e997e230 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.151+0000 7f66a5600700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e997e230 0x55c7e99806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.152+0000 7f66a5600700 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e997e230 0x55c7e99806a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b4e9726dc126d00 server_cookie=bf12e46fe65e298f in_seq=0 out_seq=0 2024-05-27T10:20:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.152+0000 7f6687400700 10 client.15537 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:20:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f6687400700 1 -- 172.21.15.42:0/24250373 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c7e988bd90 con 0x55c7e997e230 2024-05-27T10:20:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f6687400700 10 client.15537 handle_command_reply: tid=0 2024-05-27T10:20:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 client.15537 shutdown 2024-05-27T10:20:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 msgr2=0x55c7e996e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e996e5b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6698015000 tx=0x7f66980099e0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 shutdown_connections 2024-05-27T10:20:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e997e230 0x55c7e99806a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e996e5b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e99741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f6687400700 0 client.15537 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/24250373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e98c89f0 0x55c7e99746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 >> 172.21.15.42:0/24250373 conn(0x55c7e9876390 msgr2=0x55c7e995b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 shutdown_connections 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.155+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/24250373 wait complete. 2024-05-27T10:20:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.156+0000 7f66b6bd0b80 20 client.15537 trim_cache size 0 max 16384 2024-05-27T10:20:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 Processor -- start 2024-05-27T10:20:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 -- start start 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e9957660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 0x55c7e9985e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e99863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01af180 con 0x55c7e98c89f0 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01aca10 con 0x55c7e995be70 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66a00512b0 con 0x55c7e97dce40 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66a4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 0x55c7e9985e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.173+0000 7f66a4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 0x55c7e9985e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37372/0 (socket says 172.21.15.42:37372) 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 -- 172.21.15.42:0/747241664 learned_addr learned my addr 172.21.15.42:0/747241664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a6000700 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e99863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a5600700 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e9957660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 -- 172.21.15.42:0/747241664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 msgr2=0x55c7e9957660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e9957660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 -- 172.21.15.42:0/747241664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 msgr2=0x55c7e99863a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e99863a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 -- 172.21.15.42:0/747241664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e99589c0 con 0x55c7e98c89f0 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.174+0000 7f66a4c00700 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 0x55c7e9985e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f668c010c20 tx=0x7f668c010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371edbccb9662c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.175+0000 7f6692000700 1 -- 172.21.15.42:0/747241664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c01b440 con 0x55c7e98c89f0 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.175+0000 7f6692000700 1 -- 172.21.15.42:0/747241664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f668c01a870 con 0x55c7e98c89f0 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.175+0000 7f6692000700 1 -- 172.21.15.42:0/747241664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c00f880 con 0x55c7e98c89f0 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/747241664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 msgr2=0x55c7e9985e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 0x55c7e9985e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f668c010c20 tx=0x7f668c010f30 comp rx=0 tx=0).stop 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/747241664 shutdown_connections 2024-05-27T10:20:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e995be70 0x55c7e99863a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e97dce40 0x55c7e9957660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/747241664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e98c89f0 0x55c7e9985e60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/747241664 >> 172.21.15.42:0/747241664 conn(0x55c7e9876390 msgr2=0x55c7e9956f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/747241664 shutdown_connections 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/747241664 wait complete. 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.176+0000 7f66b6bd0b80 1 -- start start 2024-05-27T10:20:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 Processor -- start 2024-05-27T10:20:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e97dce40 0x55c7e997a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e9975d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e9976250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010b1b0 con 0x55c7e995be70 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ac340 con 0x55c7e97dce40 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ab5c0 con 0x55c7e98c89f0 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.177+0000 7f66b6bd0b80 10 client.0 _ll_register_callbacks cb 0x55c7e9823c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e9976250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e9976250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37380/0 (socket says 172.21.15.42:37380) 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 -- 172.21.15.42:0/1611988904 learned_addr learned my addr 172.21.15.42:0/1611988904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a5600700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e97dce40 0x55c7e997a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a4c00700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e9975d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 -- 172.21.15.42:0/1611988904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 msgr2=0x55c7e9975d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e9975d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 -- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e97dce40 msgr2=0x55c7e997a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e97dce40 0x55c7e997a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 -- 172.21.15.42:0/1611988904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e9976810 con 0x55c7e995be70 2024-05-27T10:20:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a4c00700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e9975d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a5600700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e97dce40 0x55c7e997a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f66a6000700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e9976250 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f669800b3f0 tx=0x7f6698020970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dc2475824c6ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.178+0000 7f6687400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f6687400700 1 -- 172.21.15.42:0/1611988904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698041060 con 0x55c7e995be70 2024-05-27T10:20:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c01a870 con 0x55c7e995be70 2024-05-27T10:20:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f6687400700 1 -- 172.21.15.42:0/1611988904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6698005870 con 0x55c7e995be70 2024-05-27T10:20:00.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f6687400700 1 -- 172.21.15.42:0/1611988904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f669802f070 con 0x55c7e995be70 2024-05-27T10:20:00.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f66b6bd0b80 10 client.15543 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:00.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7e99e8a40 con 0x55c7e995be70 2024-05-27T10:20:00.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.179+0000 7f6687400700 1 -- 172.21.15.42:0/1611988904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f668c01a870 con 0x55c7e995be70 2024-05-27T10:20:00.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f6687400700 1 -- 172.21.15.42:0/1611988904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6698032070 con 0x55c7e995be70 2024-05-27T10:20:00.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f66b6bd0b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f66b6bd0b80 10 client.15543 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f66b6bd0b80 20 client.15543 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e99e9060 0x55c7e99eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f66b6bd0b80 4 client.15543 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.180+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7e9885570 con 0x55c7e99e9060 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.181+0000 7f66a5600700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e99e9060 0x55c7e99eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.181+0000 7f66a5600700 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e99e9060 0x55c7e99eb480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf04666597aeeead server_cookie=b8e72f6b401a826e in_seq=0 out_seq=0 2024-05-27T10:20:00.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.181+0000 7f6687400700 10 client.15543 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:20:00.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f6687400700 1 -- 172.21.15.42:0/1611988904 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7e9885570 con 0x55c7e99e9060 2024-05-27T10:20:00.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f6687400700 10 client.15543 handle_command_reply: tid=0 2024-05-27T10:20:00.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 client.15543 shutdown 2024-05-27T10:20:00.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 msgr2=0x55c7e9976250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e9976250 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f669800b3f0 tx=0x7f6698020970 comp rx=0 tx=0).stop 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 shutdown_connections 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c7e99e9060 0x55c7e99eb480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7e995be70 0x55c7e9976250 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7e97dce40 0x55c7e997a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 --2- 172.21.15.42:0/1611988904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7e98c89f0 0x55c7e9975d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 >> 172.21.15.42:0/1611988904 conn(0x55c7e9876390 msgr2=0x55c7e996caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f6687400700 0 client.15543 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.182+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 shutdown_connections 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.183+0000 7f66b6bd0b80 1 -- 172.21.15.42:0/1611988904 wait complete. 2024-05-27T10:20:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.183+0000 7f66b6bd0b80 20 client.15543 trim_cache size 0 max 16384 2024-05-27T10:20:00.185 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:20:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.183+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66880b2b10 msgr2=0x7f66880b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.183+0000 7f66a8000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66880b2b10 0x7f66880b5020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55c7e995dff0 tx=0x7f669800b7b0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.183+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 msgr2=0x7f66a019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.183+0000 7f66a8000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 0x7f66a019ce00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f669400c9b0 tx=0x7f669400cd70 comp rx=0 tx=0).stop 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 shutdown_connections 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66880b2b10 0x7f66880b5020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66a010e8e0 0x7f66a01a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a0105470 0x7f66a019ce00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 --2- 172.21.15.42:0/2301148627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a0105dc0 0x7f66a019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 >> 172.21.15.42:0/2301148627 conn(0x7f66a00fef20 msgr2=0x7f66a01030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 shutdown_connections 2024-05-27T10:20:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:00.184+0000 7f66a8000700 1 -- 172.21.15.42:0/2301148627 wait complete. 2024-05-27T10:20:00.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:00 smithi143 ceph-mon[35545]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-27T10:20:00.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:00 smithi143 ceph-mon[35545]: overall HEALTH_OK 2024-05-27T10:20:00.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:00 smithi104 ceph-mon[34812]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-27T10:20:00.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:00 smithi104 ceph-mon[34812]: overall HEALTH_OK 2024-05-27T10:20:00.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:20:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:20:00.277+0000 7f9505800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:20:00.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:20:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:20:00.277+0000 7f9505800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:20:00.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:20:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:20:00.277+0000 7f9505800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:20:00.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:20:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:20:00.277+0000 7f9505800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:20:00.594 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:20:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:20:00.277+0000 7f9505800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T10:20:01.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T10:20:01.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 -- 172.21.15.42:0/3182129990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 msgr2=0x7f7f241111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/3182129990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241111c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00b0d0 tx=0x7f7f1c00b3e0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 -- 172.21.15.42:0/3182129990 shutdown_connections 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/3182129990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241111c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/3182129990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f241045e0 0x7f7f241049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/3182129990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f24108f10 0x7f7f24109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 -- 172.21.15.42:0/3182129990 >> 172.21.15.42:0/3182129990 conn(0x7f7f24100030 msgr2=0x7f7f24102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 -- 172.21.15.42:0/3182129990 shutdown_connections 2024-05-27T10:20:01.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.680+0000 7f7f2aa00700 1 -- 172.21.15.42:0/3182129990 wait complete. 2024-05-27T10:20:01.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.682+0000 7f7f2aa00700 1 Processor -- start 2024-05-27T10:20:01.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.682+0000 7f7f2aa00700 1 -- start start 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f2aa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f241045e0 0x7f7f2419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f2aa00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f24108f10 0x7f7f2419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f2aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f2aa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24114390 con 0x7f7f24109860 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f2aa00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24114210 con 0x7f7f24108f10 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f2aa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24114510 con 0x7f7f241045e0 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f28a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f28a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37406/0 (socket says 172.21.15.42:37406) 2024-05-27T10:20:01.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.683+0000 7f7f28a00700 1 -- 172.21.15.42:0/104731320 learned_addr learned my addr 172.21.15.42:0/104731320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:01.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f23400700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f24108f10 0x7f7f2419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f23e00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f241045e0 0x7f7f2419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f28a00700 1 -- 172.21.15.42:0/104731320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f241045e0 msgr2=0x7f7f2419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f28a00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f241045e0 0x7f7f2419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f28a00700 1 -- 172.21.15.42:0/104731320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f24108f10 msgr2=0x7f7f2419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f28a00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f24108f10 0x7f7f2419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f28a00700 1 -- 172.21.15.42:0/104731320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f241a1e70 con 0x7f7f24109860 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f23400700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f24108f10 0x7f7f2419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.684+0000 7f7f28a00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241a17d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007d10 tx=0x7f7f1c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.685+0000 7f7f20c00700 1 -- 172.21.15.42:0/104731320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c01f070 con 0x7f7f24109860 2024-05-27T10:20:01.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.685+0000 7f7f20c00700 1 -- 172.21.15.42:0/104731320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f1c003e10 con 0x7f7f24109860 2024-05-27T10:20:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.685+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241a2100 con 0x7f7f24109860 2024-05-27T10:20:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.685+0000 7f7f20c00700 1 -- 172.21.15.42:0/104731320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c01b820 con 0x7f7f24109860 2024-05-27T10:20:01.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.685+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a25e0 con 0x7f7f24109860 2024-05-27T10:20:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.689+0000 7f7f20c00700 1 -- 172.21.15.42:0/104731320 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f7f1c022070 con 0x7f7f24109860 2024-05-27T10:20:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.690+0000 7f7f39521b80 1 Processor -- start 2024-05-27T10:20:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.690+0000 7f7f39521b80 1 -- start start 2024-05-27T10:20:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.691+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.691+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac61ce0 0x5634cac5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.691+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac5bd60 0x5634cac5c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.691+0000 7f7f39521b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5634cab89df0 con 0x5634caadcdf0 2024-05-27T10:20:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.691+0000 7f7f39521b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5634cab79990 con 0x5634cac61ce0 2024-05-27T10:20:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.691+0000 7f7f39521b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634cab7d9d0 con 0x5634cac5bd60 2024-05-27T10:20:01.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.692+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.692+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37412/0 (socket says 172.21.15.42:37412) 2024-05-27T10:20:01.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.692+0000 7f7f23400700 1 -- 172.21.15.42:0/1495154344 learned_addr learned my addr 172.21.15.42:0/1495154344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:01.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.692+0000 7f7f23e00700 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac61ce0 0x5634cac5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.692+0000 7f7f20c00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7f080b2ba0 0x7f7f080b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.692+0000 7f7f20c00700 1 -- 172.21.15.42:0/104731320 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7f1c00f020 con 0x7f7f24109860 2024-05-27T10:20:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac5bd60 0x5634cac5c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f28a00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7f080b2ba0 0x7f7f080b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 -- 172.21.15.42:0/1495154344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac5bd60 msgr2=0x5634cac5c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac5bd60 0x5634cac5c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 -- 172.21.15.42:0/1495154344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac61ce0 msgr2=0x5634cac5b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac61ce0 0x5634cac5b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 -- 172.21.15.42:0/1495154344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634cab423d0 con 0x5634caadcdf0 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23e00700 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac61ce0 0x5634cac5b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.693+0000 7f7f23400700 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac617a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f10013550 tx=0x7f7f1000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18db4b656736a2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.694+0000 7f7f28a00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7f080b2ba0 0x7f7f080b50b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0e1420 tx=0x7f7f1c00b3e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.694+0000 7f7f19600700 1 -- 172.21.15.42:0/1495154344 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10019030 con 0x5634caadcdf0 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.694+0000 7f7f19600700 1 -- 172.21.15.42:0/1495154344 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f100044b0 con 0x5634caadcdf0 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.694+0000 7f7f19600700 1 -- 172.21.15.42:0/1495154344 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10003dc0 con 0x5634caadcdf0 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 -- 172.21.15.42:0/1495154344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 msgr2=0x5634cac617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac617a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f10013550 tx=0x7f7f1000b6b0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 -- 172.21.15.42:0/1495154344 shutdown_connections 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac5bd60 0x5634cac5c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac617a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 --2- 172.21.15.42:0/1495154344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac61ce0 0x5634cac5b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.695+0000 7f7f39521b80 1 -- 172.21.15.42:0/1495154344 >> 172.21.15.42:0/1495154344 conn(0x5634cab3cf70 msgr2=0x5634cac56880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.696+0000 7f7f39521b80 1 -- 172.21.15.42:0/1495154344 shutdown_connections 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.696+0000 7f7f39521b80 1 -- 172.21.15.42:0/1495154344 wait complete. 2024-05-27T10:20:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.696+0000 7f7f39521b80 1 -- start start 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.697+0000 7f7f39521b80 1 Processor -- start 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.697+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac65510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac65a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac65f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f39521b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5634caa7ff50 con 0x5634caadcdf0 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f39521b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5634cab1a710 con 0x5634cac5bd60 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f39521b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634cac687b0 con 0x5634cac61ce0 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f39521b80 10 client.0 _ll_register_callbacks cb 0x5634cab23c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f23e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac65a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac65510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f23e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac65a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:42750/0 (socket says 172.21.15.42:42750) 2024-05-27T10:20:01.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac65510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37416/0 (socket says 172.21.15.42:37416) 2024-05-27T10:20:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.698+0000 7f7f23e00700 1 -- 172.21.15.42:0/683357001 learned_addr learned my addr 172.21.15.42:0/683357001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f23400700 1 -- 172.21.15.42:0/683357001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 msgr2=0x5634cac65f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f23400700 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac65f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f23400700 1 -- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 msgr2=0x5634cac65a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f23400700 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac65a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f23400700 1 -- 172.21.15.42:0/683357001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634caac9880 con 0x5634caadcdf0 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f23400700 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac65510 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b9e0 tx=0x7f7f10013ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1b5e5335306c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.699+0000 7f7f0e000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.700+0000 7f7f0e000700 1 -- 172.21.15.42:0/683357001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10019040 con 0x5634caadcdf0 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.700+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f100044b0 con 0x5634caadcdf0 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.700+0000 7f7f0e000700 1 -- 172.21.15.42:0/683357001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7f10005450 con 0x5634caadcdf0 2024-05-27T10:20:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.700+0000 7f7f0e000700 1 -- 172.21.15.42:0/683357001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f10005650 con 0x5634caadcdf0 2024-05-27T10:20:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.701+0000 7f7f39521b80 10 client.15561 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.701+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634cab4a010 con 0x5634caadcdf0 2024-05-27T10:20:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.701+0000 7f7f0e000700 1 -- 172.21.15.42:0/683357001 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f100044b0 con 0x5634caadcdf0 2024-05-27T10:20:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.702+0000 7f7f0e000700 1 -- 172.21.15.42:0/683357001 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7f10003b30 con 0x5634caadcdf0 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.703+0000 7f7f39521b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.703+0000 7f7f39521b80 10 client.15561 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.703+0000 7f7f39521b80 20 client.15561 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.703+0000 7f7f39521b80 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cac75300 0x5634cac76770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.703+0000 7f7f39521b80 4 client.15561 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.703+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634cab8bd40 con 0x5634cac75300 2024-05-27T10:20:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.704+0000 7f7f23e00700 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cac75300 0x5634cac76770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.705+0000 7f7f23e00700 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cac75300 0x5634cac76770 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2053a4ea27fb0d server_cookie=1bdfe91b2302d2b2 in_seq=0 out_seq=0 2024-05-27T10:20:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.705+0000 7f7f0e000700 10 client.15561 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f0e000700 1 -- 172.21.15.42:0/683357001 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5634cab8bd40 con 0x5634cac75300 2024-05-27T10:20:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f0e000700 10 client.15561 handle_command_reply: tid=0 2024-05-27T10:20:01.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 client.15561 shutdown 2024-05-27T10:20:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 msgr2=0x5634cac65510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac65510 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b9e0 tx=0x7f7f10013ef0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 shutdown_connections 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cac75300 0x5634cac76770 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac65f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac65510 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 --2- 172.21.15.42:0/683357001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac65a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f0e000700 0 client.15561 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.707+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 >> 172.21.15.42:0/683357001 conn(0x5634cab3cf70 msgr2=0x5634cac56530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.708+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 shutdown_connections 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.708+0000 7f7f39521b80 1 -- 172.21.15.42:0/683357001 wait complete. 2024-05-27T10:20:01.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.708+0000 7f7f39521b80 20 client.15561 trim_cache size 0 max 16384 2024-05-27T10:20:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 Processor -- start 2024-05-27T10:20:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 -- start start 2024-05-27T10:20:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5634cac74ab0 con 0x5634caadcdf0 2024-05-27T10:20:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5634cac75180 con 0x5634cac5bd60 2024-05-27T10:20:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.734+0000 7f7f39521b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634cac75850 con 0x5634cac61ce0 2024-05-27T10:20:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac5b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37424/0 (socket says 172.21.15.42:37424) 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 -- 172.21.15.42:0/341946660 learned_addr learned my addr 172.21.15.42:0/341946660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23e00700 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 -- 172.21.15.42:0/341946660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 msgr2=0x5634cac679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 -- 172.21.15.42:0/341946660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 msgr2=0x5634cac64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac64830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.735+0000 7f7f23400700 1 -- 172.21.15.42:0/341946660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634cac682d0 con 0x5634caadcdf0 2024-05-27T10:20:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.736+0000 7f7f23400700 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac5b820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004140 tx=0x7f7f10005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70603e1e3b9d0a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:01.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.736+0000 7f7f18c00700 1 -- 172.21.15.42:0/341946660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10005450 con 0x5634caadcdf0 2024-05-27T10:20:01.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.736+0000 7f7f18c00700 1 -- 172.21.15.42:0/341946660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f10022070 con 0x5634caadcdf0 2024-05-27T10:20:01.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.736+0000 7f7f18c00700 1 -- 172.21.15.42:0/341946660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10004970 con 0x5634caadcdf0 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 -- 172.21.15.42:0/341946660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 msgr2=0x5634cac5b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac5b820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004140 tx=0x7f7f10005a00 comp rx=0 tx=0).stop 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 -- 172.21.15.42:0/341946660 shutdown_connections 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac5b820 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 --2- 172.21.15.42:0/341946660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 -- 172.21.15.42:0/341946660 >> 172.21.15.42:0/341946660 conn(0x5634cab3cf70 msgr2=0x5634cac5f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 -- 172.21.15.42:0/341946660 shutdown_connections 2024-05-27T10:20:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.738+0000 7f7f39521b80 1 -- 172.21.15.42:0/341946660 wait complete. 2024-05-27T10:20:01.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.739+0000 7f7f39521b80 1 -- start start 2024-05-27T10:20:01.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.739+0000 7f7f39521b80 1 Processor -- start 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.739+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac70c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.739+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac71160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f39521b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac7ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f39521b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5634cac765f0 con 0x5634caadcdf0 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f39521b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5634cac75f20 con 0x5634cac5bd60 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f39521b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634cac76cc0 con 0x5634cac61ce0 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f39521b80 10 client.0 _ll_register_callbacks cb 0x5634cab23c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac70c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac70c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37426/0 (socket says 172.21.15.42:37426) 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 -- 172.21.15.42:0/3632693726 learned_addr learned my addr 172.21.15.42:0/3632693726 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23e00700 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac71160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 -- 172.21.15.42:0/3632693726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 msgr2=0x5634cac7ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac7ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 -- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 msgr2=0x5634cac71160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac71160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.740+0000 7f7f23400700 1 -- 172.21.15.42:0/3632693726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634cac7f120 con 0x5634caadcdf0 2024-05-27T10:20:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.741+0000 7f7f23400700 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac70c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7f10013550 tx=0x7f7f1000b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b76488cf7538ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.741+0000 7f7f0e000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.741+0000 7f7f0e000700 1 -- 172.21.15.42:0/3632693726 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10019070 con 0x5634caadcdf0 2024-05-27T10:20:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.741+0000 7f7f0e000700 1 -- 172.21.15.42:0/3632693726 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7f10003d50 con 0x5634caadcdf0 2024-05-27T10:20:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.741+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f10022070 con 0x5634caadcdf0 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f0e000700 1 -- 172.21.15.42:0/3632693726 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f10046050 con 0x5634caadcdf0 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 10 client.15567 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634cac80380 con 0x5634caadcdf0 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f0e000700 1 -- 172.21.15.42:0/3632693726 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f10022070 con 0x5634caadcdf0 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f0e000700 1 -- 172.21.15.42:0/3632693726 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7f1003f060 con 0x5634caadcdf0 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 10 client.15567 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 20 client.15567 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cace24f0 0x5634cace3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 4 client.15567 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T10:20:01.743 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:01.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.742+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "status"}) -- 0x5634cab85520 con 0x5634cace24f0 2024-05-27T10:20:01.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.743+0000 7f7f23e00700 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cace24f0 0x5634cace3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:01.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.744+0000 7f7f23e00700 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cace24f0 0x5634cace3960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b435400bd72d6d82 server_cookie=897f574fe87f85b5 in_seq=0 out_seq=0 2024-05-27T10:20:01.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.744+0000 7f7f0e000700 10 client.15567 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:01.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f0e000700 1 -- 172.21.15.42:0/3632693726 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5634cab85520 con 0x5634cace24f0 2024-05-27T10:20:01.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f0e000700 10 client.15567 handle_command_reply: tid=0 2024-05-27T10:20:01.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 client.15567 shutdown 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 msgr2=0x5634cac70c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac70c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7f10013550 tx=0x7f7f1000b440 comp rx=0 tx=0).stop 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 shutdown_connections 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5634cace24f0 0x5634cace3960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634cac61ce0 0x5634cac7ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634caadcdf0 0x5634cac70c20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f0e000700 0 client.15567 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 --2- 172.21.15.42:0/3632693726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634cac5bd60 0x5634cac71160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 >> 172.21.15.42:0/3632693726 conn(0x5634cab3cf70 msgr2=0x5634cac60cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.745+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 shutdown_connections 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.746+0000 7f7f39521b80 1 -- 172.21.15.42:0/3632693726 wait complete. 2024-05-27T10:20:01.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.746+0000 7f7f39521b80 20 client.15567 trim_cache size 0 max 16384 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "ff1a20ea-1c0f-11ef-bc9b-c7b262605968", 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 34499, 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 343.55896613300001, 2024-05-27T10:20:01.747 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-05-27T10:20:01.748 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 82, 2024-05-27T10:20:01.748 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 82, 2024-05-27T10:20:01.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 366.86536558400002 2024-05-27T10:20:01.748 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:20:01.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.746+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7f080b2ba0 msgr2=0x7f7f080b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.746+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7f080b2ba0 0x7f7f080b50b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0e1420 tx=0x7f7f1c00b3e0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.746+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 msgr2=0x7f7f241a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.746+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241a17d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007d10 tx=0x7f7f1c007df0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 shutdown_connections 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7f080b2ba0 0x7f7f080b50b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f24109860 0x7f7f241a17d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f241045e0 0x7f7f2419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 --2- 172.21.15.42:0/104731320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f24108f10 0x7f7f2419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:01.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 >> 172.21.15.42:0/104731320 conn(0x7f7f24100030 msgr2=0x7f7f241086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:01.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 shutdown_connections 2024-05-27T10:20:01.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:01.747+0000 7f7f2aa00700 1 -- 172.21.15.42:0/104731320 wait complete. 2024-05-27T10:20:02.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:02 smithi042 ceph-mon[31661]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-27T10:20:02.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:02 smithi042 ceph-mon[31661]: mgrmap e25: x(active, since 78s), standbys: z, y 2024-05-27T10:20:02.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:02 smithi104 ceph-mon[34812]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-27T10:20:02.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:02 smithi104 ceph-mon[34812]: mgrmap e25: x(active, since 78s), standbys: z, y 2024-05-27T10:20:02.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:02 smithi143 ceph-mon[35545]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-27T10:20:02.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:02 smithi143 ceph-mon[35545]: mgrmap e25: x(active, since 78s), standbys: z, y 2024-05-27T10:20:02.844 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-27T10:20:04.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:04 smithi042 ceph-mon[31661]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:04.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:04 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:20:04.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:04 smithi104 ceph-mon[34812]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:04.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:04 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:20:04.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:04 smithi143 ceph-mon[35545]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:04.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:04 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:20:05.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:05 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:20:05.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:05 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:20:05.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:05 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:20:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:05 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:20:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:05 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:20:05.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:05 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 -- 172.21.15.42:0/2193358915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc109860 msgr2=0x7f37bc1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2193358915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc109860 0x7f37bc1111c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b0d0 tx=0x7f37b400b3e0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 -- 172.21.15.42:0/2193358915 shutdown_connections 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2193358915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc109860 0x7f37bc1111c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2193358915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37bc1045e0 0x7f37bc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2193358915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37bc108f10 0x7f37bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.466+0000 7f37c2e00700 1 -- 172.21.15.42:0/2193358915 >> 172.21.15.42:0/2193358915 conn(0x7f37bc100030 msgr2=0x7f37bc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.467+0000 7f37c2e00700 1 -- 172.21.15.42:0/2193358915 shutdown_connections 2024-05-27T10:20:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.467+0000 7f37c2e00700 1 -- 172.21.15.42:0/2193358915 wait complete. 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.468+0000 7f37c2e00700 1 Processor -- start 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.468+0000 7f37c2e00700 1 -- start start 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.468+0000 7f37c2e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 0x7f37bc075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.468+0000 7f37c2e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37bc108f10 0x7f37bc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.468+0000 7f37c2e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37bc109860 0x7f37bc076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37c2e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc114340 con 0x7f37bc1045e0 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37c2e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1141c0 con 0x7f37bc108f10 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37c2e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1144c0 con 0x7f37bc109860 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37bbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 0x7f37bc075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37bbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 0x7f37bc075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59130/0 (socket says 172.21.15.42:59130) 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37bbe00700 1 -- 172.21.15.42:0/2366248413 learned_addr learned my addr 172.21.15.42:0/2366248413 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37c0e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37bc109860 0x7f37bc076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.469+0000 7f37b3400700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37bc108f10 0x7f37bc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.470+0000 7f37bbe00700 1 -- 172.21.15.42:0/2366248413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37bc109860 msgr2=0x7f37bc076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.470+0000 7f37bbe00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37bc109860 0x7f37bc076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.470+0000 7f37bbe00700 1 -- 172.21.15.42:0/2366248413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37bc108f10 msgr2=0x7f37bc075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.470+0000 7f37bbe00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37bc108f10 0x7f37bc075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.470+0000 7f37bbe00700 1 -- 172.21.15.42:0/2366248413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc1a9000 con 0x7f37bc1045e0 2024-05-27T10:20:05.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.470+0000 7f37bbe00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 0x7f37bc075ab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00d980 tx=0x7f37ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:05.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.471+0000 7f37b9600700 1 -- 172.21.15.42:0/2366248413 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac016070 con 0x7f37bc1045e0 2024-05-27T10:20:05.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.471+0000 7f37b9600700 1 -- 172.21.15.42:0/2366248413 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37ac004830 con 0x7f37bc1045e0 2024-05-27T10:20:05.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.471+0000 7f37b9600700 1 -- 172.21.15.42:0/2366248413 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0054c0 con 0x7f37bc1045e0 2024-05-27T10:20:05.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.471+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc1a92f0 con 0x7f37bc1045e0 2024-05-27T10:20:05.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.471+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a97e0 con 0x7f37bc1045e0 2024-05-27T10:20:05.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.476+0000 7f37b9600700 1 -- 172.21.15.42:0/2366248413 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f37ac004220 con 0x7f37bc1045e0 2024-05-27T10:20:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 Processor -- start 2024-05-27T10:20:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 -- start start 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b98330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fa22ac6e40 con 0x55fa22b059f0 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa22ab69e0 con 0x55fa22a19e40 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37b9600700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f37a00b2cf0 0x7f37a00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.477+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fa22abaa20 con 0x55fa22b98e70 2024-05-27T10:20:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37b9600700 1 -- 172.21.15.42:0/2366248413 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f37ac0e2480 con 0x7f37bc1045e0 2024-05-27T10:20:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b98330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b98330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59140/0 (socket says 172.21.15.42:59140) 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 -- 172.21.15.42:0/998187757 learned_addr learned my addr 172.21.15.42:0/998187757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37bbe00700 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37b3400700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f37a00b2cf0 0x7f37a00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 -- 172.21.15.42:0/998187757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 msgr2=0x55fa22ba5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 -- 172.21.15.42:0/998187757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 msgr2=0x55fa22b98870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.478+0000 7f37c0e00700 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b98870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.479+0000 7f37c0e00700 1 -- 172.21.15.42:0/998187757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa22a7f420 con 0x55fa22b059f0 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.479+0000 7f37c0e00700 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b98330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009290 tx=0x7f37b4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa40cbd65f47a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:05.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.479+0000 7f37b3400700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f37a00b2cf0 0x7f37a00b5200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37bc103af0 tx=0x7f37a400a380 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.479+0000 7f37b1600700 1 -- 172.21.15.42:0/998187757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4003c00 con 0x55fa22b059f0 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.479+0000 7f37b1600700 1 -- 172.21.15.42:0/998187757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b4026070 con 0x55fa22b059f0 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.479+0000 7f37b1600700 1 -- 172.21.15.42:0/998187757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4005370 con 0x55fa22b059f0 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 -- 172.21.15.42:0/998187757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 msgr2=0x55fa22b98330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b98330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009290 tx=0x7f37b4009800 comp rx=0 tx=0).stop 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 -- 172.21.15.42:0/998187757 shutdown_connections 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b98330 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/998187757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b98870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.480+0000 7f37d1a42b80 1 -- 172.21.15.42:0/998187757 >> 172.21.15.42:0/998187757 conn(0x55fa22ab3390 msgr2=0x55fa22b944c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.481+0000 7f37d1a42b80 1 -- 172.21.15.42:0/998187757 shutdown_connections 2024-05-27T10:20:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.481+0000 7f37d1a42b80 1 -- 172.21.15.42:0/998187757 wait complete. 2024-05-27T10:20:05.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.481+0000 7f37d1a42b80 1 -- start start 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.481+0000 7f37d1a42b80 1 Processor -- start 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22ba26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b059f0 0x55fa22ba2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fa229bcfa0 con 0x55fa22a19e40 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa22a57760 con 0x55fa22b059f0 2024-05-27T10:20:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fa22ba5b20 con 0x55fa22b98e70 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.482+0000 7f37d1a42b80 10 client.0 _ll_register_callbacks cb 0x55fa22a60c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22ba26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22ba26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59154/0 (socket says 172.21.15.42:59154) 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 -- 172.21.15.42:0/441287713 learned_addr learned my addr 172.21.15.42:0/441287713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37b3400700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b059f0 0x55fa22ba2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37c0e00700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 -- 172.21.15.42:0/441287713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 msgr2=0x55fa22ba3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 -- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b059f0 msgr2=0x55fa22ba2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b059f0 0x55fa22ba2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37bbe00700 1 -- 172.21.15.42:0/441287713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa22a068d0 con 0x55fa22a19e40 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.483+0000 7f37b3400700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b059f0 0x55fa22ba2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37bbe00700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22ba26c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0a67e0 tx=0x7f37ac0a91b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d6f01042fa8fc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:05.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37aa000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37aa000700 1 -- 172.21.15.42:0/441287713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0c3e60 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37aa000700 1 -- 172.21.15.42:0/441287713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f37ac004cc0 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b4026070 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37aa000700 1 -- 172.21.15.42:0/441287713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37ac0c7020 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37d1a42b80 10 client.15579 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa22a87060 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.484+0000 7f37aa000700 1 -- 172.21.15.42:0/441287713 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b4026070 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37aa000700 1 -- 172.21.15.42:0/441287713 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f37ac0b1f40 con 0x55fa22a19e40 2024-05-27T10:20:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37d1a42b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37d1a42b80 10 client.15579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37d1a42b80 20 client.15579 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22bb2a30 0x55fa22bb4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37d1a42b80 4 client.15579 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa22ac8d90 con 0x55fa22bb2a30 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.485+0000 7f37b3400700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22bb2a30 0x55fa22bb4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.486+0000 7f37b3400700 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22bb2a30 0x55fa22bb4e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fab3dcac01eda103 server_cookie=14dec42806c5829f in_seq=0 out_seq=0 2024-05-27T10:20:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.486+0000 7f37aa000700 10 client.15579 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:20:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.488+0000 7f37aa000700 1 -- 172.21.15.42:0/441287713 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fa22ac8d90 con 0x55fa22bb2a30 2024-05-27T10:20:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.488+0000 7f37aa000700 10 client.15579 handle_command_reply: tid=0 2024-05-27T10:20:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 client.15579 shutdown 2024-05-27T10:20:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 msgr2=0x55fa22ba26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22ba26c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0a67e0 tx=0x7f37ac0a91b0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 shutdown_connections 2024-05-27T10:20:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22bb2a30 0x55fa22bb4e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22ba26c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/441287713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b059f0 0x55fa22ba2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 >> 172.21.15.42:0/441287713 conn(0x55fa22ab3390 msgr2=0x55fa22b983b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37aa000700 0 client.15579 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:20:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 shutdown_connections 2024-05-27T10:20:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 1 -- 172.21.15.42:0/441287713 wait complete. 2024-05-27T10:20:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.489+0000 7f37d1a42b80 20 client.15579 trim_cache size 0 max 16384 2024-05-27T10:20:05.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.506+0000 7f37d1a42b80 1 Processor -- start 2024-05-27T10:20:05.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.506+0000 7f37d1a42b80 1 -- start start 2024-05-27T10:20:05.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22b9de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b059f0 0x55fa22b9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b98e70 0x55fa22bba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b35b0 con 0x55fa22a19e40 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b0e40 con 0x55fa22b98e70 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1aec90 con 0x55fa22b059f0 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37bbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22b9de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37bbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22b9de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59160/0 (socket says 172.21.15.42:59160) 2024-05-27T10:20:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.507+0000 7f37bbe00700 1 -- 172.21.15.42:0/3783064007 learned_addr learned my addr 172.21.15.42:0/3783064007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37bbe00700 1 -- 172.21.15.42:0/3783064007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b059f0 msgr2=0x55fa22b9a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37b3400700 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b059f0 0x55fa22b9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37bbe00700 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b059f0 0x55fa22b9a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37c0e00700 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b98e70 0x55fa22bba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37bbe00700 1 -- 172.21.15.42:0/3783064007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b98e70 msgr2=0x55fa22bba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37bbe00700 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b98e70 0x55fa22bba650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37bbe00700 1 -- 172.21.15.42:0/3783064007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa22b95000 con 0x55fa22a19e40 2024-05-27T10:20:05.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37b3400700 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b059f0 0x55fa22b9a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37bbe00700 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22b9de90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00d950 tx=0x7f37ac0a96f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03944cc5ec59ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.508+0000 7f37b0c00700 1 -- 172.21.15.42:0/3783064007 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0c3e60 con 0x55fa22a19e40 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.509+0000 7f37b0c00700 1 -- 172.21.15.42:0/3783064007 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37ac0b23d0 con 0x55fa22a19e40 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.509+0000 7f37b0c00700 1 -- 172.21.15.42:0/3783064007 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0b1f40 con 0x55fa22a19e40 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.509+0000 7f37d1a42b80 1 -- 172.21.15.42:0/3783064007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 msgr2=0x55fa22b9de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.509+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22b9de90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00d950 tx=0x7f37ac0a96f0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 -- 172.21.15.42:0/3783064007 shutdown_connections 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22b98e70 0x55fa22bba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22a19e40 0x55fa22b9de90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/3783064007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b059f0 0x55fa22b9a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 -- 172.21.15.42:0/3783064007 >> 172.21.15.42:0/3783064007 conn(0x55fa22ab3390 msgr2=0x55fa22b99ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 -- 172.21.15.42:0/3783064007 shutdown_connections 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 -- 172.21.15.42:0/3783064007 wait complete. 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 -- start start 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.510+0000 7f37d1a42b80 1 Processor -- start 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b9ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1af300 con 0x55fa22b059f0 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b0770 con 0x55fa22a19e40 2024-05-27T10:20:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1af9d0 con 0x55fa22b98e70 2024-05-27T10:20:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37d1a42b80 10 client.0 _ll_register_callbacks cb 0x55fa22a60c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37b3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b9ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37b3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b9ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59168/0 (socket says 172.21.15.42:59168) 2024-05-27T10:20:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.511+0000 7f37b3400700 1 -- 172.21.15.42:0/501288273 learned_addr learned my addr 172.21.15.42:0/501288273 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:05.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37bbe00700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37c0e00700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37b3400700 1 -- 172.21.15.42:0/501288273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 msgr2=0x55fa22ba9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37b3400700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37b3400700 1 -- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 msgr2=0x55fa22b9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37b3400700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b9c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37b3400700 1 -- 172.21.15.42:0/501288273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa22b9d5b0 con 0x55fa22b059f0 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37bbe00700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37b3400700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b9ccb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55fa22b980f0 tx=0x7f37a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4730273e3801618a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.512+0000 7f37aa000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37aa000700 1 -- 172.21.15.42:0/501288273 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4016070 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37aa000700 1 -- 172.21.15.42:0/501288273 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f37a4027330 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ac0b23d0 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37aa000700 1 -- 172.21.15.42:0/501288273 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37a4034020 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37d1a42b80 10 client.15585 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa22bab3a0 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.513+0000 7f37aa000700 1 -- 172.21.15.42:0/501288273 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37ac0b23d0 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37aa000700 1 -- 172.21.15.42:0/501288273 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f37a4019050 con 0x55fa22b059f0 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37d1a42b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37d1a42b80 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37d1a42b80 20 client.15585 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22c1d520 0x55fa22c1f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37d1a42b80 4 client.15585 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:20:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa22ac2570 con 0x55fa22c1d520 2024-05-27T10:20:05.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.514+0000 7f37bbe00700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22c1d520 0x55fa22c1f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.515+0000 7f37bbe00700 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22c1d520 0x55fa22c1f940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d6d8a356f65b094 server_cookie=9558021934860c9b in_seq=0 out_seq=0 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.515+0000 7f37aa000700 10 client.15585 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37aa000700 1 -- 172.21.15.42:0/501288273 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa22ac2570 con 0x55fa22c1d520 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37aa000700 10 client.15585 handle_command_reply: tid=0 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 client.15585 shutdown 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 msgr2=0x55fa22b9ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b9ccb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55fa22b980f0 tx=0x7f37a40047c0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 shutdown_connections 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55fa22c1d520 0x55fa22c1f940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa22b98e70 0x55fa22ba9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa22a19e40 0x55fa22b9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 --2- 172.21.15.42:0/501288273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa22b059f0 0x55fa22b9ccb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37aa000700 0 client.15585 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 >> 172.21.15.42:0/501288273 conn(0x55fa22ab3390 msgr2=0x55fa22b94400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 shutdown_connections 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 1 -- 172.21.15.42:0/501288273 wait complete. 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.516+0000 7f37d1a42b80 20 client.15585 trim_cache size 0 max 16384 2024-05-27T10:20:05.520 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f37a00b2cf0 msgr2=0x7f37a00b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f37a00b2cf0 0x7f37a00b5200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37bc103af0 tx=0x7f37a400a380 comp rx=0 tx=0).stop 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 msgr2=0x7f37bc075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 0x7f37bc075ab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00d980 tx=0x7f37ac00dc90 comp rx=0 tx=0).stop 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 shutdown_connections 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f37a00b2cf0 0x7f37a00b5200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37bc109860 0x7f37bc076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37bc1045e0 0x7f37bc075ab0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 --2- 172.21.15.42:0/2366248413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37bc108f10 0x7f37bc075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.517+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 >> 172.21.15.42:0/2366248413 conn(0x7f37bc100030 msgr2=0x7f37bc108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.518+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 shutdown_connections 2024-05-27T10:20:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:05.518+0000 7f37c2e00700 1 -- 172.21.15.42:0/2366248413 wait complete. 2024-05-27T10:20:05.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:05 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:20:05.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:05 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:20:05.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:05 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:20:06.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T10:20:06.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:06 smithi042 ceph-mon[31661]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T10:20:06.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:06 smithi104 ceph-mon[34812]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T10:20:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:07.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:06 smithi143 ceph-mon[35545]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T10:20:08.646 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:08 smithi104 ceph-mon[34812]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-27T10:20:08.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:08 smithi042 ceph-mon[31661]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-27T10:20:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:08 smithi143 ceph-mon[35545]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-27T10:20:09.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 -- 172.21.15.42:0/2878667249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 msgr2=0x7ff4fc10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 --2- 172.21.15.42:0/2878667249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc10b350 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80099c0 tx=0x7ff4e8009cd0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 -- 172.21.15.42:0/2878667249 shutdown_connections 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 --2- 172.21.15.42:0/2878667249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc10b890 0x7ff4fc1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 --2- 172.21.15.42:0/2878667249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4fc106610 0x7ff4fc106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 --2- 172.21.15.42:0/2878667249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc10b350 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 -- 172.21.15.42:0/2878667249 >> 172.21.15.42:0/2878667249 conn(0x7ff4fc102020 msgr2=0x7ff4fc104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 -- 172.21.15.42:0/2878667249 shutdown_connections 2024-05-27T10:20:09.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.649+0000 7ff502e00700 1 -- 172.21.15.42:0/2878667249 wait complete. 2024-05-27T10:20:09.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.651+0000 7ff502e00700 1 Processor -- start 2024-05-27T10:20:09.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.651+0000 7ff502e00700 1 -- start start 2024-05-27T10:20:09.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc106610 0x7ff4fc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4fc10b890 0x7ff4fc1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc116370 con 0x7ff4fc10af40 2024-05-27T10:20:09.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1161f0 con 0x7ff4fc10b890 2024-05-27T10:20:09.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1164f0 con 0x7ff4fc106610 2024-05-27T10:20:09.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff501000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff501000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc19d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59176/0 (socket says 172.21.15.42:59176) 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff501000700 1 -- 172.21.15.42:0/3568525770 learned_addr learned my addr 172.21.15.42:0/3568525770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff501a00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc106610 0x7ff4fc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.652+0000 7ff502400700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4fc10b890 0x7ff4fc1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501000700 1 -- 172.21.15.42:0/3568525770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc106610 msgr2=0x7ff4fc19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501000700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc106610 0x7ff4fc19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501000700 1 -- 172.21.15.42:0/3568525770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4fc10b890 msgr2=0x7ff4fc1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501000700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4fc10b890 0x7ff4fc1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501000700 1 -- 172.21.15.42:0/3568525770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc1a1c50 con 0x7ff4fc10af40 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501a00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc106610 0x7ff4fc19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:09.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff501000700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc19d320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800b580 tx=0x7ff4e800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:09.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.653+0000 7ff4fa000700 1 -- 172.21.15.42:0/3568525770 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8026070 con 0x7ff4fc10af40 2024-05-27T10:20:09.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.654+0000 7ff4fa000700 1 -- 172.21.15.42:0/3568525770 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e8005030 con 0x7ff4fc10af40 2024-05-27T10:20:09.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.654+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc1a1ee0 con 0x7ff4fc10af40 2024-05-27T10:20:09.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.654+0000 7ff4fa000700 1 -- 172.21.15.42:0/3568525770 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80049f0 con 0x7ff4fc10af40 2024-05-27T10:20:09.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.654+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc1a23c0 con 0x7ff4fc10af40 2024-05-27T10:20:09.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.658+0000 7ff4fa000700 1 -- 172.21.15.42:0/3568525770 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7ff4e8029070 con 0x7ff4fc10af40 2024-05-27T10:20:09.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.659+0000 7ff512f7cb80 1 Processor -- start 2024-05-27T10:20:09.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.659+0000 7ff512f7cb80 1 -- start start 2024-05-27T10:20:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e0375ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 0x5616e0380ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376820 0x5616e0383020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5616e02a4e40 con 0x5616e0376410 2024-05-27T10:20:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616e02949e0 con 0x5616e01f7e40 2024-05-27T10:20:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616e0298a20 con 0x5616e0376820 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff4fa000700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff4e40b2ba0 0x7ff4e40b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff501a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 0x5616e0380ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff501a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 0x5616e0380ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59186/0 (socket says 172.21.15.42:59186) 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff501a00700 1 -- 172.21.15.42:0/4210375329 learned_addr learned my addr 172.21.15.42:0/4210375329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff4fa000700 1 -- 172.21.15.42:0/3568525770 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff4e80e3390 con 0x7ff4fc10af40 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff501000700 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376820 0x5616e0383020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.660+0000 7ff502400700 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e0375ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff502400700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff4e40b2ba0 0x7ff4e40b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff501a00700 1 -- 172.21.15.42:0/4210375329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376820 msgr2=0x5616e0383020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff501a00700 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376820 0x5616e0383020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff501a00700 1 -- 172.21.15.42:0/4210375329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 msgr2=0x5616e0375ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff501a00700 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e0375ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff501a00700 1 -- 172.21.15.42:0/4210375329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e025d420 con 0x5616e0376410 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.661+0000 7ff501a00700 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 0x5616e0380ad0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000bd90 tx=0x7ff4f000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b944c3fe3e2eae4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.662+0000 7ff502400700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff4e40b2ba0 0x7ff4e40b50b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5616e0372690 tx=0x7ff4f4009680 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.662+0000 7ff4eea00700 1 -- 172.21.15.42:0/4210375329 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000fda0 con 0x5616e0376410 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.662+0000 7ff4eea00700 1 -- 172.21.15.42:0/4210375329 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4f00118d0 con 0x5616e0376410 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.662+0000 7ff4eea00700 1 -- 172.21.15.42:0/4210375329 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0010660 con 0x5616e0376410 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 -- 172.21.15.42:0/4210375329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 msgr2=0x5616e0380ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 0x5616e0380ad0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000bd90 tx=0x7ff4f000f640 comp rx=0 tx=0).stop 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 -- 172.21.15.42:0/4210375329 shutdown_connections 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376820 0x5616e0383020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e0375ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/4210375329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376410 0x5616e0380ad0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 -- 172.21.15.42:0/4210375329 >> 172.21.15.42:0/4210375329 conn(0x5616e0257fc0 msgr2=0x5616e0371d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 -- 172.21.15.42:0/4210375329 shutdown_connections 2024-05-27T10:20:09.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.663+0000 7ff512f7cb80 1 -- 172.21.15.42:0/4210375329 wait complete. 2024-05-27T10:20:09.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.664+0000 7ff512f7cb80 1 -- start start 2024-05-27T10:20:09.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.664+0000 7ff512f7cb80 1 Processor -- start 2024-05-27T10:20:09.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.664+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e01f7e40 0x5616e0380520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.664+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376410 0x5616e0380a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e0380fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5616e019afa0 con 0x5616e0376820 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616e0235760 con 0x5616e0376410 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616e0383990 con 0x5616e01f7e40 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff512f7cb80 10 client.0 _ll_register_callbacks cb 0x5616e023ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e0380fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e0380fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59194/0 (socket says 172.21.15.42:59194) 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 -- 172.21.15.42:0/18560139 learned_addr learned my addr 172.21.15.42:0/18560139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff501a00700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e01f7e40 0x5616e0380520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff501000700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376410 0x5616e0380a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 -- 172.21.15.42:0/18560139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e01f7e40 msgr2=0x5616e0380520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e01f7e40 0x5616e0380520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 -- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376410 msgr2=0x5616e0380a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376410 0x5616e0380a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.665+0000 7ff502400700 1 -- 172.21.15.42:0/18560139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e01e48d0 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff502400700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e0380fa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4018ca0 tx=0x7ff4f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ad16fc76cf9837 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff4e3400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff4e3400700 1 -- 172.21.15.42:0/18560139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f402b060 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff4e3400700 1 -- 172.21.15.42:0/18560139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff4f4012040 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f00118d0 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff4e3400700 1 -- 172.21.15.42:0/18560139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4f4018e40 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff512f7cb80 10 client.15597 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.666+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5616e0265060 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff4e3400700 1 -- 172.21.15.42:0/18560139 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4f00118d0 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff4e3400700 1 -- 172.21.15.42:0/18560139 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff4f4030a90 con 0x5616e0376820 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff512f7cb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:09.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff512f7cb80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:09.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff512f7cb80 20 client.15597 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:09.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e0390980 0x5616e0392e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff512f7cb80 4 client.15597 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:09.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.667+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616e02a6d90 con 0x5616e0390980 2024-05-27T10:20:09.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.668+0000 7ff501a00700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e0390980 0x5616e0392e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.669+0000 7ff501a00700 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e0390980 0x5616e0392e30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c2c19d5160f1ac server_cookie=40326effdd621ccd in_seq=0 out_seq=0 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.669+0000 7ff4e3400700 10 client.15597 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff4e3400700 1 -- 172.21.15.42:0/18560139 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5616e02a6d90 con 0x5616e0390980 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff4e3400700 10 client.15597 handle_command_reply: tid=0 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 client.15597 shutdown 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 msgr2=0x5616e0380fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e0380fa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4018ca0 tx=0x7ff4f4004ad0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 shutdown_connections 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e0390980 0x5616e0392e30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e0380fa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e01f7e40 0x5616e0380520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/18560139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376410 0x5616e0380a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 >> 172.21.15.42:0/18560139 conn(0x5616e0257fc0 msgr2=0x5616e0370ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff4e3400700 0 client.15597 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 shutdown_connections 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 1 -- 172.21.15.42:0/18560139 wait complete. 2024-05-27T10:20:09.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.672+0000 7ff512f7cb80 20 client.15597 trim_cache size 0 max 16384 2024-05-27T10:20:09.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.689+0000 7ff512f7cb80 1 Processor -- start 2024-05-27T10:20:09.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 -- start start 2024-05-27T10:20:09.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e037a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0382990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e038ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b34e0 con 0x5616e0376820 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b0d70 con 0x5616e01f7e40 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1aebe0 con 0x5616e0376410 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff502400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e038ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff502400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e038ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59208/0 (socket says 172.21.15.42:59208) 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.690+0000 7ff502400700 1 -- 172.21.15.42:0/764364855 learned_addr learned my addr 172.21.15.42:0/764364855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff501a00700 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e037a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff501000700 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0382990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff502400700 1 -- 172.21.15.42:0/764364855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 msgr2=0x5616e0382990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff502400700 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0382990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff502400700 1 -- 172.21.15.42:0/764364855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 msgr2=0x5616e037a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff502400700 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e037a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff502400700 1 -- 172.21.15.42:0/764364855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e0373350 con 0x5616e0376820 2024-05-27T10:20:09.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff501a00700 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e037a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:09.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.691+0000 7ff502400700 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e038ca60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40267b0 tx=0x7ff4f4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927d74708adda91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:09.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.692+0000 7ff4ee000700 1 -- 172.21.15.42:0/764364855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f403d070 con 0x5616e0376820 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.692+0000 7ff4ee000700 1 -- 172.21.15.42:0/764364855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4f4005c60 con 0x5616e0376820 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.692+0000 7ff4ee000700 1 -- 172.21.15.42:0/764364855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4026b20 con 0x5616e0376820 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 -- 172.21.15.42:0/764364855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 msgr2=0x5616e038ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e038ca60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40267b0 tx=0x7ff4f4009340 comp rx=0 tx=0).stop 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 -- 172.21.15.42:0/764364855 shutdown_connections 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e0376820 0x5616e038ca60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e01f7e40 0x5616e037a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/764364855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0382990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 -- 172.21.15.42:0/764364855 >> 172.21.15.42:0/764364855 conn(0x5616e0257fc0 msgr2=0x5616e037ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 -- 172.21.15.42:0/764364855 shutdown_connections 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.693+0000 7ff512f7cb80 1 -- 172.21.15.42:0/764364855 wait complete. 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.694+0000 7ff512f7cb80 1 -- start start 2024-05-27T10:20:09.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.694+0000 7ff512f7cb80 1 Processor -- start 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.694+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 0x5616e0383cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.694+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0384230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.694+0000 7ff512f7cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376820 0x5616e03f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1af230 con 0x5616e01f7e40 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b06a0 con 0x5616e0376820 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff512f7cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1af900 con 0x5616e0376410 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff512f7cb80 10 client.0 _ll_register_callbacks cb 0x5616e023ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 0x5616e0383cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 0x5616e0383cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59218/0 (socket says 172.21.15.42:59218) 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 -- 172.21.15.42:0/2378915454 learned_addr learned my addr 172.21.15.42:0/2378915454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501000700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0384230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff502400700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376820 0x5616e03f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 -- 172.21.15.42:0/2378915454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 msgr2=0x5616e0384230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0384230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 -- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376820 msgr2=0x5616e03f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376820 0x5616e03f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.695+0000 7ff501a00700 1 -- 172.21.15.42:0/2378915454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e0381310 con 0x5616e01f7e40 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff501a00700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 0x5616e0383cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x5616e0373050 tx=0x7ff4f000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03572c86932db75 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff4e3400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff4e3400700 1 -- 172.21.15.42:0/2378915454 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0004250 con 0x5616e01f7e40 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff4e3400700 1 -- 172.21.15.42:0/2378915454 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff4f00043f0 con 0x5616e01f7e40 2024-05-27T10:20:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f4005c60 con 0x5616e01f7e40 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff4e3400700 1 -- 172.21.15.42:0/2378915454 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4f000fb40 con 0x5616e01f7e40 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff512f7cb80 10 client.15603 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.696+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5616e03fa880 con 0x5616e01f7e40 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff4e3400700 1 -- 172.21.15.42:0/2378915454 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4f4005c60 con 0x5616e01f7e40 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff4e3400700 1 -- 172.21.15.42:0/2378915454 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff4f00048d0 con 0x5616e01f7e40 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff512f7cb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff512f7cb80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:09.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff512f7cb80 20 client.15603 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:09.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e03fae80 0x5616e03fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:09.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff512f7cb80 4 client.15603 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:20:09.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.697+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616e02a0570 con 0x5616e03fae80 2024-05-27T10:20:09.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.698+0000 7ff501000700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e03fae80 0x5616e03fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.699+0000 7ff501000700 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e03fae80 0x5616e03fd2a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2563d51f64eb219c server_cookie=27e98c510c50aea7 in_seq=0 out_seq=0 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.699+0000 7ff4e3400700 10 client.15603 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.699+0000 7ff4e3400700 1 -- 172.21.15.42:0/2378915454 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616e02a0570 con 0x5616e03fae80 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.699+0000 7ff4e3400700 10 client.15603 handle_command_reply: tid=0 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.699+0000 7ff512f7cb80 1 client.15603 shutdown 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 msgr2=0x5616e0383cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 0x5616e0383cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x5616e0373050 tx=0x7ff4f000b8f0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 shutdown_connections 2024-05-27T10:20:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616e03fae80 0x5616e03fd2a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616e0376820 0x5616e03f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616e01f7e40 0x5616e0383cf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 --2- 172.21.15.42:0/2378915454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e0376410 0x5616e0384230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff4e3400700 0 client.15603 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 >> 172.21.15.42:0/2378915454 conn(0x5616e0257fc0 msgr2=0x5616e0380300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 shutdown_connections 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 1 -- 172.21.15.42:0/2378915454 wait complete. 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.700+0000 7ff512f7cb80 20 client.15603 trim_cache size 0 max 16384 2024-05-27T10:20:09.705 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff4e40b2ba0 msgr2=0x7ff4e40b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff4e40b2ba0 0x7ff4e40b50b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5616e0372690 tx=0x7ff4f4009680 comp rx=0 tx=0).stop 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 msgr2=0x7ff4fc19d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc19d320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800b580 tx=0x7ff4e800f510 comp rx=0 tx=0).stop 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 shutdown_connections 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff4e40b2ba0 0x7ff4e40b50b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4fc10b890 0x7ff4fc1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4fc106610 0x7ff4fc19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 --2- 172.21.15.42:0/3568525770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4fc10af40 0x7ff4fc19d320 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 >> 172.21.15.42:0/3568525770 conn(0x7ff4fc102020 msgr2=0x7ff4fc10c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 shutdown_connections 2024-05-27T10:20:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:09.701+0000 7ff502e00700 1 -- 172.21.15.42:0/3568525770 wait complete. 2024-05-27T10:20:10.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:10 smithi042 ceph-mon[31661]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:20:10.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:10 smithi104 ceph-mon[34812]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:20:11.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:10 smithi143 ceph-mon[35545]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:20:11.871 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T10:20:11.871 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T10:20:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 -- 172.21.15.42:0/339385495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 msgr2=0x7f457010fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 --2- 172.21.15.42:0/339385495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f457010fdb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f455c009900 tx=0x7f455c02a480 comp rx=0 tx=0).stop 2024-05-27T10:20:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 -- 172.21.15.42:0/339385495 shutdown_connections 2024-05-27T10:20:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 --2- 172.21.15.42:0/339385495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 0x7f4570112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 --2- 172.21.15.42:0/339385495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f457010fdb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 --2- 172.21.15.42:0/339385495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4570075480 0x7f4570075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 -- 172.21.15.42:0/339385495 >> 172.21.15.42:0/339385495 conn(0x7f4570079240 msgr2=0x7f4570079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 -- 172.21.15.42:0/339385495 shutdown_connections 2024-05-27T10:20:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.139+0000 7f4576a00640 1 -- 172.21.15.42:0/339385495 wait complete. 2024-05-27T10:20:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.140+0000 7f4576a00640 1 Processor -- start 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.140+0000 7f4576a00640 1 -- start start 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4576a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4570075480 0x7f45701a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4576a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f45701a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4576a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 0x7f457019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4576a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4570115d30 con 0x7f457010d990 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4576a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4570115bb0 con 0x7f45701102f0 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4576a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4570115eb0 con 0x7f4570075480 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4574c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f45701a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4574c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f45701a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59236/0 (socket says 172.21.15.42:59236) 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4574c00640 1 -- 172.21.15.42:0/1592880153 learned_addr learned my addr 172.21.15.42:0/1592880153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.141+0000 7f4575600640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4570075480 0x7f45701a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4576000640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 0x7f457019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4574c00640 1 -- 172.21.15.42:0/1592880153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4570075480 msgr2=0x7f45701a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4574c00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4570075480 0x7f45701a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4574c00640 1 -- 172.21.15.42:0/1592880153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 msgr2=0x7f457019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4574c00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 0x7f457019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4574c00640 1 -- 172.21.15.42:0/1592880153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457019ce80 con 0x7f457010d990 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4576000640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 0x7f457019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f4574c00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f45701a2b10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f455c005940 tx=0x7f455c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.142+0000 7f456e000640 1 -- 172.21.15.42:0/1592880153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c03f030 con 0x7f457010d990 2024-05-27T10:20:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.143+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457019d0b0 con 0x7f457010d990 2024-05-27T10:20:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.143+0000 7f456e000640 1 -- 172.21.15.42:0/1592880153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f455c003c90 con 0x7f457010d990 2024-05-27T10:20:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.143+0000 7f456e000640 1 -- 172.21.15.42:0/1592880153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c035070 con 0x7f457010d990 2024-05-27T10:20:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.143+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457019d5a0 con 0x7f457010d990 2024-05-27T10:20:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.147+0000 7f457f004740 1 Processor -- start 2024-05-27T10:20:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 -- start start 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d5aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f54398f2f0 0x55f543d5afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f543ccbb90 con 0x55f5439974d0 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f543cbb730 con 0x55f5439b81b0 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f457f004740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f543cbf770 con 0x55f54398f2f0 2024-05-27T10:20:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.148+0000 7f456e000640 1 -- 172.21.15.42:0/1592880153 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f455c042030 con 0x7f457010d990 2024-05-27T10:20:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4576000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d5aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d5b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59250/0 (socket says 172.21.15.42:59250) 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 -- 172.21.15.42:0/3014692707 learned_addr learned my addr 172.21.15.42:0/3014692707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4575600640 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f54398f2f0 0x55f543d5afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 -- 172.21.15.42:0/3014692707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f54398f2f0 msgr2=0x55f543d5afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f54398f2f0 0x55f543d5afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 -- 172.21.15.42:0/3014692707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 msgr2=0x55f543d5aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d5aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.149+0000 7f4574c00640 1 -- 172.21.15.42:0/3014692707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f543c548b0 con 0x55f5439974d0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.150+0000 7f4574c00640 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d5b4e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f455c100300 tx=0x7f455c10b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608a3e29a5c6dfc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.150+0000 7f4562600640 1 -- 172.21.15.42:0/3014692707 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c10bcf0 con 0x55f5439974d0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.150+0000 7f4562600640 1 -- 172.21.15.42:0/3014692707 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f455c110070 con 0x55f5439974d0 2024-05-27T10:20:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.150+0000 7f4562600640 1 -- 172.21.15.42:0/3014692707 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c114630 con 0x55f5439974d0 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f456e000640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f45580b2b00 0x7f45580b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f456e000640 1 -- 172.21.15.42:0/1592880153 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f455c102e00 con 0x7f457010d990 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f4576000640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f45580b2b00 0x7f45580b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 -- 172.21.15.42:0/3014692707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 msgr2=0x55f543d5b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d5b4e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f455c100300 tx=0x7f455c10b0e0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 -- 172.21.15.42:0/3014692707 shutdown_connections 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d5b4e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f54398f2f0 0x55f543d5afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 --2- 172.21.15.42:0/3014692707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d5aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 -- 172.21.15.42:0/3014692707 >> 172.21.15.42:0/3014692707 conn(0x55f5439ac300 msgr2=0x55f543d57210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.151+0000 7f457f004740 1 -- 172.21.15.42:0/3014692707 shutdown_connections 2024-05-27T10:20:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.152+0000 7f457f004740 1 -- 172.21.15.42:0/3014692707 wait complete. 2024-05-27T10:20:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.152+0000 7f4576000640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f45580b2b00 0x7f45580b5010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55f543d57bd0 tx=0x7f45680093d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.152+0000 7f457f004740 1 -- start start 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 Processor -- start 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d64ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439974d0 0x55f543d65410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f5439a1790 con 0x55f54398f2f0 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f543c63bc0 con 0x55f5439b81b0 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.153+0000 7f457f004740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f543d67950 con 0x55f5439974d0 2024-05-27T10:20:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f457f004740 10 client.0 _ll_register_callbacks cb 0x55f543c41400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d64ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d64ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59262/0 (socket says 172.21.15.42:59262) 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 -- 172.21.15.42:0/3642696594 learned_addr learned my addr 172.21.15.42:0/3642696594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4574c00640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439974d0 0x55f543d65410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4576000640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 -- 172.21.15.42:0/3642696594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439974d0 msgr2=0x55f543d65410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439974d0 0x55f543d65410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 -- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 msgr2=0x55f543d70c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d70c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.154+0000 7f4575600640 1 -- 172.21.15.42:0/3642696594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5439d0210 con 0x55f54398f2f0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f4575600640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d64ed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4564003f00 tx=0x7f4564004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5409a93d3c9e66d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f4556a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f4556a00640 1 -- 172.21.15.42:0/3642696594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564012a80 con 0x55f54398f2f0 2024-05-27T10:20:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f4556a00640 1 -- 172.21.15.42:0/3642696594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4564012c20 con 0x55f54398f2f0 2024-05-27T10:20:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455c110070 con 0x55f54398f2f0 2024-05-27T10:20:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f4556a00640 1 -- 172.21.15.42:0/3642696594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4564012e20 con 0x55f54398f2f0 2024-05-27T10:20:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f457f004740 10 client.15615 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.155+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f543a9c620 con 0x55f54398f2f0 2024-05-27T10:20:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f4556a00640 1 -- 172.21.15.42:0/3642696594 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f455c110070 con 0x55f54398f2f0 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f4556a00640 1 -- 172.21.15.42:0/3642696594 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4564019020 con 0x55f54398f2f0 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f457f004740 10 client.15615 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f457f004740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f457f004740 20 client.15615 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f457f004740 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f543976d80 0x55f543d76d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f457f004740 4 client.15615 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.156+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f543ccdae0 con 0x55f543976d80 2024-05-27T10:20:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.157+0000 7f4574c00640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f543976d80 0x55f543d76d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.158+0000 7f4574c00640 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f543976d80 0x55f543d76d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27ff6ac743b0c2f server_cookie=ec05788486446809 in_seq=0 out_seq=0 2024-05-27T10:20:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.158+0000 7f4556a00640 10 client.15615 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f4556a00640 1 -- 172.21.15.42:0/3642696594 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f543ccdae0 con 0x55f543976d80 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f4556a00640 10 client.15615 handle_command_reply: tid=0 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 client.15615 shutdown 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 msgr2=0x55f543d64ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d64ed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4564003f00 tx=0x7f4564004410 comp rx=0 tx=0).stop 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 shutdown_connections 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439b81b0 0x55f543d70c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f543976d80 0x55f543d76d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439974d0 0x55f543d65410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 --2- 172.21.15.42:0/3642696594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d64ed0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 >> 172.21.15.42:0/3642696594 conn(0x55f5439ac300 msgr2=0x55f543d5c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f4556a00640 0 client.15615 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 shutdown_connections 2024-05-27T10:20:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.160+0000 7f457f004740 1 -- 172.21.15.42:0/3642696594 wait complete. 2024-05-27T10:20:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.161+0000 7f457f004740 20 client.15615 trim_cache size 0 max 16384 2024-05-27T10:20:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.177+0000 7f457f004740 1 Processor -- start 2024-05-27T10:20:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 -- start start 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f54398f2f0 0x55f543d66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b4300 con 0x55f5439974d0 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b1bd0 con 0x55f54398f2f0 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f457f004740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f45701afa20 con 0x55f5439b81b0 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f4574c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f4574c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59276/0 (socket says 172.21.15.42:59276) 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.178+0000 7f4574c00640 1 -- 172.21.15.42:0/3418412575 learned_addr learned my addr 172.21.15.42:0/3418412575 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4575600640 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f54398f2f0 0x55f543d66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4576000640 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d70ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4574c00640 1 -- 172.21.15.42:0/3418412575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 msgr2=0x55f543d70ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4574c00640 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d70ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4574c00640 1 -- 172.21.15.42:0/3418412575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f54398f2f0 msgr2=0x55f543d66e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4574c00640 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f54398f2f0 0x55f543d66e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4574c00640 1 -- 172.21.15.42:0/3418412575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f543d58360 con 0x55f5439974d0 2024-05-27T10:20:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.179+0000 7f4574c00640 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d709b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f455c000c00 tx=0x7f455c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcd00c3f8a7dd1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.180+0000 7f4561c00640 1 -- 172.21.15.42:0/3418412575 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c049040 con 0x55f5439974d0 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.180+0000 7f4561c00640 1 -- 172.21.15.42:0/3418412575 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f455c110040 con 0x55f5439974d0 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.180+0000 7f4561c00640 1 -- 172.21.15.42:0/3418412575 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c0c9840 con 0x55f5439974d0 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 -- 172.21.15.42:0/3418412575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 msgr2=0x55f543d709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d709b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f455c000c00 tx=0x7f455c009260 comp rx=0 tx=0).stop 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 -- 172.21.15.42:0/3418412575 shutdown_connections 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d70ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5439974d0 0x55f543d709b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 --2- 172.21.15.42:0/3418412575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f54398f2f0 0x55f543d66e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 -- 172.21.15.42:0/3418412575 >> 172.21.15.42:0/3418412575 conn(0x55f5439ac300 msgr2=0x55f543d5b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 -- 172.21.15.42:0/3418412575 shutdown_connections 2024-05-27T10:20:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.181+0000 7f457f004740 1 -- 172.21.15.42:0/3418412575 wait complete. 2024-05-27T10:20:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.182+0000 7f457f004740 1 -- start start 2024-05-27T10:20:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.182+0000 7f457f004740 1 Processor -- start 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439974d0 0x55f543d680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b0090 con 0x55f54398f2f0 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b1500 con 0x55f5439974d0 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b0760 con 0x55f5439b81b0 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f457f004740 10 client.0 _ll_register_callbacks cb 0x7f4564009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f4575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f4575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d67b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59290/0 (socket says 172.21.15.42:59290) 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f4575600640 1 -- 172.21.15.42:0/2278833039 learned_addr learned my addr 172.21.15.42:0/2278833039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f4574c00640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439974d0 0x55f543d680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.183+0000 7f4576000640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4575600640 1 -- 172.21.15.42:0/2278833039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 msgr2=0x55f543d685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4575600640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4575600640 1 -- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439974d0 msgr2=0x55f543d680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4575600640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439974d0 0x55f543d680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4575600640 1 -- 172.21.15.42:0/2278833039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f543d5a7f0 con 0x55f54398f2f0 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4575600640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d67b60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4564003f00 tx=0x7f45640120d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be77a362474ac89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4556a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4556a00640 1 -- 172.21.15.42:0/2278833039 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564004910 con 0x55f54398f2f0 2024-05-27T10:20:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f4556a00640 1 -- 172.21.15.42:0/2278833039 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4564004ab0 con 0x55f54398f2f0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.184+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455c110040 con 0x55f54398f2f0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f4556a00640 1 -- 172.21.15.42:0/2278833039 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4564004cb0 con 0x55f54398f2f0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 10 client.15621 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f543dde5b0 con 0x55f54398f2f0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f4556a00640 1 -- 172.21.15.42:0/2278833039 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f455c110040 con 0x55f54398f2f0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f4556a00640 1 -- 172.21.15.42:0/2278833039 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4564019040 con 0x55f54398f2f0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 10 client.15621 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 20 client.15621 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f54394c1c0 0x55f543de0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 4 client.15621 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.185+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f543cc72c0 con 0x55f54394c1c0 2024-05-27T10:20:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.186+0000 7f4574c00640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f54394c1c0 0x55f543de0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.187+0000 7f4574c00640 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f54394c1c0 0x55f543de0da0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab20f33f54028b95 server_cookie=da9cade5de4a6737 in_seq=0 out_seq=0 2024-05-27T10:20:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.187+0000 7f4556a00640 10 client.15621 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f4556a00640 1 -- 172.21.15.42:0/2278833039 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f543cc72c0 con 0x55f54394c1c0 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f4556a00640 10 client.15621 handle_command_reply: tid=0 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 client.15621 shutdown 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 msgr2=0x55f543d67b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d67b60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4564003f00 tx=0x7f45640120d0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 shutdown_connections 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f54394c1c0 0x55f543de0da0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f5439b81b0 0x55f543d685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5439974d0 0x55f543d680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 --2- 172.21.15.42:0/2278833039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f54398f2f0 0x55f543d67b60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f4556a00640 0 client.15621 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 >> 172.21.15.42:0/2278833039 conn(0x55f5439ac300 msgr2=0x55f543d5b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 shutdown_connections 2024-05-27T10:20:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 1 -- 172.21.15.42:0/2278833039 wait complete. 2024-05-27T10:20:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.204+0000 7f457f004740 20 client.15621 trim_cache size 0 max 16384 2024-05-27T10:20:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:20:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:20:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f45580b2b00 msgr2=0x7f45580b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f45580b2b00 0x7f45580b5010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55f543d57bd0 tx=0x7f45680093d0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 msgr2=0x7f45701a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f45701a2b10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f455c005940 tx=0x7f455c0042f0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 shutdown_connections 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f45580b2b00 0x7f45580b5010 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701102f0 0x7f457019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457010d990 0x7f45701a2b10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 --2- 172.21.15.42:0/1592880153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4570075480 0x7f45701a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.205+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 >> 172.21.15.42:0/1592880153 conn(0x7f4570079240 msgr2=0x7f4570102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.206+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 shutdown_connections 2024-05-27T10:20:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:12.206+0000 7f4576a00640 1 -- 172.21.15.42:0/1592880153 wait complete. 2024-05-27T10:20:12.355 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:12 smithi042 ceph-mon[31661]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-27T10:20:12.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:12 smithi104 ceph-mon[34812]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-27T10:20:13.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:12 smithi143 ceph-mon[35545]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-27T10:20:14.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.935+0000 7f8fb1e00700 1 -- 172.21.15.42:0/3253681537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac075d20 msgr2=0x7f8fac076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.935+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/3253681537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac075d20 0x7f8fac076130 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009990 tx=0x7f8f9c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.937 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:14 smithi042 ceph-mon[31661]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 -- 172.21.15.42:0/3253681537 shutdown_connections 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/3253681537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fac110b90 0x7f8fac112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/3253681537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac075d20 0x7f8fac076130 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/3253681537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac076670 0x7f8fac110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 -- 172.21.15.42:0/3253681537 >> 172.21.15.42:0/3253681537 conn(0x7f8fac100030 msgr2=0x7f8fac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 -- 172.21.15.42:0/3253681537 shutdown_connections 2024-05-27T10:20:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.936+0000 7f8fb1e00700 1 -- 172.21.15.42:0/3253681537 wait complete. 2024-05-27T10:20:14.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.937+0000 7f8fb1e00700 1 Processor -- start 2024-05-27T10:20:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 -- start start 2024-05-27T10:20:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac075d20 0x7f8fac19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 0x7f8fac19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fac110b90 0x7f8fac1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac116580 con 0x7f8fac076670 2024-05-27T10:20:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac116400 con 0x7f8fac110b90 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.938+0000 7f8fb1e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac116700 con 0x7f8fac075d20 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 0x7f8fac19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 0x7f8fac19d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48446/0 (socket says 172.21.15.42:48446) 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 -- 172.21.15.42:0/2137241805 learned_addr learned my addr 172.21.15.42:0/2137241805 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fab400700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fac110b90 0x7f8fac1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8faaa00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac075d20 0x7f8fac19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 -- 172.21.15.42:0/2137241805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac075d20 msgr2=0x7f8fac19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac075d20 0x7f8fac19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 -- 172.21.15.42:0/2137241805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fac110b90 msgr2=0x7f8fac1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fac110b90 0x7f8fac1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8fa2000700 1 -- 172.21.15.42:0/2137241805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac1a1e20 con 0x7f8fac076670 2024-05-27T10:20:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.939+0000 7f8faaa00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac075d20 0x7f8fac19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.940+0000 7f8fa2000700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 0x7f8fac19d4f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400e9a0 tx=0x7f8f9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.940+0000 7f8fb0a00700 1 -- 172.21.15.42:0/2137241805 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400cbd0 con 0x7f8fac076670 2024-05-27T10:20:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.940+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac1a2110 con 0x7f8fac076670 2024-05-27T10:20:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.940+0000 7f8fb0a00700 1 -- 172.21.15.42:0/2137241805 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f9400cd70 con 0x7f8fac076670 2024-05-27T10:20:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.940+0000 7f8fb0a00700 1 -- 172.21.15.42:0/2137241805 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94018680 con 0x7f8fac076670 2024-05-27T10:20:14.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.940+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a9a30 con 0x7f8fac076670 2024-05-27T10:20:14.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.944+0000 7f8fb0a00700 1 -- 172.21.15.42:0/2137241805 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f8f94010950 con 0x7f8fac076670 2024-05-27T10:20:14.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 Processor -- start 2024-05-27T10:20:14.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 -- start start 2024-05-27T10:20:14.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fb0a00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8f900b2ba0 0x7f8f900b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fb0a00700 1 -- 172.21.15.42:0/2137241805 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8f940e2bc0 con 0x7f8fac076670 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cc33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158b44ec0 0x55b158cc38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fab400700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8f900b2ba0 0x7f8f900b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 0x55b158cd0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b158bf1ec0 con 0x55b158cc3ef0 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b158be1a60 con 0x55b158c30a70 2024-05-27T10:20:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.946+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b158be5aa0 con 0x55b158b44ec0 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 0x55b158cd0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 0x55b158cd0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48462/0 (socket says 172.21.15.42:48462) 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 -- 172.21.15.42:0/1597942360 learned_addr learned my addr 172.21.15.42:0/1597942360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fa2000700 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158b44ec0 0x55b158cc38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8faaa00700 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cc33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 -- 172.21.15.42:0/1597942360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158b44ec0 msgr2=0x55b158cc38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158b44ec0 0x55b158cc38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 -- 172.21.15.42:0/1597942360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 msgr2=0x55b158cc33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cc33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.947+0000 7f8fab400700 1 -- 172.21.15.42:0/1597942360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b158baa4a0 con 0x55b158cc3ef0 2024-05-27T10:20:14.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:14 smithi104 ceph-mon[34812]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-27T10:20:14.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.948+0000 7f8fab400700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8f900b2ba0 0x7f8f900b50b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4007a30 tx=0x7f8fa4009c10 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:14.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.948+0000 7f8fab400700 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 0x55b158cd0190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40189b0 tx=0x7f8fa401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3000a32333aac5a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:14.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.948+0000 7f8fa0c00700 1 -- 172.21.15.42:0/1597942360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4021070 con 0x55b158cc3ef0 2024-05-27T10:20:14.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.948+0000 7f8fa0c00700 1 -- 172.21.15.42:0/1597942360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fa4018d70 con 0x55b158cc3ef0 2024-05-27T10:20:14.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.948+0000 7f8fa0c00700 1 -- 172.21.15.42:0/1597942360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4012040 con 0x55b158cc3ef0 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1597942360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 msgr2=0x55b158cd0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 0x55b158cd0190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40189b0 tx=0x7f8fa401b000 comp rx=0 tx=0).stop 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1597942360 shutdown_connections 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158cc3ef0 0x55b158cd0190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cc33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1597942360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158b44ec0 0x55b158cc38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.949+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1597942360 >> 172.21.15.42:0/1597942360 conn(0x55b158bde410 msgr2=0x55b158cbf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.950+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1597942360 shutdown_connections 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.950+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1597942360 wait complete. 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.950+0000 7f8fc09f4b80 1 -- start start 2024-05-27T10:20:14.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 Processor -- start 2024-05-27T10:20:14.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158ccd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158c30a70 0x55b158ccdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158cc3ef0 0x55b158cce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b158ae8020 con 0x55b158b44ec0 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b158b827e0 con 0x55b158cc3ef0 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b158cd0ba0 con 0x55b158c30a70 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.951+0000 7f8fc09f4b80 10 client.0 _ll_register_callbacks cb 0x55b158b8bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158ccd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158ccd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48478/0 (socket says 172.21.15.42:48478) 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 -- 172.21.15.42:0/3146564413 learned_addr learned my addr 172.21.15.42:0/3146564413 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8fab400700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158cc3ef0 0x55b158cce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8fa2000700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158c30a70 0x55b158ccdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 -- 172.21.15.42:0/3146564413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158c30a70 msgr2=0x55b158ccdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158c30a70 0x55b158ccdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 -- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158cc3ef0 msgr2=0x55b158cce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158cc3ef0 0x55b158cce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8faaa00700 1 -- 172.21.15.42:0/3146564413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b158b31950 con 0x55b158b44ec0 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.952+0000 7f8fa2000700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158c30a70 0x55b158ccdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.953+0000 7f8faaa00700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158ccd6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009990 tx=0x7f8f9c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93830bb032df1560 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.953+0000 7f8f99600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.953+0000 7f8f99600700 1 -- 172.21.15.42:0/3146564413 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c027060 con 0x55b158b44ec0 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.953+0000 7f8f99600700 1 -- 172.21.15.42:0/3146564413 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8f9c00b6b0 con 0x55b158b44ec0 2024-05-27T10:20:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.953+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa4018d70 con 0x55b158b44ec0 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.954+0000 7f8f99600700 1 -- 172.21.15.42:0/3146564413 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f9c00b8b0 con 0x55b158b44ec0 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.954+0000 7f8fc09f4b80 10 client.15633 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.954+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b158bb20e0 con 0x55b158b44ec0 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.954+0000 7f8f99600700 1 -- 172.21.15.42:0/3146564413 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fa4018d70 con 0x55b158b44ec0 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.954+0000 7f8f99600700 1 -- 172.21.15.42:0/3146564413 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f9c00f9f0 con 0x55b158b44ec0 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fc09f4b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fc09f4b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fc09f4b80 20 client.15633 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:14.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158cdda40 0x55b158cdfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fc09f4b80 4 client.15633 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:14.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b158bf3e10 con 0x55b158cdda40 2024-05-27T10:20:14.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.955+0000 7f8fa2000700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158cdda40 0x55b158cdfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.956+0000 7f8fa2000700 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158cdda40 0x55b158cdfea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0fb2a3baf3a7105 server_cookie=c16d8cde78c3a911 in_seq=0 out_seq=0 2024-05-27T10:20:14.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.956+0000 7f8f99600700 10 client.15633 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:14.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.958+0000 7f8f99600700 1 -- 172.21.15.42:0/3146564413 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b158bf3e10 con 0x55b158cdda40 2024-05-27T10:20:14.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.958+0000 7f8f99600700 10 client.15633 handle_command_reply: tid=0 2024-05-27T10:20:14.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.958+0000 7f8fc09f4b80 1 client.15633 shutdown 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 msgr2=0x55b158ccd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158ccd6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009990 tx=0x7f8f9c0057d0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 shutdown_connections 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158cdda40 0x55b158cdfea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158cc3ef0 0x55b158cce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158ccd6d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/3146564413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158c30a70 0x55b158ccdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 >> 172.21.15.42:0/3146564413 conn(0x55b158bde410 msgr2=0x55b158cc3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:14.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8f99600700 0 client.15633 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:14.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 shutdown_connections 2024-05-27T10:20:14.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/3146564413 wait complete. 2024-05-27T10:20:14.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.959+0000 7f8fc09f4b80 20 client.15633 trim_cache size 0 max 16384 2024-05-27T10:20:14.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 Processor -- start 2024-05-27T10:20:14.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 -- start start 2024-05-27T10:20:14.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158b44ec0 0x55b158cc8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 0x55b158ccc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158cd9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1b3720 con 0x55b158c30a70 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1b0fb0 con 0x55b158b44ec0 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.983+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1aee20 con 0x55b158cc3ef0 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 0x55b158ccc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 0x55b158ccc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48480/0 (socket says 172.21.15.42:48480) 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 -- 172.21.15.42:0/1151560997 learned_addr learned my addr 172.21.15.42:0/1151560997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8faaa00700 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158b44ec0 0x55b158cc8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fab400700 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158cd9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 -- 172.21.15.42:0/1151560997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 msgr2=0x55b158cd9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158cd9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 -- 172.21.15.42:0/1151560997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158b44ec0 msgr2=0x55b158cc8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158b44ec0 0x55b158cc8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.984+0000 7f8fa2000700 1 -- 172.21.15.42:0/1151560997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b158cbb2c0 con 0x55b158c30a70 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.985+0000 7f8fa2000700 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 0x55b158ccc8c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8f940a9c40 tx=0x7f8f9400efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347e35af41f50c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.985+0000 7f8f9be00700 1 -- 172.21.15.42:0/1151560997 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f940aa3b0 con 0x55b158c30a70 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.985+0000 7f8f9be00700 1 -- 172.21.15.42:0/1151560997 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f940a9cb0 con 0x55b158c30a70 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.985+0000 7f8f9be00700 1 -- 172.21.15.42:0/1151560997 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94010430 con 0x55b158c30a70 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1151560997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 msgr2=0x55b158ccc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 0x55b158ccc8c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8f940a9c40 tx=0x7f8f9400efa0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1151560997 shutdown_connections 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158cd9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158b44ec0 0x55b158cc8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/1151560997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158c30a70 0x55b158ccc8c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1151560997 >> 172.21.15.42:0/1151560997 conn(0x55b158bde410 msgr2=0x55b158cc75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:14.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1151560997 shutdown_connections 2024-05-27T10:20:14.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/1151560997 wait complete. 2024-05-27T10:20:14.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.986+0000 7f8fc09f4b80 1 -- start start 2024-05-27T10:20:14.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 Processor -- start 2024-05-27T10:20:14.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158cd2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cd2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158d47530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1af470 con 0x55b158b44ec0 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1b08e0 con 0x55b158c30a70 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1afb40 con 0x55b158cc3ef0 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.987+0000 7f8fc09f4b80 10 client.0 _ll_register_callbacks cb 0x55b158b8bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158cd2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158cd2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48492/0 (socket says 172.21.15.42:48492) 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 -- 172.21.15.42:0/880067514 learned_addr learned my addr 172.21.15.42:0/880067514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8fa2000700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cd2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8fab400700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158d47530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 -- 172.21.15.42:0/880067514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 msgr2=0x55b158d47530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158d47530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 -- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 msgr2=0x55b158cd2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cd2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 -- 172.21.15.42:0/880067514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b158cce4c0 con 0x55b158b44ec0 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8fa2000700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cd2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8faaa00700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158cd2320 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c000c00 tx=0x7f8f9c018ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdcfd6e8d94928a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.988+0000 7f8f99600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.989+0000 7f8f99600700 1 -- 172.21.15.42:0/880067514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c039060 con 0x55b158b44ec0 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.989+0000 7f8f99600700 1 -- 172.21.15.42:0/880067514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8f9c00f460 con 0x55b158b44ec0 2024-05-27T10:20:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.989+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f940a9cb0 con 0x55b158b44ec0 2024-05-27T10:20:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.989+0000 7f8f99600700 1 -- 172.21.15.42:0/880067514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f9c027040 con 0x55b158b44ec0 2024-05-27T10:20:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.989+0000 7f8fc09f4b80 10 client.15639 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.989+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b158d48b50 con 0x55b158b44ec0 2024-05-27T10:20:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8f99600700 1 -- 172.21.15.42:0/880067514 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f940a9cb0 con 0x55b158b44ec0 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8f99600700 1 -- 172.21.15.42:0/880067514 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f9c02c460 con 0x55b158b44ec0 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8fc09f4b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8fc09f4b80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8fc09f4b80 20 client.15639 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158d49290 0x55b158d4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8fc09f4b80 4 client.15639 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.990+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b158bed5f0 con 0x55b158d49290 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.991+0000 7f8fa2000700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158d49290 0x55b158d4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.992+0000 7f8f99600700 10 client.15639 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.992+0000 7f8fa2000700 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158d49290 0x55b158d4b6b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65385f361b05d7ae server_cookie=33d8245d1716d25c in_seq=0 out_seq=0 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8f99600700 1 -- 172.21.15.42:0/880067514 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b158bed5f0 con 0x55b158d49290 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8f99600700 10 client.15639 handle_command_reply: tid=0 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 client.15639 shutdown 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 msgr2=0x55b158cd2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158cd2320 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c000c00 tx=0x7f8f9c018ab0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 shutdown_connections 2024-05-27T10:20:14.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b158d49290 0x55b158d4b6b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b158cc3ef0 0x55b158d47530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b158b44ec0 0x55b158cd2320 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 --2- 172.21.15.42:0/880067514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b158c30a70 0x55b158cd2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 >> 172.21.15.42:0/880067514 conn(0x55b158bde410 msgr2=0x55b158cc75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8f99600700 0 client.15639 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 shutdown_connections 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.995+0000 7f8fc09f4b80 1 -- 172.21.15.42:0/880067514 wait complete. 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.996+0000 7f8fc09f4b80 20 client.15639 trim_cache size 0 max 16384 2024-05-27T10:20:14.997 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:20:14.998 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-05-27T10:20:14.998 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "81cf08a5-d603-4e47-865e-7b14ac3597ae", 2024-05-27T10:20:14.998 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-05-27T10:20:14.998 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:20:14.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.996+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8f900b2ba0 msgr2=0x7f8f900b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.996+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8f900b2ba0 0x7f8f900b50b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4007a30 tx=0x7f8fa4009c10 comp rx=0 tx=0).stop 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.996+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 msgr2=0x7f8fac19d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.996+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 0x7f8fac19d4f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400e9a0 tx=0x7f8f9400ed60 comp rx=0 tx=0).stop 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 shutdown_connections 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8f900b2ba0 0x7f8f900b50b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fac110b90 0x7f8fac1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fac075d20 0x7f8fac19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 --2- 172.21.15.42:0/2137241805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac076670 0x7f8fac19d4f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 >> 172.21.15.42:0/2137241805 conn(0x7f8fac100030 msgr2=0x7f8fac1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 shutdown_connections 2024-05-27T10:20:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:14.997+0000 7f8fb1e00700 1 -- 172.21.15.42:0/2137241805 wait complete. 2024-05-27T10:20:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:14 smithi143 ceph-mon[35545]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-27T10:20:16.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:20:16.554 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:16.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:16 smithi042 ceph-mon[31661]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-27T10:20:16.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:16 smithi042 ceph-mon[31661]: scrub queued for path: / 2024-05-27T10:20:16.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:16 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:20:16.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:16 smithi042 ceph-mon[31661]: scrub summary: active paths [/] 2024-05-27T10:20:16.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:16 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:20:16.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:16 smithi042 ceph-mon[31661]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T10:20:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:16 smithi104 ceph-mon[34812]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-27T10:20:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:16 smithi104 ceph-mon[34812]: scrub queued for path: / 2024-05-27T10:20:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:16 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:20:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:16 smithi104 ceph-mon[34812]: scrub summary: active paths [/] 2024-05-27T10:20:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:16 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:20:16.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:16 smithi104 ceph-mon[34812]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T10:20:17.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:16 smithi143 ceph-mon[35545]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-27T10:20:17.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:16 smithi143 ceph-mon[35545]: scrub queued for path: / 2024-05-27T10:20:17.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:16 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:20:17.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:16 smithi143 ceph-mon[35545]: scrub summary: active paths [/] 2024-05-27T10:20:17.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:16 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:20:17.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:16 smithi143 ceph-mon[35545]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T10:20:17.845 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.4 2024-05-27T10:20:17.846 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:20:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:18.693 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:18 smithi042 ceph-mon[31661]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-27T10:20:18.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:18 smithi104 ceph-mon[34812]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-27T10:20:19.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:18 smithi143 ceph-mon[35545]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.296+0000 7fb702000700 1 -- 172.21.15.42:0/1877808843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc1109a0 msgr2=0x7fb6fc112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.296+0000 7fb702000700 1 --2- 172.21.15.42:0/1877808843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc1109a0 0x7fb6fc112e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b0a0 tx=0x7fb6f400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 -- 172.21.15.42:0/1877808843 shutdown_connections 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 --2- 172.21.15.42:0/1877808843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc1109a0 0x7fb6fc112e00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 --2- 172.21.15.42:0/1877808843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc075d20 0x7fb6fc076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 --2- 172.21.15.42:0/1877808843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6fc076670 0x7fb6fc110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 -- 172.21.15.42:0/1877808843 >> 172.21.15.42:0/1877808843 conn(0x7fb6fc079d00 msgr2=0x7fb6fc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 -- 172.21.15.42:0/1877808843 shutdown_connections 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.297+0000 7fb702000700 1 -- 172.21.15.42:0/1877808843 wait complete. 2024-05-27T10:20:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.298+0000 7fb702000700 1 Processor -- start 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.298+0000 7fb702000700 1 -- start start 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb702000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc075d20 0x7fb6fc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb702000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 0x7fb6fc19d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb702000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6fc1109a0 0x7fb6fc1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb702000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc116330 con 0x7fb6fc076670 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb702000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1161b0 con 0x7fb6fc075d20 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb702000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1164b0 con 0x7fb6fc1109a0 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb6fbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 0x7fb6fc19d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb6fbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 0x7fb6fc19d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48506/0 (socket says 172.21.15.42:48506) 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb6fbe00700 1 -- 172.21.15.42:0/2319480434 learned_addr learned my addr 172.21.15.42:0/2319480434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb700c00700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc075d20 0x7fb6fc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.299+0000 7fb701600700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6fc1109a0 0x7fb6fc1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6fbe00700 1 -- 172.21.15.42:0/2319480434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6fc1109a0 msgr2=0x7fb6fc1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6fc1109a0 0x7fb6fc1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6fbe00700 1 -- 172.21.15.42:0/2319480434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc075d20 msgr2=0x7fb6fc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc075d20 0x7fb6fc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6fbe00700 1 -- 172.21.15.42:0/2319480434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc1a1bf0 con 0x7fb6fc076670 2024-05-27T10:20:19.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 0x7fb6fc19d2c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d950 tx=0x7fb6e800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:19.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6f9600700 1 -- 172.21.15.42:0/2319480434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8016070 con 0x7fb6fc076670 2024-05-27T10:20:19.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6f9600700 1 -- 172.21.15.42:0/2319480434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e8010460 con 0x7fb6fc076670 2024-05-27T10:20:19.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc1a1ee0 con 0x7fb6fc076670 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.300+0000 7fb6f9600700 1 -- 172.21.15.42:0/2319480434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800f620 con 0x7fb6fc076670 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.301+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc1a9800 con 0x7fb6fc076670 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.305+0000 7fb6f9600700 1 -- 172.21.15.42:0/2319480434 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb6e8019070 con 0x7fb6fc076670 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.306+0000 7fb7120b3b80 1 Processor -- start 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.306+0000 7fb7120b3b80 1 -- start start 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb6f9600700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb6e40b29f0 0x7fb6e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fcfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0410 0x5619d1fdaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 0x5619d1fdd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5619d1efee40 con 0x5619d1fd0820 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5619d1eee9e0 con 0x5619d1e51e40 2024-05-27T10:20:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5619d1ef2a20 con 0x5619d1fd0410 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb6f9600700 1 -- 172.21.15.42:0/2319480434 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb6e80e21f0 con 0x7fb6fc076670 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb701600700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb6e40b29f0 0x7fb6e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb701600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 0x5619d1fdd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb701600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 0x5619d1fdd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48516/0 (socket says 172.21.15.42:48516) 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb701600700 1 -- 172.21.15.42:0/2704808688 learned_addr learned my addr 172.21.15.42:0/2704808688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.307+0000 7fb700c00700 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fcfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0410 0x5619d1fdaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 -- 172.21.15.42:0/2704808688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0410 msgr2=0x5619d1fdaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0410 0x5619d1fdaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 -- 172.21.15.42:0/2704808688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 msgr2=0x5619d1fcfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fcfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 -- 172.21.15.42:0/2704808688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d1eb7420 con 0x5619d1fd0820 2024-05-27T10:20:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb6e40b29f0 0x7fb6e40b4f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc077bc0 tx=0x7fb6f4009dc0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.308+0000 7fb701600700 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 0x5619d1fdd020 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b0a0 tx=0x7fb6f4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf2f49b27680ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.309+0000 7fb6ee000700 1 -- 172.21.15.42:0/2704808688 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4009270 con 0x5619d1fd0820 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.309+0000 7fb6ee000700 1 -- 172.21.15.42:0/2704808688 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f4007d30 con 0x5619d1fd0820 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.309+0000 7fb6ee000700 1 -- 172.21.15.42:0/2704808688 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4020cb0 con 0x5619d1fd0820 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2704808688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 msgr2=0x5619d1fdd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 0x5619d1fdd020 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b0a0 tx=0x7fb6f4025000 comp rx=0 tx=0).stop 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2704808688 shutdown_connections 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0820 0x5619d1fdd020 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fcfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2704808688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0410 0x5619d1fdaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2704808688 >> 172.21.15.42:0/2704808688 conn(0x5619d1eb1fc0 msgr2=0x5619d1fcbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2704808688 shutdown_connections 2024-05-27T10:20:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.310+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2704808688 wait complete. 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.311+0000 7fb7120b3b80 1 -- start start 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.311+0000 7fb7120b3b80 1 Processor -- start 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fda640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fdab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0820 0x5619d1fdb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5619d1df4fa0 con 0x5619d1fd0410 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5619d1e8f760 con 0x5619d1e51e40 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5619d1fdd930 con 0x5619d1fd0820 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb7120b3b80 10 client.0 _ll_register_callbacks cb 0x5619d1e98c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fdab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fdab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48526/0 (socket says 172.21.15.42:48526) 2024-05-27T10:20:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 -- 172.21.15.42:0/1371239551 learned_addr learned my addr 172.21.15.42:0/1371239551 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb701600700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0820 0x5619d1fdb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb700c00700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fda640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 -- 172.21.15.42:0/1371239551 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0820 msgr2=0x5619d1fdb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0820 0x5619d1fdb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 -- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 msgr2=0x5619d1fda640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.312+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fda640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb6fbe00700 1 -- 172.21.15.42:0/1371239551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d1e3e8d0 con 0x5619d1fd0410 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fdab80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80df990 tx=0x7fb6e80a8c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e672f47773c30c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb6e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb6e2a00700 1 -- 172.21.15.42:0/1371239551 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8020070 con 0x5619d1fd0410 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb6e2a00700 1 -- 172.21.15.42:0/1371239551 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb6e80a3be0 con 0x5619d1fd0410 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f4007d30 con 0x5619d1fd0410 2024-05-27T10:20:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb6e2a00700 1 -- 172.21.15.42:0/1371239551 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6e80dfdc0 con 0x5619d1fd0410 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb7120b3b80 10 client.15651 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.313+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5619d1ebf060 con 0x5619d1fd0410 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb6e2a00700 1 -- 172.21.15.42:0/1371239551 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f4007d30 con 0x5619d1fd0410 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb6e2a00700 1 -- 172.21.15.42:0/1371239551 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb6e80a9720 con 0x5619d1fd0410 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb7120b3b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb7120b3b80 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb7120b3b80 20 client.15651 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fea7b0 0x5619d1fecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb7120b3b80 4 client.15651 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.314+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d1f00d90 con 0x5619d1fea7b0 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.315+0000 7fb700c00700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fea7b0 0x5619d1fecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.316+0000 7fb6e2a00700 10 client.15651 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.316+0000 7fb700c00700 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fea7b0 0x5619d1fecbd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b266f242bfbd148e server_cookie=51047bf8b6bd5908 in_seq=0 out_seq=0 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb6e2a00700 1 -- 172.21.15.42:0/1371239551 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5619d1f00d90 con 0x5619d1fea7b0 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb6e2a00700 10 client.15651 handle_command_reply: tid=0 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 client.15651 shutdown 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 msgr2=0x5619d1fdab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fdab80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80df990 tx=0x7fb6e80a8c30 comp rx=0 tx=0).stop 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 shutdown_connections 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fea7b0 0x5619d1fecbd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1fd0820 0x5619d1fdb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb6e2a00700 0 client.15651 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fda640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/1371239551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fdab80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 >> 172.21.15.42:0/1371239551 conn(0x5619d1eb1fc0 msgr2=0x5619d1fcaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 shutdown_connections 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.318+0000 7fb7120b3b80 1 -- 172.21.15.42:0/1371239551 wait complete. 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.319+0000 7fb7120b3b80 20 client.15651 trim_cache size 0 max 16384 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 Processor -- start 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 -- start start 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1eb1fc0 0x5619d1fd5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fe69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1b3640 con 0x5619d1fd0410 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1b0ed0 con 0x5619d1e51e40 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.342+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1aed40 con 0x5619d1eb1fc0 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fe69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fe69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48542/0 (socket says 172.21.15.42:48542) 2024-05-27T10:20:19.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 -- 172.21.15.42:0/2542351078 learned_addr learned my addr 172.21.15.42:0/2542351078 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb700c00700 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1eb1fc0 0x5619d1fd5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 -- 172.21.15.42:0/2542351078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1eb1fc0 msgr2=0x5619d1fd5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1eb1fc0 0x5619d1fd5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 -- 172.21.15.42:0/2542351078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 msgr2=0x5619d1fdb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fdb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb701600700 1 -- 172.21.15.42:0/2542351078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d1fcd350 con 0x5619d1fd0410 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.343+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fdb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.344+0000 7fb701600700 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fe69d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400d010 tx=0x7fb6f400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f5702acf5bdc77 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.344+0000 7fb6ed600700 1 -- 172.21.15.42:0/2542351078 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40078a0 con 0x5619d1fd0410 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.344+0000 7fb6ed600700 1 -- 172.21.15.42:0/2542351078 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f4035460 con 0x5619d1fd0410 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.344+0000 7fb6ed600700 1 -- 172.21.15.42:0/2542351078 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4020450 con 0x5619d1fd0410 2024-05-27T10:20:19.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2542351078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 msgr2=0x5619d1fe69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fe69d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400d010 tx=0x7fb6f400b420 comp rx=0 tx=0).stop 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2542351078 shutdown_connections 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d1fe69d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1eb1fc0 0x5619d1fd5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/2542351078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1e51e40 0x5619d1fdb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2542351078 >> 172.21.15.42:0/2542351078 conn(0x5619d1fc4f60 msgr2=0x5619d1fdbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2542351078 shutdown_connections 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.345+0000 7fb7120b3b80 1 -- 172.21.15.42:0/2542351078 wait complete. 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.346+0000 7fb7120b3b80 1 -- start start 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.346+0000 7fb7120b3b80 1 Processor -- start 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1e51e40 0x5619d1fdf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1eb1fc0 0x5619d1fdf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d204e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1af390 con 0x5619d1fd0410 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1b0800 con 0x5619d1eb1fc0 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1afa60 con 0x5619d1e51e40 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb7120b3b80 10 client.0 _ll_register_callbacks cb 0x5619d1e98c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb701600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d204e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb701600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d204e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48558/0 (socket says 172.21.15.42:48558) 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb701600700 1 -- 172.21.15.42:0/339869540 learned_addr learned my addr 172.21.15.42:0/339869540 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.347+0000 7fb700c00700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1e51e40 0x5619d1fdf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb6fbe00700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1eb1fc0 0x5619d1fdf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb701600700 1 -- 172.21.15.42:0/339869540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1e51e40 msgr2=0x5619d1fdf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb701600700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1e51e40 0x5619d1fdf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb701600700 1 -- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1eb1fc0 msgr2=0x5619d1fdf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb701600700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1eb1fc0 0x5619d1fdf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb701600700 1 -- 172.21.15.42:0/339869540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d1fd1420 con 0x5619d1fd0410 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb701600700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d204e1c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b3f0 tx=0x7fb6f4007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b01f0846cc0fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb6e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.348+0000 7fb6e2a00700 1 -- 172.21.15.42:0/339869540 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f403b030 con 0x5619d1fd0410 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f4035460 con 0x5619d1fd0410 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb6e2a00700 1 -- 172.21.15.42:0/339869540 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb6f4026070 con 0x5619d1fd0410 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb6e2a00700 1 -- 172.21.15.42:0/339869540 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6f40094f0 con 0x5619d1fd0410 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 10 client.15657 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5619d20558a0 con 0x5619d1fd0410 2024-05-27T10:20:19.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb6e2a00700 1 -- 172.21.15.42:0/339869540 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f4035460 con 0x5619d1fd0410 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb6e2a00700 1 -- 172.21.15.42:0/339869540 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb6f4011070 con 0x5619d1fd0410 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 20 client.15657 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fd0820 0x5619d2057a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.349+0000 7fb7120b3b80 4 client.15657 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.350+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619d1efa570 con 0x5619d1fd0820 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.350+0000 7fb700c00700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fd0820 0x5619d2057a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.351+0000 7fb700c00700 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fd0820 0x5619d2057a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4018da5b3f25e5a server_cookie=bd62841da8185a20 in_seq=0 out_seq=0 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.351+0000 7fb6e2a00700 10 client.15657 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.351+0000 7fb6e2a00700 1 -- 172.21.15.42:0/339869540 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5619d1efa570 con 0x5619d1fd0820 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.351+0000 7fb6e2a00700 10 client.15657 handle_command_reply: tid=0 2024-05-27T10:20:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.351+0000 7fb7120b3b80 1 client.15657 shutdown 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 msgr2=0x5619d204e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d204e1c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b3f0 tx=0x7fb6f4007cf0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 shutdown_connections 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5619d1fd0820 0x5619d2057a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619d1fd0410 0x5619d204e1c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619d1e51e40 0x5619d1fdf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 --2- 172.21.15.42:0/339869540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d1eb1fc0 0x5619d1fdf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb6e2a00700 0 client.15657 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 >> 172.21.15.42:0/339869540 conn(0x5619d1fc4f60 msgr2=0x5619d1fdacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: "81cf08a5-d603-4e47-865e-7b14ac3597ae": { 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "81cf08a5-d603-4e47-865e-7b14ac3597ae", 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-05-27T10:20:19.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:19.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:19.755 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:20:19.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 shutdown_connections 2024-05-27T10:20:19.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 1 -- 172.21.15.42:0/339869540 wait complete. 2024-05-27T10:20:19.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.352+0000 7fb7120b3b80 20 client.15657 trim_cache size 0 max 16384 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb6e40b29f0 msgr2=0x7fb6e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb6e40b29f0 0x7fb6e40b4f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc077bc0 tx=0x7fb6f4009dc0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 msgr2=0x7fb6fc19d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 0x7fb6fc19d2c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d950 tx=0x7fb6e800dd10 comp rx=0 tx=0).stop 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 shutdown_connections 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb6e40b29f0 0x7fb6e40b4f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6fc1109a0 0x7fb6fc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc075d20 0x7fb6fc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 --2- 172.21.15.42:0/2319480434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6fc076670 0x7fb6fc19d2c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 >> 172.21.15.42:0/2319480434 conn(0x7fb6fc079d00 msgr2=0x7fb6fc100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 shutdown_connections 2024-05-27T10:20:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:19.353+0000 7fb702000700 1 -- 172.21.15.42:0/2319480434 wait complete. 2024-05-27T10:20:20.699 INFO:tasks.cephfs.filesystem:scrub status for tag:81cf08a5-d603-4e47-865e-7b14ac3597ae - {'path': '/', 'tag': '81cf08a5-d603-4e47-865e-7b14ac3597ae', 'options': 'recursive,force'} 2024-05-27T10:20:20.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:20 smithi042 ceph-mon[31661]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-27T10:20:20.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:20 smithi042 ceph-mon[31661]: scrub summary: idle 2024-05-27T10:20:20.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:20 smithi104 ceph-mon[34812]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-27T10:20:20.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:20 smithi104 ceph-mon[34812]: scrub summary: idle 2024-05-27T10:20:21.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:20 smithi143 ceph-mon[35545]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-27T10:20:21.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:20 smithi143 ceph-mon[35545]: scrub summary: idle 2024-05-27T10:20:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 -- 172.21.15.42:0/828008298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 msgr2=0x7f1cac1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 --2- 172.21.15.42:0/828008298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac1049f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00099c0 tx=0x7f1ca0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 -- 172.21.15.42:0/828008298 shutdown_connections 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 --2- 172.21.15.42:0/828008298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac109860 0x7f1cac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 --2- 172.21.15.42:0/828008298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac1049f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 --2- 172.21.15.42:0/828008298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac108f10 0x7f1cac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 -- 172.21.15.42:0/828008298 >> 172.21.15.42:0/828008298 conn(0x7f1cac100030 msgr2=0x7f1cac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 -- 172.21.15.42:0/828008298 shutdown_connections 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.297+0000 7f1cb4000700 1 -- 172.21.15.42:0/828008298 wait complete. 2024-05-27T10:20:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.299+0000 7f1cb4000700 1 Processor -- start 2024-05-27T10:20:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.299+0000 7f1cb4000700 1 -- start start 2024-05-27T10:20:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb4000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb4000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac108f10 0x7f1cac19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb4000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac109860 0x7f1cac19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb4000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac114390 con 0x7f1cac1045e0 2024-05-27T10:20:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb4000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac114210 con 0x7f1cac108f10 2024-05-27T10:20:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb4000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac114510 con 0x7f1cac109860 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb1600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48574/0 (socket says 172.21.15.42:48574) 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.300+0000 7f1cb1600700 1 -- 172.21.15.42:0/2747147932 learned_addr learned my addr 172.21.15.42:0/2747147932 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb0c00700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac108f10 0x7f1cac19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb2000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac109860 0x7f1cac19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb1600700 1 -- 172.21.15.42:0/2747147932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac109860 msgr2=0x7f1cac19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb1600700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac109860 0x7f1cac19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb1600700 1 -- 172.21.15.42:0/2747147932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac108f10 msgr2=0x7f1cac19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb1600700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac108f10 0x7f1cac19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb1600700 1 -- 172.21.15.42:0/2747147932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac19fb60 con 0x7f1cac1045e0 2024-05-27T10:20:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb0c00700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac108f10 0x7f1cac19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb2000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac109860 0x7f1cac19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.301+0000 7f1cb1600700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac19ad40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000b580 tx=0x7f1ca000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.302+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0026070 con 0x7f1cac1045e0 2024-05-27T10:20:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.302+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac19fdf0 con 0x7f1cac1045e0 2024-05-27T10:20:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.302+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca00054c0 con 0x7f1cac1045e0 2024-05-27T10:20:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.302+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004c00 con 0x7f1cac1045e0 2024-05-27T10:20:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.302+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac1a0320 con 0x7f1cac1045e0 2024-05-27T10:20:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.303+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac0512b0 con 0x7f1cac1045e0 2024-05-27T10:20:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.306+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ca0029070 con 0x7f1cac1045e0 2024-05-27T10:20:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.309+0000 7f1caa000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1c940b2a70 0x7f1c940b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.309+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1ca00e38e0 con 0x7f1cac1045e0 2024-05-27T10:20:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.309+0000 7f1cb0c00700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1c940b2a70 0x7f1c940b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.310+0000 7f1cb0c00700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1c940b2a70 0x7f1c940b4f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c98005fd0 tx=0x7f1c98005e20 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.312+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ca0013070 con 0x7f1cac1045e0 2024-05-27T10:20:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.593+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cac19c0a0 con 0x7f1cac1045e0 2024-05-27T10:20:22.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.594+0000 7f1caa000700 1 -- 172.21.15.42:0/2747147932 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f1ca00ab340 con 0x7f1cac1045e0 2024-05-27T10:20:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:20:22.596 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:20:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.598+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1c940b2a70 msgr2=0x7f1c940b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.598+0000 7f1cb4000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1c940b2a70 0x7f1c940b4f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c98005fd0 tx=0x7f1c98005e20 comp rx=0 tx=0).stop 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.598+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 msgr2=0x7f1cac19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.598+0000 7f1cb4000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac19ad40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000b580 tx=0x7f1ca000f740 comp rx=0 tx=0).stop 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 shutdown_connections 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1c940b2a70 0x7f1c940b4f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cac109860 0x7f1cac19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac1045e0 0x7f1cac19ad40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 --2- 172.21.15.42:0/2747147932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cac108f10 0x7f1cac19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 >> 172.21.15.42:0/2747147932 conn(0x7f1cac100030 msgr2=0x7f1cac10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 shutdown_connections 2024-05-27T10:20:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:22.599+0000 7f1cb4000700 1 -- 172.21.15.42:0/2747147932 wait complete. 2024-05-27T10:20:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:20:22.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:22 smithi104 ceph-mon[34812]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-27T10:20:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:22 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2747147932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:20:23.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:22 smithi143 ceph-mon[35545]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-27T10:20:23.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:22 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2747147932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:20:23.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:22 smithi042 ceph-mon[31661]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-27T10:20:23.062 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:22 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2747147932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:20:23.652 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 149.4 --await-for 45.0' 2024-05-27T10:20:23.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 149.4 --await-for 45.0 2024-05-27T10:20:24.127 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:24.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:24 smithi042 ceph-mon[31661]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-27T10:20:24.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:24 smithi104 ceph-mon[34812]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-27T10:20:25.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:24 smithi143 ceph-mon[35545]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-27T10:20:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.449+0000 7f6693200640 1 -- 172.21.15.42:0/1509629530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 msgr2=0x7f668c116e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.449+0000 7f6693200640 1 --2- 172.21.15.42:0/1509629530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c116e90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6688009ca0 tx=0x7f668802a8c0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.449+0000 7f6693200640 1 -- 172.21.15.42:0/1509629530 shutdown_connections 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.449+0000 7f6693200640 1 --2- 172.21.15.42:0/1509629530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c116e90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.449+0000 7f6693200640 1 --2- 172.21.15.42:0/1509629530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f668c112110 0x7f668c114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.450+0000 7f6693200640 1 --2- 172.21.15.42:0/1509629530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f668c076590 0x7f668c0769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.450+0000 7f6693200640 1 -- 172.21.15.42:0/1509629530 >> 172.21.15.42:0/1509629530 conn(0x7f668c0ff170 msgr2=0x7f668c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.450+0000 7f6693200640 1 -- 172.21.15.42:0/1509629530 shutdown_connections 2024-05-27T10:20:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.450+0000 7f6693200640 1 -- 172.21.15.42:0/1509629530 wait complete. 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 Processor -- start 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 -- start start 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f668c076590 0x7f668c1a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f668c112110 0x7f668c1a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f668c11a200 con 0x7f668c114a70 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f668c11a080 con 0x7f668c076590 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.451+0000 7f6693200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f668c11a380 con 0x7f668c112110 2024-05-27T10:20:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c1a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52918/0 (socket says 172.21.15.42:52918) 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 -- 172.21.15.42:0/2408508835 learned_addr learned my addr 172.21.15.42:0/2408508835 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6691400640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f668c112110 0x7f668c1a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6691e00640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f668c076590 0x7f668c1a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 -- 172.21.15.42:0/2408508835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f668c112110 msgr2=0x7f668c1a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f668c112110 0x7f668c1a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 -- 172.21.15.42:0/2408508835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f668c076590 msgr2=0x7f668c1a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f668c076590 0x7f668c1a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.452+0000 7f6692800640 1 -- 172.21.15.42:0/2408508835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c1a10c0 con 0x7f668c114a70 2024-05-27T10:20:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.453+0000 7f6692800640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c1a0a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6688005e30 tx=0x7f6688003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.453+0000 7f6686a00640 1 -- 172.21.15.42:0/2408508835 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688032c10 con 0x7f668c114a70 2024-05-27T10:20:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.453+0000 7f6686a00640 1 -- 172.21.15.42:0/2408508835 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6688032db0 con 0x7f668c114a70 2024-05-27T10:20:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.453+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c1a1350 con 0x7f668c114a70 2024-05-27T10:20:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.453+0000 7f6686a00640 1 -- 172.21.15.42:0/2408508835 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688030ab0 con 0x7f668c114a70 2024-05-27T10:20:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.453+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1ad690 con 0x7f668c114a70 2024-05-27T10:20:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.456+0000 7f669b7c7740 1 Processor -- start 2024-05-27T10:20:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f6686a00640 1 -- 172.21.15.42:0/2408508835 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6688030c50 con 0x7f668c114a70 2024-05-27T10:20:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 -- start start 2024-05-27T10:20:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x560888968a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888968fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5608888d9b90 con 0x5608885c61b0 2024-05-27T10:20:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5608888c9730 con 0x5608885a54d0 2024-05-27T10:20:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.457+0000 7f669b7c7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5608888cd770 con 0x56088859d2f0 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x560888968a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x560888968a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52922/0 (socket says 172.21.15.42:52922) 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 -- 172.21.15.42:0/989365741 learned_addr learned my addr 172.21.15.42:0/989365741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691e00640 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888968fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 -- 172.21.15.42:0/989365741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 msgr2=0x560888968fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888968fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 -- 172.21.15.42:0/989365741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 msgr2=0x5608889694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.458+0000 7f6691400640 1 -- 172.21.15.42:0/989365741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608888628b0 con 0x5608885c61b0 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f6691400640 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x560888968a60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6678009c60 tx=0x7f66780137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e9987cd691835a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f6686a00640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66740b29d0 0x7f66740b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f667f000640 1 -- 172.21.15.42:0/989365741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678013070 con 0x5608885c61b0 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f667f000640 1 -- 172.21.15.42:0/989365741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6678017070 con 0x5608885c61b0 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f667f000640 1 -- 172.21.15.42:0/989365741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678003c20 con 0x5608885c61b0 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f6686a00640 1 -- 172.21.15.42:0/2408508835 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f66881039e0 con 0x7f668c114a70 2024-05-27T10:20:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.459+0000 7f6691e00640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66740b29d0 0x7f66740b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f6691e00640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66740b29d0 0x7f66740b4ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x560888969e10 tx=0x7f6680005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 -- 172.21.15.42:0/989365741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 msgr2=0x560888968a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x560888968a60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6678009c60 tx=0x7f66780137d0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 -- 172.21.15.42:0/989365741 shutdown_connections 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888968fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 --2- 172.21.15.42:0/989365741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x560888968a60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.460+0000 7f669b7c7740 1 -- 172.21.15.42:0/989365741 >> 172.21.15.42:0/989365741 conn(0x5608885ba300 msgr2=0x560888965210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.461+0000 7f669b7c7740 1 -- 172.21.15.42:0/989365741 shutdown_connections 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.461+0000 7f669b7c7740 1 -- 172.21.15.42:0/989365741 wait complete. 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.461+0000 7f669b7c7740 1 -- start start 2024-05-27T10:20:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.461+0000 7f669b7c7740 1 Processor -- start 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56088859d2f0 0x560888972bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 0x560888973130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5608885af790 con 0x5608885a54d0 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560888871bc0 con 0x56088859d2f0 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560888975b90 con 0x5608885c61b0 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f669b7c7740 10 client.0 _ll_register_callbacks cb 0x56088884f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f6691400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 0x560888973130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f6691400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 0x560888973130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52926/0 (socket says 172.21.15.42:52926) 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.462+0000 7f6691400640 1 -- 172.21.15.42:0/4120072546 learned_addr learned my addr 172.21.15.42:0/4120072546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691e00640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56088859d2f0 0x560888972bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6692800640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691400640 1 -- 172.21.15.42:0/4120072546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 msgr2=0x56088897eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691400640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691400640 1 -- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56088859d2f0 msgr2=0x560888972bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691400640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56088859d2f0 0x560888972bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691400640 1 -- 172.21.15.42:0/4120072546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608885de210 con 0x5608885a54d0 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6691400640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 0x560888973130 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6678005d80 tx=0x7f667801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e10bfd6632e825 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6673400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.463+0000 7f6673400640 1 -- 172.21.15.42:0/4120072546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667801ce60 con 0x5608885a54d0 2024-05-27T10:20:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f6673400640 1 -- 172.21.15.42:0/4120072546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6678013070 con 0x5608885a54d0 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6678017070 con 0x5608885a54d0 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f6673400640 1 -- 172.21.15.42:0/4120072546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66780035f0 con 0x5608885a54d0 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f669b7c7740 10 client.15675 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608886aa620 con 0x5608885a54d0 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f6673400640 1 -- 172.21.15.42:0/4120072546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6678017070 con 0x5608885a54d0 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.464+0000 7f6673400640 1 -- 172.21.15.42:0/4120072546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f667800f440 con 0x5608885a54d0 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f669b7c7740 10 client.15675 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f669b7c7740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f669b7c7740 20 client.15675 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f669b7c7740 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560888584d80 0x560888984dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f669b7c7740 4 client.15675 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608888dbae0 con 0x560888584d80 2024-05-27T10:20:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.465+0000 7f6691e00640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560888584d80 0x560888984dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.466+0000 7f6691e00640 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560888584d80 0x560888984dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347a6025ca93f61d server_cookie=3cb07e815bc1977c in_seq=0 out_seq=0 2024-05-27T10:20:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.466+0000 7f6673400640 10 client.15675 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f6673400640 1 -- 172.21.15.42:0/4120072546 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5608888dbae0 con 0x560888584d80 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f6673400640 10 client.15675 handle_command_reply: tid=0 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 client.15675 shutdown 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 msgr2=0x560888973130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 0x560888973130 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6678005d80 tx=0x7f667801cc40 comp rx=0 tx=0).stop 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 shutdown_connections 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560888584d80 0x560888984dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885a54d0 0x560888973130 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 --2- 172.21.15.42:0/4120072546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56088859d2f0 0x560888972bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 >> 172.21.15.42:0/4120072546 conn(0x5608885ba300 msgr2=0x560888968890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f6673400640 0 client.15675 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 shutdown_connections 2024-05-27T10:20:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 1 -- 172.21.15.42:0/4120072546 wait complete. 2024-05-27T10:20:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.469+0000 7f669b7c7740 20 client.15675 trim_cache size 0 max 16384 2024-05-27T10:20:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.486+0000 7f669b7c7740 1 Processor -- start 2024-05-27T10:20:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 -- start start 2024-05-27T10:20:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 0x560888975320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x56088897ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b8670 con 0x56088859d2f0 2024-05-27T10:20:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b5f40 con 0x5608885a54d0 2024-05-27T10:20:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f669b7c7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b3d90 con 0x5608885c61b0 2024-05-27T10:20:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.487+0000 7f6691e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 0x560888975320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 0x560888975320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52942/0 (socket says 172.21.15.42:52942) 2024-05-27T10:20:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 -- 172.21.15.42:0/3360424440 learned_addr learned my addr 172.21.15.42:0/3360424440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6692800640 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691400640 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x56088897ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 -- 172.21.15.42:0/3360424440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 msgr2=0x56088897f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 -- 172.21.15.42:0/3360424440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 msgr2=0x56088897ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x56088897ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 -- 172.21.15.42:0/3360424440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560888965de0 con 0x56088859d2f0 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.488+0000 7f6691e00640 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 0x560888975320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f668c1a1b60 tx=0x7f6680016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975119cbbda73fba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.489+0000 7f667e600640 1 -- 172.21.15.42:0/3360424440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668002e070 con 0x56088859d2f0 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.489+0000 7f667e600640 1 -- 172.21.15.42:0/3360424440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66800044d0 con 0x56088859d2f0 2024-05-27T10:20:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.489+0000 7f667e600640 1 -- 172.21.15.42:0/3360424440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680029250 con 0x56088859d2f0 2024-05-27T10:20:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 -- 172.21.15.42:0/3360424440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 msgr2=0x560888975320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 0x560888975320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f668c1a1b60 tx=0x7f6680016000 comp rx=0 tx=0).stop 2024-05-27T10:20:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 -- 172.21.15.42:0/3360424440 shutdown_connections 2024-05-27T10:20:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5608885c61b0 0x56088897f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x56088897ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3360424440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56088859d2f0 0x560888975320 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 -- 172.21.15.42:0/3360424440 >> 172.21.15.42:0/3360424440 conn(0x5608885ba300 msgr2=0x56088896e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 -- 172.21.15.42:0/3360424440 shutdown_connections 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.490+0000 7f669b7c7740 1 -- 172.21.15.42:0/3360424440 wait complete. 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.491+0000 7f669b7c7740 1 -- start start 2024-05-27T10:20:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.491+0000 7f669b7c7740 1 Processor -- start 2024-05-27T10:20:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888977fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x5608889eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b4400 con 0x5608885c61b0 2024-05-27T10:20:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b5870 con 0x5608885a54d0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b4ad0 con 0x56088859d2f0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f669b7c7740 10 client.0 _ll_register_callbacks cb 0x7f6678009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f6692800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x5608889eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f6692800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x5608889eb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52956/0 (socket says 172.21.15.42:52956) 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f6692800640 1 -- 172.21.15.42:0/3025862125 learned_addr learned my addr 172.21.15.42:0/3025862125 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f6691e00640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888977fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.492+0000 7f6691400640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6692800640 1 -- 172.21.15.42:0/3025862125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 msgr2=0x560888977fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6692800640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888977fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6692800640 1 -- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 msgr2=0x5608889784f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6692800640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889784f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6692800640 1 -- 172.21.15.42:0/3025862125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560888978dd0 con 0x5608885c61b0 2024-05-27T10:20:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6691400640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6692800640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x5608889eb2d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f66880c80c0 tx=0x7f66880ca5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e07308efbede8d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6673400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6673400640 1 -- 172.21.15.42:0/3025862125 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66880cae50 con 0x5608885c61b0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f6673400640 1 -- 172.21.15.42:0/3025862125 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f66880cb2c0 con 0x5608885c61b0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.493+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66800044d0 con 0x5608885c61b0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f6673400640 1 -- 172.21.15.42:0/3025862125 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66880cf200 con 0x5608885c61b0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 10 client.15681 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608889ec910 con 0x5608885c61b0 2024-05-27T10:20:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f6673400640 1 -- 172.21.15.42:0/3025862125 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66800044d0 con 0x5608885c61b0 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f6673400640 1 -- 172.21.15.42:0/3025862125 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f66880d1410 con 0x5608885c61b0 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 10 client.15681 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 20 client.15681 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56088855a1c0 0x5608889ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 4 client.15681 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.494+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608888d52c0 con 0x56088855a1c0 2024-05-27T10:20:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.495+0000 7f6691e00640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56088855a1c0 0x5608889ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.496+0000 7f6691e00640 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56088855a1c0 0x5608889ef090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f555b529c67ea9ee server_cookie=53bfdfc6e4fae107 in_seq=0 out_seq=0 2024-05-27T10:20:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.496+0000 7f6673400640 10 client.15681 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f6673400640 1 -- 172.21.15.42:0/3025862125 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5608888d52c0 con 0x56088855a1c0 2024-05-27T10:20:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f6673400640 10 client.15681 handle_command_reply: tid=0 2024-05-27T10:20:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 client.15681 shutdown 2024-05-27T10:20:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 msgr2=0x5608889eb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x5608889eb2d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f66880c80c0 tx=0x7f66880ca5c0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 shutdown_connections 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608885c61b0 0x5608889eb2d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5608885a54d0 0x5608889784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56088855a1c0 0x5608889ef090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 --2- 172.21.15.42:0/3025862125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56088859d2f0 0x560888977fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.512+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 >> 172.21.15.42:0/3025862125 conn(0x5608885ba300 msgr2=0x56088896df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.513+0000 7f6673400640 0 client.15681 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.513+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 shutdown_connections 2024-05-27T10:20:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.513+0000 7f669b7c7740 1 -- 172.21.15.42:0/3025862125 wait complete. 2024-05-27T10:20:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.513+0000 7f669b7c7740 20 client.15681 trim_cache size 0 max 16384 2024-05-27T10:20:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:20:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:20:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66740b29d0 msgr2=0x7f66740b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66740b29d0 0x7f66740b4ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x560888969e10 tx=0x7f6680005f90 comp rx=0 tx=0).stop 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 msgr2=0x7f668c1a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c1a0a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6688005e30 tx=0x7f6688003f30 comp rx=0 tx=0).stop 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 shutdown_connections 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f66740b29d0 0x7f66740b4ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f668c114a70 0x7f668c1a0a20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f668c112110 0x7f668c1a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 --2- 172.21.15.42:0/2408508835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f668c076590 0x7f668c1a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.514+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 >> 172.21.15.42:0/2408508835 conn(0x7f668c0ff170 msgr2=0x7f668c0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.515+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 shutdown_connections 2024-05-27T10:20:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:25.515+0000 7f6693200640 1 -- 172.21.15.42:0/2408508835 wait complete. 2024-05-27T10:20:26.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:26 smithi042 ceph-mon[31661]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T10:20:26.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 -- 172.21.15.42:0/2360474530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 msgr2=0x7fa05c10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 --2- 172.21.15.42:0/2360474530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c10e370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa048009990 tx=0x7fa048009ca0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 -- 172.21.15.42:0/2360474530 shutdown_connections 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 --2- 172.21.15.42:0/2360474530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 0x7fa05c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 --2- 172.21.15.42:0/2360474530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 0x7fa05c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 --2- 172.21.15.42:0/2360474530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c10e370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.900+0000 7fa063c00700 1 -- 172.21.15.42:0/2360474530 >> 172.21.15.42:0/2360474530 conn(0x7fa05c0fedb0 msgr2=0x7fa05c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.901+0000 7fa063c00700 1 -- 172.21.15.42:0/2360474530 shutdown_connections 2024-05-27T10:20:26.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.901+0000 7fa063c00700 1 -- 172.21.15.42:0/2360474530 wait complete. 2024-05-27T10:20:26.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.902+0000 7fa063c00700 1 Processor -- start 2024-05-27T10:20:26.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 -- start start 2024-05-27T10:20:26.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 0x7fa05c1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 0x7fa05c103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c114300 con 0x7fa05c105e20 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c114180 con 0x7fa05c1054d0 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.903+0000 7fa063c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c114480 con 0x7fa05c10e900 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c103700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52978/0 (socket says 172.21.15.42:52978) 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 -- 172.21.15.42:0/3236265716 learned_addr learned my addr 172.21.15.42:0/3236265716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa061200700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 0x7fa05c1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa061c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 0x7fa05c103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 -- 172.21.15.42:0/3236265716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 msgr2=0x7fa05c103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 0x7fa05c103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 -- 172.21.15.42:0/3236265716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 msgr2=0x7fa05c1031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 0x7fa05c1031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa05be00700 1 -- 172.21.15.42:0/3236265716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c1a90d0 con 0x7fa05c105e20 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa061200700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 0x7fa05c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.904+0000 7fa061c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 0x7fa05c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.905+0000 7fa05be00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c103700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa048005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.905+0000 7fa059600700 1 -- 172.21.15.42:0/3236265716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048027070 con 0x7fa05c105e20 2024-05-27T10:20:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.905+0000 7fa059600700 1 -- 172.21.15.42:0/3236265716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa04800fb90 con 0x7fa05c105e20 2024-05-27T10:20:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.905+0000 7fa059600700 1 -- 172.21.15.42:0/3236265716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0480173f0 con 0x7fa05c105e20 2024-05-27T10:20:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.905+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c1a9360 con 0x7fa05c105e20 2024-05-27T10:20:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.905+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c1a9890 con 0x7fa05c105e20 2024-05-27T10:20:26.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.909+0000 7fa059600700 1 -- 172.21.15.42:0/3236265716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa04802a070 con 0x7fa05c105e20 2024-05-27T10:20:26.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.911+0000 7fa059600700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa0440b2a70 0x7fa0440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.911+0000 7fa059600700 1 -- 172.21.15.42:0/3236265716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fa0480e3620 con 0x7fa05c105e20 2024-05-27T10:20:26.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.911+0000 7fa061200700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa0440b2a70 0x7fa0440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.911+0000 7fa0726cbb80 1 Processor -- start 2024-05-27T10:20:26.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 -- start start 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa061200700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa0440b2a70 0x7fa0440b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa050009cb0 tx=0x7fa050009260 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 0x55981a14b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559819fc5fd0 0x55981a14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a145670 0x55981a145a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55981a072fd0 con 0x55981a0b1b80 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55981a062b70 con 0x559819fc5fd0 2024-05-27T10:20:26.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55981a066bb0 con 0x55981a145670 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.912+0000 7fa061200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 0x55981a14b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 0x55981a14b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52990/0 (socket says 172.21.15.42:52990) 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 -- 172.21.15.42:0/3678305456 learned_addr learned my addr 172.21.15.42:0/3678305456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa05be00700 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559819fc5fd0 0x55981a14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061c00700 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a145670 0x55981a145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 -- 172.21.15.42:0/3678305456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a145670 msgr2=0x55981a145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a145670 0x55981a145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 -- 172.21.15.42:0/3678305456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559819fc5fd0 msgr2=0x55981a14bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559819fc5fd0 0x55981a14bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 -- 172.21.15.42:0/3678305456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55981a02b5b0 con 0x55981a0b1b80 2024-05-27T10:20:26.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061c00700 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a145670 0x55981a145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:26.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.913+0000 7fa061200700 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 0x55981a14b5c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa050011fb0 tx=0x7fa050014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d2f8f4bc61ba2b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:26.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.914+0000 7fa04e000700 1 -- 172.21.15.42:0/3678305456 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050025070 con 0x55981a0b1b80 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.914+0000 7fa04e000700 1 -- 172.21.15.42:0/3678305456 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa050016070 con 0x55981a0b1b80 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.914+0000 7fa04e000700 1 -- 172.21.15.42:0/3678305456 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050012a60 con 0x55981a0b1b80 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3678305456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 msgr2=0x55981a14b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 0x55981a14b5c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa050011fb0 tx=0x7fa050014000 comp rx=0 tx=0).stop 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3678305456 shutdown_connections 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a145670 0x55981a145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a0b1b80 0x55981a14b5c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3678305456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559819fc5fd0 0x55981a14bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3678305456 >> 172.21.15.42:0/3678305456 conn(0x55981a05f520 msgr2=0x55981a1406e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3678305456 shutdown_connections 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.915+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3678305456 wait complete. 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.916+0000 7fa0726cbb80 1 -- start start 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.916+0000 7fa0726cbb80 1 Processor -- start 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 0x55981a14f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a0b1b80 0x55981a14f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a145670 0x55981a14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559819f69130 con 0x559819fc5fd0 2024-05-27T10:20:26.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55981a0038f0 con 0x55981a145670 2024-05-27T10:20:26.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55981a146460 con 0x55981a0b1b80 2024-05-27T10:20:26.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa0726cbb80 10 client.0 _ll_register_callbacks cb 0x55981a00ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:26.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa061200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 0x55981a14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa061200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 0x55981a14f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53006/0 (socket says 172.21.15.42:53006) 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.917+0000 7fa061200700 1 -- 172.21.15.42:0/3605617929 learned_addr learned my addr 172.21.15.42:0/3605617929 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa05be00700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a0b1b80 0x55981a14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061c00700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a145670 0x55981a14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061200700 1 -- 172.21.15.42:0/3605617929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a0b1b80 msgr2=0x55981a14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061200700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a0b1b80 0x55981a14f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061200700 1 -- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a145670 msgr2=0x55981a14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061200700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a145670 0x55981a14fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061200700 1 -- 172.21.15.42:0/3605617929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559819fb2a60 con 0x559819fc5fd0 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa061200700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 0x55981a14f420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa050009fa0 tx=0x7fa0500057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfc84ef2f081bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.918+0000 7fa042a00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:26.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa042a00700 1 -- 172.21.15.42:0/3605617929 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05002f030 con 0x559819fc5fd0 2024-05-27T10:20:26.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa042a00700 1 -- 172.21.15.42:0/3605617929 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fa050012720 con 0x559819fc5fd0 2024-05-27T10:20:26.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa050016070 con 0x559819fc5fd0 2024-05-27T10:20:26.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa042a00700 1 -- 172.21.15.42:0/3605617929 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa050039050 con 0x559819fc5fd0 2024-05-27T10:20:26.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa0726cbb80 10 client.15693 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:26.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55981a0331f0 con 0x559819fc5fd0 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.919+0000 7fa042a00700 1 -- 172.21.15.42:0/3605617929 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa050016070 con 0x559819fc5fd0 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa042a00700 1 -- 172.21.15.42:0/3605617929 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa050004570 con 0x559819fc5fd0 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa0726cbb80 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa0726cbb80 10 client.15693 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa0726cbb80 20 client.15693 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a15f0c0 0x55981a1614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa0726cbb80 4 client.15693 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.920+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55981a074f20 con 0x55981a15f0c0 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.921+0000 7fa05be00700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a15f0c0 0x55981a1614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.921+0000 7fa05be00700 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a15f0c0 0x55981a1614e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37a8fae78f78f6a server_cookie=967fc9756509c262 in_seq=0 out_seq=0 2024-05-27T10:20:26.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.921+0000 7fa042a00700 10 client.15693 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.923+0000 7fa042a00700 1 -- 172.21.15.42:0/3605617929 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55981a074f20 con 0x55981a15f0c0 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.923+0000 7fa042a00700 10 client.15693 handle_command_reply: tid=0 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 client.15693 shutdown 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 msgr2=0x55981a14f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 0x55981a14f420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa050009fa0 tx=0x7fa0500057e0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 shutdown_connections 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a15f0c0 0x55981a1614e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a145670 0x55981a14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559819fc5fd0 0x55981a14f420 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3605617929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55981a0b1b80 0x55981a14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa042a00700 0 client.15693 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 >> 172.21.15.42:0/3605617929 conn(0x55981a05f520 msgr2=0x55981a13fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 shutdown_connections 2024-05-27T10:20:26.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.924+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3605617929 wait complete. 2024-05-27T10:20:26.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.925+0000 7fa0726cbb80 20 client.15693 trim_cache size 0 max 16384 2024-05-27T10:20:26.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:26 smithi104 ceph-mon[34812]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T10:20:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 Processor -- start 2024-05-27T10:20:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 -- start start 2024-05-27T10:20:26.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a14c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1b36f0 con 0x55981a145670 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1b0f80 con 0x55981a0b1b80 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.948+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1aedf0 con 0x559819fc5fd0 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a14cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53018/0 (socket says 172.21.15.42:53018) 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 -- 172.21.15.42:0/3002569158 learned_addr learned my addr 172.21.15.42:0/3002569158 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061200700 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa05be00700 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 -- 172.21.15.42:0/3002569158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 msgr2=0x55981a15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 -- 172.21.15.42:0/3002569158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 msgr2=0x55981a14c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a14c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa061c00700 1 -- 172.21.15.42:0/3002569158 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55981a14b5c0 con 0x55981a145670 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.949+0000 7fa05be00700 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a14c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.950+0000 7fa061c00700 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a14cd90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa054007580 tx=0x7fa05400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6407b6b0dcea90 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.950+0000 7fa04d600700 1 -- 172.21.15.42:0/3002569158 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05401a020 con 0x55981a145670 2024-05-27T10:20:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.950+0000 7fa04d600700 1 -- 172.21.15.42:0/3002569158 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0540168c0 con 0x55981a145670 2024-05-27T10:20:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.950+0000 7fa04d600700 1 -- 172.21.15.42:0/3002569158 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054015890 con 0x55981a145670 2024-05-27T10:20:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3002569158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 msgr2=0x55981a14cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a14cd90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa054007580 tx=0x7fa05400b040 comp rx=0 tx=0).stop 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3002569158 shutdown_connections 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a14cd90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/3002569158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a14c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3002569158 >> 172.21.15.42:0/3002569158 conn(0x55981a05f520 msgr2=0x55981a1499b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3002569158 shutdown_connections 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.951+0000 7fa0726cbb80 1 -- 172.21.15.42:0/3002569158 wait complete. 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 -- start start 2024-05-27T10:20:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 Processor -- start 2024-05-27T10:20:26.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a164900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a1631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1af440 con 0x55981a145670 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1b08b0 con 0x55981a0b1b80 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.952+0000 7fa0726cbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1afb10 con 0x559819fc5fd0 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa0726cbb80 10 client.0 _ll_register_callbacks cb 0x55981a00ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a1631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a1631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53022/0 (socket says 172.21.15.42:53022) 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 -- 172.21.15.42:0/1619680226 learned_addr learned my addr 172.21.15.42:0/1619680226 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061200700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa05be00700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 -- 172.21.15.42:0/1619680226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 msgr2=0x55981a164900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a164900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 -- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 msgr2=0x55981a162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.953+0000 7fa061c00700 1 -- 172.21.15.42:0/1619680226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55981a163710 con 0x55981a145670 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.954+0000 7fa061c00700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a1631d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa054009960 tx=0x7fa054017a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81484d16de5b1c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.954+0000 7fa042a00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.954+0000 7fa042a00700 1 -- 172.21.15.42:0/1619680226 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0540156d0 con 0x55981a145670 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.954+0000 7fa042a00700 1 -- 172.21.15.42:0/1619680226 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fa054015870 con 0x55981a145670 2024-05-27T10:20:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.954+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0540168c0 con 0x55981a145670 2024-05-27T10:20:26.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa042a00700 1 -- 172.21.15.42:0/1619680226 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa054015a70 con 0x55981a145670 2024-05-27T10:20:26.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 10 client.15699 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:26.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55981a1ca4e0 con 0x55981a145670 2024-05-27T10:20:26.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa042a00700 1 -- 172.21.15.42:0/1619680226 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0540168c0 con 0x55981a145670 2024-05-27T10:20:26.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa042a00700 1 -- 172.21.15.42:0/1619680226 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa054028070 con 0x55981a145670 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 10 client.15699 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 10 client.15699 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 20 client.15699 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a1cab70 0x55981a1ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 4 client.15699 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.4, "await_for": 45.0}] 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.955+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.4, "await_for": 45.0}) -- 0x55981a14afe0 con 0x55981a1cab70 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.956+0000 7fa061200700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a1cab70 0x55981a1ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.957+0000 7fa061200700 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a1cab70 0x55981a1ccf90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57520ae039496690 server_cookie=c4e5e0930c617df3 in_seq=0 out_seq=0 2024-05-27T10:20:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.957+0000 7fa042a00700 10 client.15699 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:26.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.983+0000 7fa042a00700 1 -- 172.21.15.42:0/1619680226 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55981a14afe0 con 0x55981a1cab70 2024-05-27T10:20:26.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.983+0000 7fa042a00700 10 client.15699 handle_command_reply: tid=0 2024-05-27T10:20:26.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.983+0000 7fa0726cbb80 1 client.15699 shutdown 2024-05-27T10:20:26.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 msgr2=0x55981a1631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a1631d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa054009960 tx=0x7fa054017a80 comp rx=0 tx=0).stop 2024-05-27T10:20:26.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 shutdown_connections 2024-05-27T10:20:26.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55981a1cab70 0x55981a1ccf90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55981a145670 0x55981a1631d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa042a00700 0 client.15699 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559819fc5fd0 0x55981a164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 --2- 172.21.15.42:0/1619680226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55981a0b1b80 0x55981a162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 >> 172.21.15.42:0/1619680226 conn(0x55981a05f520 msgr2=0x55981a1499b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 shutdown_connections 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 1 -- 172.21.15.42:0/1619680226 wait complete. 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.984+0000 7fa0726cbb80 20 client.15699 trim_cache size 0 max 16384 2024-05-27T10:20:26.986 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 26, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 2, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "91f1c12c": { 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 2, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:20:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 149.4, 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.985+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa0440b2a70 msgr2=0x7fa0440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.985+0000 7fa063c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa0440b2a70 0x7fa0440b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa050009cb0 tx=0x7fa050009260 comp rx=0 tx=0).stop 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.985+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 msgr2=0x7fa05c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.985+0000 7fa063c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c103700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa048005e80 comp rx=0 tx=0).stop 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 shutdown_connections 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa0440b2a70 0x7fa0440b4f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa05c10e900 0x7fa05c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa05c1054d0 0x7fa05c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 --2- 172.21.15.42:0/3236265716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa05c105e20 0x7fa05c103700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:26.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 >> 172.21.15.42:0/3236265716 conn(0x7fa05c0fedb0 msgr2=0x7fa05c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:26.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 shutdown_connections 2024-05-27T10:20:26.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:26.986+0000 7fa063c00700 1 -- 172.21.15.42:0/3236265716 wait complete. 2024-05-27T10:20:27.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:26 smithi143 ceph-mon[35545]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T10:20:28.170 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '91f1c12c', quiesce took 10.3 seconds. Will release after: 19.099999999999998 2024-05-27T10:20:28.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:28 smithi104 ceph-mon[34812]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-27T10:20:29.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:28 smithi143 ceph-mon[35545]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-27T10:20:29.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:28 smithi042 ceph-mon[31661]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-27T10:20:30.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:30 smithi104 ceph-mon[34812]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-27T10:20:31.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:30 smithi143 ceph-mon[35545]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-27T10:20:31.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:30 smithi042 ceph-mon[31661]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-27T10:20:32.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:32 smithi104 ceph-mon[34812]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:33.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:32 smithi143 ceph-mon[35545]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:33.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:32 smithi042 ceph-mon[31661]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:34.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:34 smithi104 ceph-mon[34812]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:35.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:34 smithi143 ceph-mon[35545]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:35.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:34 smithi042 ceph-mon[31661]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:36.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:36 smithi104 ceph-mon[34812]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:20:37.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:36 smithi143 ceph-mon[35545]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:20:37.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:36 smithi042 ceph-mon[31661]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 -- 172.21.15.42:0/3908619395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 msgr2=0x7f721410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 --2- 172.21.15.42:0/3908619395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721410da30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7200009900 tx=0x7f720002a410 comp rx=0 tx=0).stop 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 -- 172.21.15.42:0/3908619395 shutdown_connections 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 --2- 172.21.15.42:0/3908619395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721410e000 0x7f72141104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 --2- 172.21.15.42:0/3908619395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721410da30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 --2- 172.21.15.42:0/3908619395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7214104bb0 0x7f7214104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 -- 172.21.15.42:0/3908619395 >> 172.21.15.42:0/3908619395 conn(0x7f72140fe210 msgr2=0x7f7214100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 -- 172.21.15.42:0/3908619395 shutdown_connections 2024-05-27T10:20:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.749+0000 7f7213200640 1 -- 172.21.15.42:0/3908619395 wait complete. 2024-05-27T10:20:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.751+0000 7f7213200640 1 Processor -- start 2024-05-27T10:20:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.751+0000 7f7213200640 1 -- start start 2024-05-27T10:20:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.751+0000 7f7213200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7214104bb0 0x7f721419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7213200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7213200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f721410e000 0x7f72141a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7213200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7214113b10 con 0x7f721410b610 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7213200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7214113990 con 0x7f7214104bb0 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7213200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7214113c90 con 0x7f721410e000 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43214/0 (socket says 172.21.15.42:43214) 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 -- 172.21.15.42:0/492776593 learned_addr learned my addr 172.21.15.42:0/492776593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211e00640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7214104bb0 0x7f721419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7212800640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f721410e000 0x7f72141a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 -- 172.21.15.42:0/492776593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f721410e000 msgr2=0x7f72141a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f721410e000 0x7f72141a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 -- 172.21.15.42:0/492776593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7214104bb0 msgr2=0x7f721419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7214104bb0 0x7f721419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.752+0000 7f7211400640 1 -- 172.21.15.42:0/492776593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72141a1650 con 0x7f721410b610 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.753+0000 7f7211400640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721419cd50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7200009230 tx=0x7f7200004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.753+0000 7f720aa00640 1 -- 172.21.15.42:0/492776593 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720003f030 con 0x7f721410b610 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.753+0000 7f720aa00640 1 -- 172.21.15.42:0/492776593 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7200003a80 con 0x7f721410b610 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.753+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72141a18e0 con 0x7f721410b610 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.753+0000 7f720aa00640 1 -- 172.21.15.42:0/492776593 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200035070 con 0x7f721410b610 2024-05-27T10:20:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.754+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72141a1e20 con 0x7f721410b610 2024-05-27T10:20:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.757+0000 7f722000b740 1 Processor -- start 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.757+0000 7f722000b740 1 -- start start 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f722000b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9c61b0 0x5580cad68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f722000b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca99d2f0 0x5580cad68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f722000b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f722000b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5580cacd9b90 con 0x5580ca9a54d0 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f722000b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5580cacc9730 con 0x5580ca9c61b0 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f720aa00640 1 -- 172.21.15.42:0/492776593 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7200042030 con 0x7f721410b610 2024-05-27T10:20:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f722000b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5580caccd770 con 0x5580ca99d2f0 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f7211400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f7211400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad694e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43224/0 (socket says 172.21.15.42:43224) 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f7211400640 1 -- 172.21.15.42:0/1302817053 learned_addr learned my addr 172.21.15.42:0/1302817053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f7211e00640 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca99d2f0 0x5580cad68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.758+0000 7f7212800640 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9c61b0 0x5580cad68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.759+0000 7f7211400640 1 -- 172.21.15.42:0/1302817053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca99d2f0 msgr2=0x5580cad68fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.759+0000 7f7211400640 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca99d2f0 0x5580cad68fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.759+0000 7f7211400640 1 -- 172.21.15.42:0/1302817053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9c61b0 msgr2=0x5580cad68a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.759+0000 7f7211400640 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9c61b0 0x5580cad68a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.759+0000 7f7211400640 1 -- 172.21.15.42:0/1302817053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cac628b0 con 0x5580ca9a54d0 2024-05-27T10:20:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.759+0000 7f7211400640 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad694e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7200100440 tx=0x7f7200106f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a7703792d32b54 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.760+0000 7f71ff400640 1 -- 172.21.15.42:0/1302817053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200109820 con 0x5580ca9a54d0 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.760+0000 7f71ff400640 1 -- 172.21.15.42:0/1302817053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7200109e50 con 0x5580ca9a54d0 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.760+0000 7f71ff400640 1 -- 172.21.15.42:0/1302817053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200107c90 con 0x5580ca9a54d0 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.760+0000 7f720aa00640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71f80b29d0 0x7f71f80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.760+0000 7f720aa00640 1 -- 172.21.15.42:0/492776593 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7200102dc0 con 0x7f721410b610 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.760+0000 7f7212800640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71f80b29d0 0x7f71f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 -- 172.21.15.42:0/1302817053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 msgr2=0x5580cad694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad694e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7200100440 tx=0x7f7200106f70 comp rx=0 tx=0).stop 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 -- 172.21.15.42:0/1302817053 shutdown_connections 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad694e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca99d2f0 0x5580cad68fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 --2- 172.21.15.42:0/1302817053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9c61b0 0x5580cad68a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 -- 172.21.15.42:0/1302817053 >> 172.21.15.42:0/1302817053 conn(0x5580ca9ba300 msgr2=0x5580cad65210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f7212800640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71f80b29d0 0x7f71f80b4ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5580cad65bd0 tx=0x7f720c0096e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 -- 172.21.15.42:0/1302817053 shutdown_connections 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.761+0000 7f722000b740 1 -- 172.21.15.42:0/1302817053 wait complete. 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 -- start start 2024-05-27T10:20:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 Processor -- start 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca99d2f0 0x5580cad72c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5580ca9af790 con 0x5580ca9a54d0 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5580cac71bc0 con 0x5580ca99d2f0 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.762+0000 7f722000b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5580cad75950 con 0x5580ca9c61b0 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f722000b740 10 client.0 _ll_register_callbacks cb 0x5580cac4f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43230/0 (socket says 172.21.15.42:43230) 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 -- 172.21.15.42:0/1249917166 learned_addr learned my addr 172.21.15.42:0/1249917166 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211e00640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca99d2f0 0x5580cad72c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7212800640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 -- 172.21.15.42:0/1249917166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 msgr2=0x5580cad7ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad7ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 -- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca99d2f0 msgr2=0x5580cad72c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca99d2f0 0x5580cad72c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211400640 1 -- 172.21.15.42:0/1249917166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ca9de210 con 0x5580ca9a54d0 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.763+0000 7f7211e00640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca99d2f0 0x5580cad72c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f7211400640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad731c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72000ca020 tx=0x7f720002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc4fc0487e8dfa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f71f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f71f3e00640 1 -- 172.21.15.42:0/1249917166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200049070 con 0x5580ca9a54d0 2024-05-27T10:20:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f71f3e00640 1 -- 172.21.15.42:0/1249917166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f72000c6680 con 0x5580ca9a54d0 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7200109e50 con 0x5580ca9a54d0 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f71f3e00640 1 -- 172.21.15.42:0/1249917166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f72000c5f50 con 0x5580ca9a54d0 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f722000b740 10 client.15711 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580caaaa620 con 0x5580ca9a54d0 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.764+0000 7f71f3e00640 1 -- 172.21.15.42:0/1249917166 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7200109e50 con 0x5580ca9a54d0 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f71f3e00640 1 -- 172.21.15.42:0/1249917166 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7200106020 con 0x5580ca9a54d0 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f722000b740 10 client.15711 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f722000b740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f722000b740 20 client.15711 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f722000b740 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580ca984d80 0x5580cad84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f722000b740 4 client.15711 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580cacdbae0 con 0x5580ca984d80 2024-05-27T10:20:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.765+0000 7f7211e00640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580ca984d80 0x5580cad84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.766+0000 7f7211e00640 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580ca984d80 0x5580cad84a10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e500f9efb2318ac4 server_cookie=3b3073d3674775a6 in_seq=0 out_seq=0 2024-05-27T10:20:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.766+0000 7f71f3e00640 10 client.15711 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.768+0000 7f71f3e00640 1 -- 172.21.15.42:0/1249917166 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5580cacdbae0 con 0x5580ca984d80 2024-05-27T10:20:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.768+0000 7f71f3e00640 10 client.15711 handle_command_reply: tid=0 2024-05-27T10:20:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.768+0000 7f722000b740 1 client.15711 shutdown 2024-05-27T10:20:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 msgr2=0x5580cad731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad731c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72000ca020 tx=0x7f720002a410 comp rx=0 tx=0).stop 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 shutdown_connections 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580ca984d80 0x5580cad84a10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad7ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad731c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 --2- 172.21.15.42:0/1249917166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca99d2f0 0x5580cad72c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 >> 172.21.15.42:0/1249917166 conn(0x5580ca9ba300 msgr2=0x5580cad6a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f71f3e00640 0 client.15711 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 shutdown_connections 2024-05-27T10:20:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 1 -- 172.21.15.42:0/1249917166 wait complete. 2024-05-27T10:20:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.769+0000 7f722000b740 20 client.15711 trim_cache size 0 max 16384 2024-05-27T10:20:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.786+0000 7f722000b740 1 Processor -- start 2024-05-27T10:20:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.786+0000 7f722000b740 1 -- start start 2024-05-27T10:20:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f722000b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 0x5580cad753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f722000b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9a54d0 0x5580cad7ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f722000b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad87a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f722000b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72141b4260 con 0x5580ca99d2f0 2024-05-27T10:20:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f722000b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72141b1b30 con 0x5580ca9a54d0 2024-05-27T10:20:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f722000b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f72141af980 con 0x5580ca9c61b0 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f7211e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 0x5580cad753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f7211e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 0x5580cad753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43240/0 (socket says 172.21.15.42:43240) 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f7211e00640 1 -- 172.21.15.42:0/3052266162 learned_addr learned my addr 172.21.15.42:0/3052266162 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.787+0000 7f7211400640 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9a54d0 0x5580cad7ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7212800640 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad87a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7211e00640 1 -- 172.21.15.42:0/3052266162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 msgr2=0x5580cad87a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7211e00640 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad87a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7211e00640 1 -- 172.21.15.42:0/3052266162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9a54d0 msgr2=0x5580cad7ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7211e00640 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9a54d0 0x5580cad7ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7211e00640 1 -- 172.21.15.42:0/3052266162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cad6ce80 con 0x5580ca99d2f0 2024-05-27T10:20:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f7211e00640 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 0x5580cad753b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5580cad661a0 tx=0x7f7204013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978063887e0e05e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.788+0000 7f71fea00640 1 -- 172.21.15.42:0/3052266162 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204023070 con 0x5580ca99d2f0 2024-05-27T10:20:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.789+0000 7f71fea00640 1 -- 172.21.15.42:0/3052266162 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f720401c8f0 con 0x5580ca99d2f0 2024-05-27T10:20:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.789+0000 7f71fea00640 1 -- 172.21.15.42:0/3052266162 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040116a0 con 0x5580ca99d2f0 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.789+0000 7f722000b740 1 -- 172.21.15.42:0/3052266162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 msgr2=0x5580cad753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.789+0000 7f722000b740 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 0x5580cad753b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5580cad661a0 tx=0x7f7204013830 comp rx=0 tx=0).stop 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 -- 172.21.15.42:0/3052266162 shutdown_connections 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad87a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca9a54d0 0x5580cad7ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 --2- 172.21.15.42:0/3052266162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca99d2f0 0x5580cad753b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 -- 172.21.15.42:0/3052266162 >> 172.21.15.42:0/3052266162 conn(0x5580ca9ba300 msgr2=0x5580cad72bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 -- 172.21.15.42:0/3052266162 shutdown_connections 2024-05-27T10:20:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 -- 172.21.15.42:0/3052266162 wait complete. 2024-05-27T10:20:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.790+0000 7f722000b740 1 -- start start 2024-05-27T10:20:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 Processor -- start 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad87e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca95a1c0 0x5580cad85710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72141afff0 con 0x5580ca9a54d0 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72141b1460 con 0x5580ca95a1c0 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f72141b06c0 con 0x5580ca9c61b0 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.791+0000 7f722000b740 10 client.0 _ll_register_callbacks cb 0x7f720c009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad87e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad87e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43250/0 (socket says 172.21.15.42:43250) 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 -- 172.21.15.42:0/2591914548 learned_addr learned my addr 172.21.15.42:0/2591914548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 -- 172.21.15.42:0/2591914548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 msgr2=0x5580cad851d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211400640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7212800640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca95a1c0 0x5580cad85710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad851d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 -- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca95a1c0 msgr2=0x5580cad85710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca95a1c0 0x5580cad85710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211e00640 1 -- 172.21.15.42:0/2591914548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cad85c50 con 0x5580ca9a54d0 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7212800640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca95a1c0 0x5580cad85710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.792+0000 7f7211400640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.793+0000 7f7211e00640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad87e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f720400b730 tx=0x7f720400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3787301d3ab3bd49 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.793+0000 7f71f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.793+0000 7f71f3e00640 1 -- 172.21.15.42:0/2591914548 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720402e020 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.793+0000 7f71f3e00640 1 -- 172.21.15.42:0/2591914548 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f720401c460 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.793+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720401c8f0 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f71f3e00640 1 -- 172.21.15.42:0/2591914548 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7204023040 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 10 client.15717 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580cadeca10 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f71f3e00640 1 -- 172.21.15.42:0/2591914548 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f720401c8f0 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f71f3e00640 1 -- 172.21.15.42:0/2591914548 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7204004730 con 0x5580ca9a54d0 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 10 client.15717 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 20 client.15717 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580caded1c0 0x5580cadef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 4 client.15717 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:20:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.794+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580cacd52c0 con 0x5580caded1c0 2024-05-27T10:20:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.795+0000 7f7211400640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580caded1c0 0x5580cadef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.796+0000 7f7211400640 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580caded1c0 0x5580cadef600 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b5d37067c3c20b server_cookie=6385bb3033735a34 in_seq=0 out_seq=0 2024-05-27T10:20:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.796+0000 7f71f3e00640 10 client.15717 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f71f3e00640 1 -- 172.21.15.42:0/2591914548 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5580cacd52c0 con 0x5580caded1c0 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f71f3e00640 10 client.15717 handle_command_reply: tid=0 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 client.15717 shutdown 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 msgr2=0x5580cad87e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad87e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f720400b730 tx=0x7f720400bee0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 shutdown_connections 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5580ca95a1c0 0x5580cad85710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5580caded1c0 0x5580cadef600 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5580ca9c61b0 0x5580cad851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f722000b740 1 --2- 172.21.15.42:0/2591914548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580ca9a54d0 0x5580cad87e10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.808+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 >> 172.21.15.42:0/2591914548 conn(0x5580ca9ba300 msgr2=0x5580cad6d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.807+0000 7f71f3e00640 0 client.15717 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.808+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 shutdown_connections 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.808+0000 7f722000b740 1 -- 172.21.15.42:0/2591914548 wait complete. 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.808+0000 7f722000b740 20 client.15717 trim_cache size 0 max 16384 2024-05-27T10:20:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:20:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:20:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:20:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71f80b29d0 msgr2=0x7f71f80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71f80b29d0 0x7f71f80b4ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5580cad65bd0 tx=0x7f720c0096e0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 msgr2=0x7f721419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721419cd50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7200009230 tx=0x7f7200004270 comp rx=0 tx=0).stop 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 shutdown_connections 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71f80b29d0 0x7f71f80b4ee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f721410e000 0x7f72141a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721410b610 0x7f721419cd50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 --2- 172.21.15.42:0/492776593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7214104bb0 0x7f721419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 >> 172.21.15.42:0/492776593 conn(0x7f72140fe210 msgr2=0x7f7214109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 shutdown_connections 2024-05-27T10:20:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:38.809+0000 7f7213200640 1 -- 172.21.15.42:0/492776593 wait complete. 2024-05-27T10:20:38.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:38 smithi104 ceph-mon[34812]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:39.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:38 smithi143 ceph-mon[35545]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:39.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:38 smithi042 ceph-mon[31661]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:40.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:40 smithi104 ceph-mon[34812]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-27T10:20:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:40 smithi143 ceph-mon[35545]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-27T10:20:41.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:40 smithi042 ceph-mon[31661]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-27T10:20:43.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:42 smithi143 ceph-mon[35545]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:42 smithi042 ceph-mon[31661]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:43.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:42 smithi104 ceph-mon[34812]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:45.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:44 smithi143 ceph-mon[35545]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:45.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:44 smithi042 ceph-mon[31661]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:45.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:44 smithi104 ceph-mon[34812]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:47.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:46 smithi143 ceph-mon[35545]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:47.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:46 smithi042 ceph-mon[31661]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:47.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:46 smithi104 ceph-mon[34812]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:47.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:20:47.719 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:49.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:48 smithi143 ceph-mon[35545]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:49.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:48 smithi042 ceph-mon[31661]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:49.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:48 smithi104 ceph-mon[34812]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.255+0000 7f5d7f200700 1 -- 172.21.15.42:0/4108554814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 msgr2=0x7f5d8010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.255+0000 7f5d7f200700 1 --2- 172.21.15.42:0/4108554814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d8010e120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009990 tx=0x7f5d6c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 -- 172.21.15.42:0/4108554814 shutdown_connections 2024-05-27T10:20:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 --2- 172.21.15.42:0/4108554814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d8010e6f0 0x7f5d80110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 --2- 172.21.15.42:0/4108554814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d80105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 --2- 172.21.15.42:0/4108554814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d8010e120 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 -- 172.21.15.42:0/4108554814 >> 172.21.15.42:0/4108554814 conn(0x7f5d80078ab0 msgr2=0x7f5d80078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 -- 172.21.15.42:0/4108554814 shutdown_connections 2024-05-27T10:20:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.256+0000 7f5d7f200700 1 -- 172.21.15.42:0/4108554814 wait complete. 2024-05-27T10:20:50.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.257+0000 7f5d7f200700 1 Processor -- start 2024-05-27T10:20:50.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 -- start start 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d801a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d801a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d8010e6f0 0x7f5d801aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80114150 con 0x7f5d80105bd0 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80113fd0 con 0x7f5d8010e6f0 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.258+0000 7f5d7f200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801142d0 con 0x7f5d80105280 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.259+0000 7f5d7d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d801a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.259+0000 7f5d7de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d801a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:50.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.259+0000 7f5d7de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d801a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:33948/0 (socket says 172.21.15.42:33948) 2024-05-27T10:20:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.259+0000 7f5d7de00700 1 -- 172.21.15.42:0/1405195153 learned_addr learned my addr 172.21.15.42:0/1405195153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.259+0000 7f5d7e800700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d8010e6f0 0x7f5d801aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7de00700 1 -- 172.21.15.42:0/1405195153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d8010e6f0 msgr2=0x7f5d801aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7de00700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d8010e6f0 0x7f5d801aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7de00700 1 -- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 msgr2=0x7f5d801a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7de00700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d801a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7de00700 1 -- 172.21.15.42:0/1405195153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d801ad3f0 con 0x7f5d80105280 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7d400700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d801a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.260+0000 7f5d7de00700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d801a2000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d950 tx=0x7f5d7000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.261+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70016070 con 0x7f5d80105280 2024-05-27T10:20:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.261+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d801ad680 con 0x7f5d80105280 2024-05-27T10:20:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.261+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801adbe0 con 0x7f5d80105280 2024-05-27T10:20:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.261+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d70009dd0 con 0x7f5d80105280 2024-05-27T10:20:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.261+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000f3c0 con 0x7f5d80105280 2024-05-27T10:20:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.263+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d60005330 con 0x7f5d80105280 2024-05-27T10:20:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.269+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5d70019070 con 0x7f5d80105280 2024-05-27T10:20:50.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.270+0000 7f5d76a00700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d640b2a70 0x7f5d640b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:50.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.270+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5d700e2100 con 0x7f5d80105280 2024-05-27T10:20:50.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.271+0000 7f5d7d400700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d640b2a70 0x7f5d640b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.272+0000 7f5d7d400700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d640b2a70 0x7f5d640b4f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c006010 tx=0x7f5d6c00b540 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:50.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.282+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d700a9b50 con 0x7f5d80105280 2024-05-27T10:20:50.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.610+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d60005620 con 0x7f5d80105280 2024-05-27T10:20:50.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.613+0000 7f5d76a00700 1 -- 172.21.15.42:0/1405195153 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f5d700a9450 con 0x7f5d80105280 2024-05-27T10:20:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:20:50.614 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:20:50.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d640b2a70 msgr2=0x7f5d640b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:50.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d640b2a70 0x7f5d640b4f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c006010 tx=0x7f5d6c00b540 comp rx=0 tx=0).stop 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 msgr2=0x7f5d801a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d801a2000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d950 tx=0x7f5d7000dc60 comp rx=0 tx=0).stop 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 shutdown_connections 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5d640b2a70 0x7f5d640b4f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d8010e6f0 0x7f5d801aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d80105280 0x7f5d801a2000 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 --2- 172.21.15.42:0/1405195153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d80105bd0 0x7f5d801a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:50.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.616+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 >> 172.21.15.42:0/1405195153 conn(0x7f5d80078ab0 msgr2=0x7f5d80104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:50.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.617+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 shutdown_connections 2024-05-27T10:20:50.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:50.617+0000 7f5d7f200700 1 -- 172.21.15.42:0/1405195153 wait complete. 2024-05-27T10:20:50.618 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:20:50.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:20:51.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:50 smithi143 ceph-mon[35545]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:51.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:50 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1405195153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:20:51.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:50 smithi042 ceph-mon[31661]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:51.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:50 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1405195153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:20:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:50 smithi104 ceph-mon[34812]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:50 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1405195153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:20:51.701 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id 91f1c12c --release --await' 2024-05-27T10:20:51.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id 91f1c12c --release --await 2024-05-27T10:20:51.760 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.071+0000 7f1652a00640 1 -- 172.21.15.42:0/2733283538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c075420 msgr2=0x7f164c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.071+0000 7f1652a00640 1 --2- 172.21.15.42:0/2733283538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c075420 0x7f164c075830 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1640009900 tx=0x7f164002a480 comp rx=0 tx=0).stop 2024-05-27T10:20:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 -- 172.21.15.42:0/2733283538 shutdown_connections 2024-05-27T10:20:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 --2- 172.21.15.42:0/2733283538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f164c110320 0x7f164c112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 --2- 172.21.15.42:0/2733283538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 0x7f164c10fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 --2- 172.21.15.42:0/2733283538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c075420 0x7f164c075830 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 -- 172.21.15.42:0/2733283538 >> 172.21.15.42:0/2733283538 conn(0x7f164c079100 msgr2=0x7f164c079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 -- 172.21.15.42:0/2733283538 shutdown_connections 2024-05-27T10:20:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.072+0000 7f1652a00640 1 -- 172.21.15.42:0/2733283538 wait complete. 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 Processor -- start 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 -- start start 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f164c075420 0x7f164c1a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 0x7f164c1a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 0x7f164c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f164c115d90 con 0x7f164c110320 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f164c115c10 con 0x7f164c075420 2024-05-27T10:20:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.073+0000 7f1652a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f164c115f10 con 0x7f164c10d9c0 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 0x7f164c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 0x7f164c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47810/0 (socket says 172.21.15.42:47810) 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 -- 172.21.15.42:0/721614149 learned_addr learned my addr 172.21.15.42:0/721614149 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1651600640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f164c075420 0x7f164c1a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1650c00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 0x7f164c1a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 -- 172.21.15.42:0/721614149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 msgr2=0x7f164c1a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 0x7f164c1a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 -- 172.21.15.42:0/721614149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f164c075420 msgr2=0x7f164c1a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f164c075420 0x7f164c1a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1652000640 1 -- 172.21.15.42:0/721614149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c19cc90 con 0x7f164c110320 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.074+0000 7f1650c00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 0x7f164c1a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.075+0000 7f1652000640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 0x7f164c19c650 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f164400cce0 tx=0x7f164400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.075+0000 7f164a000640 1 -- 172.21.15.42:0/721614149 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164400f450 con 0x7f164c110320 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.075+0000 7f164a000640 1 -- 172.21.15.42:0/721614149 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f164400fa80 con 0x7f164c110320 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.075+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c19cf80 con 0x7f164c110320 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.075+0000 7f164a000640 1 -- 172.21.15.42:0/721614149 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644018ad0 con 0x7f164c110320 2024-05-27T10:20:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.076+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c19d480 con 0x7f164c110320 2024-05-27T10:20:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.079+0000 7f165af71740 1 Processor -- start 2024-05-27T10:20:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.082+0000 7f164a000640 1 -- 172.21.15.42:0/721614149 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f164400f5f0 con 0x7f164c110320 2024-05-27T10:20:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.082+0000 7f165af71740 1 -- start start 2024-05-27T10:20:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.083+0000 7f165af71740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33eeca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.083+0000 7f165af71740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b212f0 0x55bc33eecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.083+0000 7f165af71740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 0x55bc33eed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.083+0000 7f165af71740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bc33e5db90 con 0x55bc33b212f0 2024-05-27T10:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.083+0000 7f165af71740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc33e4d730 con 0x55bc33b4a1b0 2024-05-27T10:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.083+0000 7f165af71740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bc33e51770 con 0x55bc33b294d0 2024-05-27T10:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.084+0000 7f1650c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 0x55bc33eed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.084+0000 7f1650c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 0x55bc33eed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:33968/0 (socket says 172.21.15.42:33968) 2024-05-27T10:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.084+0000 7f1650c00640 1 -- 172.21.15.42:0/3227750927 learned_addr learned my addr 172.21.15.42:0/3227750927 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.084+0000 7f1650c00640 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33eeca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.084+0000 7f1651600640 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b212f0 0x55bc33eecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f164a000640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f16340b2950 0x7f16340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1650c00640 1 -- 172.21.15.42:0/3227750927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 msgr2=0x55bc33eeca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1650c00640 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33eeca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1650c00640 1 -- 172.21.15.42:0/3227750927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b212f0 msgr2=0x55bc33eecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1650c00640 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b212f0 0x55bc33eecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1650c00640 1 -- 172.21.15.42:0/3227750927 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc33de68b0 con 0x55bc33b294d0 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1651600640 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b212f0 0x55bc33eecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f164a000640 1 -- 172.21.15.42:0/721614149 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f16440e2db0 con 0x7f164c110320 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.085+0000 7f1652000640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f16340b2950 0x7f16340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.086+0000 7f1650c00640 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 0x55bc33eed4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55bc33ee9bd0 tx=0x7f163800b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cdcd36fd807cda2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.086+0000 7f1652000640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f16340b2950 0x7f16340b4e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16440aa070 tx=0x7f164400af30 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.086+0000 7f163e600640 1 -- 172.21.15.42:0/3227750927 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638004260 con 0x55bc33b294d0 2024-05-27T10:20:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.087+0000 7f163e600640 1 -- 172.21.15.42:0/3227750927 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1638004400 con 0x55bc33b294d0 2024-05-27T10:20:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.087+0000 7f163e600640 1 -- 172.21.15.42:0/3227750927 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638003850 con 0x55bc33b294d0 2024-05-27T10:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 -- 172.21.15.42:0/3227750927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 msgr2=0x55bc33eed4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 0x55bc33eed4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55bc33ee9bd0 tx=0x7f163800b4c0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 -- 172.21.15.42:0/3227750927 shutdown_connections 2024-05-27T10:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b294d0 0x55bc33eed4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b212f0 0x55bc33eecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 --2- 172.21.15.42:0/3227750927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33eeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 -- 172.21.15.42:0/3227750927 >> 172.21.15.42:0/3227750927 conn(0x55bc33b3e300 msgr2=0x55bc33ee9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 -- 172.21.15.42:0/3227750927 shutdown_connections 2024-05-27T10:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.088+0000 7f165af71740 1 -- 172.21.15.42:0/3227750927 wait complete. 2024-05-27T10:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.089+0000 7f165af71740 1 -- start start 2024-05-27T10:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.089+0000 7f165af71740 1 Processor -- start 2024-05-27T10:20:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b212f0 0x55bc33ef6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 0x55bc33ef7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b4a1b0 0x55bc33f02ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bc33b33790 con 0x55bc33b294d0 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc33df5bc0 con 0x55bc33b212f0 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bc33ef9b90 con 0x55bc33b4a1b0 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.090+0000 7f165af71740 10 client.0 _ll_register_callbacks cb 0x55bc33dd3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.091+0000 7f1651600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 0x55bc33ef7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.091+0000 7f1650c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b4a1b0 0x55bc33f02ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.091+0000 7f1651600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 0x55bc33ef7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47830/0 (socket says 172.21.15.42:47830) 2024-05-27T10:20:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.091+0000 7f1651600640 1 -- 172.21.15.42:0/297922313 learned_addr learned my addr 172.21.15.42:0/297922313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.091+0000 7f1650c00640 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b212f0 0x55bc33ef6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.092+0000 7f1651600640 1 -- 172.21.15.42:0/297922313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b4a1b0 msgr2=0x55bc33f02ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.092+0000 7f1651600640 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b4a1b0 0x55bc33f02ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.092+0000 7f1651600640 1 -- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b212f0 msgr2=0x55bc33ef6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.092+0000 7f1651600640 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b212f0 0x55bc33ef6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.092+0000 7f1651600640 1 -- 172.21.15.42:0/297922313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc33b62210 con 0x55bc33b294d0 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.093+0000 7f1651600640 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 0x55bc33ef7150 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f164002aa70 tx=0x7f16400045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d4b30167184aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.093+0000 7f1632a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.093+0000 7f1632a00640 1 -- 172.21.15.42:0/297922313 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640035040 con 0x55bc33b294d0 2024-05-27T10:20:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.093+0000 7f1632a00640 1 -- 172.21.15.42:0/297922313 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1640003b80 con 0x55bc33b294d0 2024-05-27T10:20:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.093+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1638004400 con 0x55bc33b294d0 2024-05-27T10:20:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.094+0000 7f1632a00640 1 -- 172.21.15.42:0/297922313 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1640003d80 con 0x55bc33b294d0 2024-05-27T10:20:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.094+0000 7f165af71740 10 client.15735 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.094+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc33c2e620 con 0x55bc33b294d0 2024-05-27T10:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.094+0000 7f1632a00640 1 -- 172.21.15.42:0/297922313 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1638004400 con 0x55bc33b294d0 2024-05-27T10:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.094+0000 7f1632a00640 1 -- 172.21.15.42:0/297922313 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f164004e430 con 0x55bc33b294d0 2024-05-27T10:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.095+0000 7f165af71740 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.095+0000 7f165af71740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.095+0000 7f165af71740 20 client.15735 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.095+0000 7f165af71740 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33b08d80 0x55bc33f11630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.095+0000 7f165af71740 4 client.15735 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.095+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc33e5fae0 con 0x55bc33b08d80 2024-05-27T10:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.098+0000 7f1650c00640 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33b08d80 0x55bc33f11630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.100+0000 7f1650c00640 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33b08d80 0x55bc33f11630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7be63d02a319af server_cookie=c26e37c49a69043a in_seq=0 out_seq=0 2024-05-27T10:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.100+0000 7f1632a00640 10 client.15735 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f1632a00640 1 -- 172.21.15.42:0/297922313 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bc33e5fae0 con 0x55bc33b08d80 2024-05-27T10:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f1632a00640 10 client.15735 handle_command_reply: tid=0 2024-05-27T10:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 client.15735 shutdown 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 msgr2=0x55bc33ef7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 0x55bc33ef7150 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f164002aa70 tx=0x7f16400045a0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 shutdown_connections 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33b08d80 0x55bc33f11630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b4a1b0 0x55bc33f02ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b294d0 0x55bc33ef7150 secure :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f164002aa70 tx=0x7f16400045a0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 --2- 172.21.15.42:0/297922313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b212f0 0x55bc33ef6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 >> 172.21.15.42:0/297922313 conn(0x55bc33b3e300 msgr2=0x55bc33eee780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.102+0000 7f1632a00640 0 client.15735 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.103+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 shutdown_connections 2024-05-27T10:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.103+0000 7f165af71740 1 -- 172.21.15.42:0/297922313 wait complete. 2024-05-27T10:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.103+0000 7f165af71740 20 client.15735 trim_cache size 0 max 16384 2024-05-27T10:20:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.135+0000 7f165af71740 1 Processor -- start 2024-05-27T10:20:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.135+0000 7f165af71740 1 -- start start 2024-05-27T10:20:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.136+0000 7f165af71740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 0x55bc33f02c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.136+0000 7f165af71740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b4a1b0 0x55bc33ef1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.136+0000 7f165af71740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33ade1c0 0x55bc33ef2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.136+0000 7f165af71740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1afde0 con 0x55bc33b4a1b0 2024-05-27T10:20:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.136+0000 7f165af71740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1ad6b0 con 0x55bc33ade1c0 2024-05-27T10:20:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.136+0000 7f165af71740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1ab580 con 0x55bc33b212f0 2024-05-27T10:20:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.137+0000 7f1650c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 0x55bc33f02c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.137+0000 7f1650c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 0x55bc33f02c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:33994/0 (socket says 172.21.15.42:33994) 2024-05-27T10:20:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.137+0000 7f1650c00640 1 -- 172.21.15.42:0/3800665113 learned_addr learned my addr 172.21.15.42:0/3800665113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.137+0000 7f1650c00640 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33ade1c0 0x55bc33ef2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.137+0000 7f1651600640 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b4a1b0 0x55bc33ef1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.138+0000 7f1650c00640 1 -- 172.21.15.42:0/3800665113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33ade1c0 msgr2=0x55bc33ef2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.138+0000 7f1650c00640 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33ade1c0 0x55bc33ef2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.138+0000 7f1650c00640 1 -- 172.21.15.42:0/3800665113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b4a1b0 msgr2=0x55bc33ef1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.138+0000 7f1650c00640 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b4a1b0 0x55bc33ef1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.138+0000 7f1650c00640 1 -- 172.21.15.42:0/3800665113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc33eed950 con 0x55bc33b212f0 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.138+0000 7f1651600640 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b4a1b0 0x55bc33ef1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:20:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.139+0000 7f1650c00640 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 0x55bc33f02c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f163800bdd0 tx=0x7f163800efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7461a0c59b561c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.139+0000 7f163dc00640 1 -- 172.21.15.42:0/3800665113 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638027040 con 0x55bc33b212f0 2024-05-27T10:20:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.139+0000 7f163dc00640 1 -- 172.21.15.42:0/3800665113 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1638034030 con 0x55bc33b212f0 2024-05-27T10:20:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.139+0000 7f163dc00640 1 -- 172.21.15.42:0/3800665113 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163800e980 con 0x55bc33b212f0 2024-05-27T10:20:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 -- 172.21.15.42:0/3800665113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 msgr2=0x55bc33f02c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 0x55bc33f02c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f163800bdd0 tx=0x7f163800efd0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 -- 172.21.15.42:0/3800665113 shutdown_connections 2024-05-27T10:20:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33ade1c0 0x55bc33ef2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33b4a1b0 0x55bc33ef1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 --2- 172.21.15.42:0/3800665113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33b212f0 0x55bc33f02c30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.140+0000 7f165af71740 1 -- 172.21.15.42:0/3800665113 >> 172.21.15.42:0/3800665113 conn(0x55bc33b3e300 msgr2=0x55bc33eed240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.141+0000 7f165af71740 1 -- 172.21.15.42:0/3800665113 shutdown_connections 2024-05-27T10:20:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.141+0000 7f165af71740 1 -- 172.21.15.42:0/3800665113 wait complete. 2024-05-27T10:20:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.142+0000 7f165af71740 1 -- start start 2024-05-27T10:20:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.142+0000 7f165af71740 1 Processor -- start 2024-05-27T10:20:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 0x55bc33efa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33efab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33f75c40 0x55bc33f78060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1abbd0 con 0x55bc33ade1c0 2024-05-27T10:20:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1acfe0 con 0x55bc33b4a1b0 2024-05-27T10:20:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1ac240 con 0x55bc33f75c40 2024-05-27T10:20:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f165af71740 10 client.0 _ll_register_callbacks cb 0x7f1640009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f1650c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 0x55bc33efa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.143+0000 7f1650c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 0x55bc33efa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47848/0 (socket says 172.21.15.42:47848) 2024-05-27T10:20:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.144+0000 7f1650c00640 1 -- 172.21.15.42:0/1922432305 learned_addr learned my addr 172.21.15.42:0/1922432305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.144+0000 7f1651600640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33efab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.144+0000 7f1650c00640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33f75c40 0x55bc33f78060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.145+0000 7f1650c00640 1 -- 172.21.15.42:0/1922432305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33f75c40 msgr2=0x55bc33f78060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.145+0000 7f1650c00640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33f75c40 0x55bc33f78060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.145+0000 7f1650c00640 1 -- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 msgr2=0x55bc33efab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.145+0000 7f1650c00640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33efab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.145+0000 7f1650c00640 1 -- 172.21.15.42:0/1922432305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc33f785f0 con 0x55bc33ade1c0 2024-05-27T10:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.145+0000 7f1651600640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33efab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.146+0000 7f1650c00640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 0x55bc33efa5c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f163800b500 tx=0x7f1638005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615e475163c0bc88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.146+0000 7f1632a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.146+0000 7f1632a00640 1 -- 172.21.15.42:0/1922432305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16380054c0 con 0x55bc33ade1c0 2024-05-27T10:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.146+0000 7f1632a00640 1 -- 172.21.15.42:0/1922432305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f16380052c0 con 0x55bc33ade1c0 2024-05-27T10:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.146+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1638034030 con 0x55bc33ade1c0 2024-05-27T10:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.147+0000 7f1632a00640 1 -- 172.21.15.42:0/1922432305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16380054c0 con 0x55bc33ade1c0 2024-05-27T10:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.147+0000 7f165af71740 10 client.15741 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.147+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc33f79880 con 0x55bc33ade1c0 2024-05-27T10:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.148+0000 7f1632a00640 1 -- 172.21.15.42:0/1922432305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1638034030 con 0x55bc33ade1c0 2024-05-27T10:20:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f1632a00640 1 -- 172.21.15.42:0/1922432305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1638038070 con 0x55bc33ade1c0 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f165af71740 10 client.15741 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f165af71740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f165af71740 20 client.15741 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f165af71740 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33f79bb0 0x55bc33f7bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f165af71740 4 client.15741 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.149+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc33e592c0 con 0x55bc33f79bb0 2024-05-27T10:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.150+0000 7f1651600640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33f79bb0 0x55bc33f7bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.151+0000 7f1651600640 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33f79bb0 0x55bc33f7bfd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f64a15189a397159 server_cookie=60dcd5c4013ac242 in_seq=0 out_seq=0 2024-05-27T10:20:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.151+0000 7f1632a00640 10 client.15741 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f1632a00640 1 -- 172.21.15.42:0/1922432305 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bc33e592c0 con 0x55bc33f79bb0 2024-05-27T10:20:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f1632a00640 10 client.15741 handle_command_reply: tid=0 2024-05-27T10:20:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 client.15741 shutdown 2024-05-27T10:20:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 msgr2=0x55bc33efa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 0x55bc33efa5c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f163800b500 tx=0x7f1638005f90 comp rx=0 tx=0).stop 2024-05-27T10:20:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 shutdown_connections 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55bc33f79bb0 0x55bc33f7bfd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bc33f75c40 0x55bc33f78060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc33b4a1b0 0x55bc33efab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 --2- 172.21.15.42:0/1922432305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc33ade1c0 0x55bc33efa5c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 >> 172.21.15.42:0/1922432305 conn(0x55bc33b3e300 msgr2=0x55bc33eed240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.161+0000 7f1632a00640 0 client.15741 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.162+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 shutdown_connections 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.162+0000 7f165af71740 1 -- 172.21.15.42:0/1922432305 wait complete. 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.162+0000 7f165af71740 20 client.15741 trim_cache size 0 max 16384 2024-05-27T10:20:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:20:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:20:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:20:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.162+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f16340b2950 msgr2=0x7f16340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f16340b2950 0x7f16340b4e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16440aa070 tx=0x7f164400af30 comp rx=0 tx=0).stop 2024-05-27T10:20:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 msgr2=0x7f164c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 0x7f164c19c650 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f164400cce0 tx=0x7f164400a620 comp rx=0 tx=0).stop 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 shutdown_connections 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f16340b2950 0x7f16340b4e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f164c110320 0x7f164c19c650 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f164c10d9c0 0x7f164c1a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 --2- 172.21.15.42:0/721614149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f164c075420 0x7f164c1a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 >> 172.21.15.42:0/721614149 conn(0x7f164c079100 msgr2=0x7f164c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 shutdown_connections 2024-05-27T10:20:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:20:52.163+0000 7f1652a00640 1 -- 172.21.15.42:0/721614149 wait complete. 2024-05-27T10:20:53.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:52 smithi143 ceph-mon[35545]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-27T10:20:53.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:52 smithi042 ceph-mon[31661]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-27T10:20:53.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:52 smithi104 ceph-mon[34812]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-27T10:20:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:54.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.675+0000 7feedf200700 1 -- 172.21.15.42:0/2058334763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 msgr2=0x7feee0110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.675+0000 7feedf200700 1 --2- 172.21.15.42:0/2058334763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0110bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feed800b0a0 tx=0x7feed800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 -- 172.21.15.42:0/2058334763 shutdown_connections 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 --2- 172.21.15.42:0/2058334763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0110bd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 --2- 172.21.15.42:0/2058334763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feee0105280 0x7feee0105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 --2- 172.21.15.42:0/2058334763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105bd0 0x7feee010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 -- 172.21.15.42:0/2058334763 >> 172.21.15.42:0/2058334763 conn(0x7feee0078ab0 msgr2=0x7feee0078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 -- 172.21.15.42:0/2058334763 shutdown_connections 2024-05-27T10:20:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.676+0000 7feedf200700 1 -- 172.21.15.42:0/2058334763 wait complete. 2024-05-27T10:20:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.678+0000 7feedf200700 1 Processor -- start 2024-05-27T10:20:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.678+0000 7feedf200700 1 -- start start 2024-05-27T10:20:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.679+0000 7feedf200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105280 0x7feee01060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.679+0000 7feedf200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feee0105bd0 0x7feee0106630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.679+0000 7feedf200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0106b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.679+0000 7feedf200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feee0114150 con 0x7feee010e6f0 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.679+0000 7feedf200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feee0113fd0 con 0x7feee0105bd0 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.679+0000 7feedf200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feee01142d0 con 0x7feee0105280 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.680+0000 7feedde00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105280 0x7feee01060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.680+0000 7feedd400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feee0105bd0 0x7feee0106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.680+0000 7feedde00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105280 0x7feee01060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:45272/0 (socket says 172.21.15.42:45272) 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.680+0000 7feedde00700 1 -- 172.21.15.42:0/1057882890 learned_addr learned my addr 172.21.15.42:0/1057882890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.680+0000 7feede800700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.682+0000 7feede800700 1 -- 172.21.15.42:0/1057882890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105280 msgr2=0x7feee01060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.682+0000 7feede800700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105280 0x7feee01060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.682+0000 7feede800700 1 -- 172.21.15.42:0/1057882890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feee0105bd0 msgr2=0x7feee0106630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.682+0000 7feede800700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feee0105bd0 0x7feee0106630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.682+0000 7feede800700 1 -- 172.21.15.42:0/1057882890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee01ad2c0 con 0x7feee010e6f0 2024-05-27T10:20:54.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.682+0000 7feede800700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0106b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feed800b070 tx=0x7feed80094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:54.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.683+0000 7feed6a00700 1 -- 172.21.15.42:0/1057882890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8021070 con 0x7feee010e6f0 2024-05-27T10:20:54.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.683+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee01ad4f0 con 0x7feee010e6f0 2024-05-27T10:20:54.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.683+0000 7feed6a00700 1 -- 172.21.15.42:0/1057882890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed801d450 con 0x7feee010e6f0 2024-05-27T10:20:54.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.683+0000 7feed6a00700 1 -- 172.21.15.42:0/1057882890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80052a0 con 0x7feee010e6f0 2024-05-27T10:20:54.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.683+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee01ada20 con 0x7feee010e6f0 2024-05-27T10:20:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.690+0000 7feed6a00700 1 -- 172.21.15.42:0/1057882890 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7feed8024070 con 0x7feee010e6f0 2024-05-27T10:20:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.690+0000 7feef39f8b80 1 Processor -- start 2024-05-27T10:20:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.691+0000 7feef39f8b80 1 -- start start 2024-05-27T10:20:54.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.691+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c28750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.691+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c28c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.692+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.692+0000 7feef39f8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564689b55e40 con 0x564689c322c0 2024-05-27T10:20:54.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.692+0000 7feef39f8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564689b459e0 con 0x564689b949f0 2024-05-27T10:20:54.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.692+0000 7feef39f8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564689b49a20 con 0x564689aa8e40 2024-05-27T10:20:54.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.693+0000 7feedd400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c28750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.693+0000 7feedd400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c28750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:47416/0 (socket says 172.21.15.42:47416) 2024-05-27T10:20:54.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.693+0000 7feedd400700 1 -- 172.21.15.42:0/1743628721 learned_addr learned my addr 172.21.15.42:0/1743628721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:54.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.693+0000 7feedd400700 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.693+0000 7feedde00700 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c28c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.693+0000 7feed6a00700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feec40b2a70 0x7feec40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.694+0000 7feed6a00700 1 -- 172.21.15.42:0/1057882890 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7feed800f020 con 0x7feee010e6f0 2024-05-27T10:20:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.694+0000 7feede800700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feec40b2a70 0x7feec40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.694+0000 7feedde00700 1 -- 172.21.15.42:0/1743628721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 msgr2=0x564689c28750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.694+0000 7feedde00700 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c28750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.694+0000 7feedde00700 1 -- 172.21.15.42:0/1743628721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 msgr2=0x564689c346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.694+0000 7feedde00700 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.695+0000 7feedde00700 1 -- 172.21.15.42:0/1743628721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564689b0e420 con 0x564689aa8e40 2024-05-27T10:20:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.695+0000 7feedd400700 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:20:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.695+0000 7feede800700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feec40b2a70 0x7feec40b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feed80aa250 tx=0x7feed8009040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.696+0000 7feedde00700 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c28c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feed000bdb0 tx=0x7feed00106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6653cb4075776e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.696+0000 7feecb400700 1 -- 172.21.15.42:0/1743628721 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0010e10 con 0x564689aa8e40 2024-05-27T10:20:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.696+0000 7feecb400700 1 -- 172.21.15.42:0/1743628721 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed00044c0 con 0x564689aa8e40 2024-05-27T10:20:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.696+0000 7feecb400700 1 -- 172.21.15.42:0/1743628721 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0011440 con 0x564689aa8e40 2024-05-27T10:20:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 -- 172.21.15.42:0/1743628721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 msgr2=0x564689c28c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c28c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feed000bdb0 tx=0x7feed00106b0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 -- 172.21.15.42:0/1743628721 shutdown_connections 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c28750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 --2- 172.21.15.42:0/1743628721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c28c90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 -- 172.21.15.42:0/1743628721 >> 172.21.15.42:0/1743628721 conn(0x564689b42390 msgr2=0x564689c23710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.698+0000 7feef39f8b80 1 -- 172.21.15.42:0/1743628721 shutdown_connections 2024-05-27T10:20:54.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.699+0000 7feef39f8b80 1 -- 172.21.15.42:0/1743628721 wait complete. 2024-05-27T10:20:54.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.699+0000 7feef39f8b80 1 -- start start 2024-05-27T10:20:54.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.700+0000 7feef39f8b80 1 Processor -- start 2024-05-27T10:20:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.700+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.701+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 0x564689c2f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.701+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689c2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.701+0000 7feef39f8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564689a4bfa0 con 0x564689b949f0 2024-05-27T10:20:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.701+0000 7feef39f8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564689ae6760 con 0x564689c322c0 2024-05-27T10:20:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.701+0000 7feef39f8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564689c34e00 con 0x564689aa8e40 2024-05-27T10:20:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.701+0000 7feef39f8b80 10 client.0 _ll_register_callbacks cb 0x564689aefc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.702+0000 7feedde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 0x564689c2f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.702+0000 7feedd400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.702+0000 7feedde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 0x564689c2f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54388/0 (socket says 172.21.15.42:54388) 2024-05-27T10:20:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.702+0000 7feedde00700 1 -- 172.21.15.42:0/2642438785 learned_addr learned my addr 172.21.15.42:0/2642438785 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.702+0000 7feedd400700 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689c2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.702+0000 7feedde00700 1 -- 172.21.15.42:0/2642438785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 msgr2=0x564689c2f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.703+0000 7feedde00700 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.703+0000 7feedde00700 1 -- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 msgr2=0x564689c2fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.703+0000 7feedde00700 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689c2fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.703+0000 7feedde00700 1 -- 172.21.15.42:0/2642438785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564689a958d0 con 0x564689b949f0 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.703+0000 7feedde00700 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 0x564689c2f780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7feed00038b0 tx=0x7feed0010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e119bf123e3b2a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.703+0000 7feebfe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:54.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.704+0000 7feebfe00700 1 -- 172.21.15.42:0/2642438785 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0003d70 con 0x564689b949f0 2024-05-27T10:20:54.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.704+0000 7feebfe00700 1 -- 172.21.15.42:0/2642438785 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7feed0003f10 con 0x564689b949f0 2024-05-27T10:20:54.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.704+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed00044c0 con 0x564689b949f0 2024-05-27T10:20:54.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.705+0000 7feebfe00700 1 -- 172.21.15.42:0/2642438785 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feed0004110 con 0x564689b949f0 2024-05-27T10:20:54.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.705+0000 7feef39f8b80 10 client.15753 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:54.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.705+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564689b16060 con 0x564689b949f0 2024-05-27T10:20:54.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feebfe00700 1 -- 172.21.15.42:0/2642438785 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed00044c0 con 0x564689b949f0 2024-05-27T10:20:54.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feebfe00700 1 -- 172.21.15.42:0/2642438785 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7feed002d450 con 0x564689b949f0 2024-05-27T10:20:54.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feef39f8b80 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:54.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feef39f8b80 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:54.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feef39f8b80 20 client.15753 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:54.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feef39f8b80 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689c41bf0 0x564689c44010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feef39f8b80 4 client.15753 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:54.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.706+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564689b57d90 con 0x564689c41bf0 2024-05-27T10:20:54.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.707+0000 7feedd400700 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689c41bf0 0x564689c44010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.708+0000 7feedd400700 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689c41bf0 0x564689c44010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ae8e158f2ab890 server_cookie=84e8b219fe691d64 in_seq=0 out_seq=0 2024-05-27T10:20:54.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.708+0000 7feebfe00700 10 client.15753 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:54.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.710+0000 7feebfe00700 1 -- 172.21.15.42:0/2642438785 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564689b57d90 con 0x564689c41bf0 2024-05-27T10:20:54.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.710+0000 7feebfe00700 10 client.15753 handle_command_reply: tid=0 2024-05-27T10:20:54.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.710+0000 7feef39f8b80 1 client.15753 shutdown 2024-05-27T10:20:54.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 msgr2=0x564689c2f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 0x564689c2f780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7feed00038b0 tx=0x7feed0010c10 comp rx=0 tx=0).stop 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 shutdown_connections 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689c41bf0 0x564689c44010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689c2fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 --2- 172.21.15.42:0/2642438785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689b949f0 0x564689c2f780 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 >> 172.21.15.42:0/2642438785 conn(0x564689b42390 msgr2=0x564689c28750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feebfe00700 0 client.15753 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 shutdown_connections 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.711+0000 7feef39f8b80 1 -- 172.21.15.42:0/2642438785 wait complete. 2024-05-27T10:20:54.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.712+0000 7feef39f8b80 20 client.15753 trim_cache size 0 max 16384 2024-05-27T10:20:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.754+0000 7feef39f8b80 1 Processor -- start 2024-05-27T10:20:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 -- start start 2024-05-27T10:20:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c49610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c49b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feee01b79e0 con 0x564689c322c0 2024-05-27T10:20:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feee01b5270 con 0x564689b949f0 2024-05-27T10:20:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.755+0000 7feef39f8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feee01b30e0 con 0x564689aa8e40 2024-05-27T10:20:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c49b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c49b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54396/0 (socket says 172.21.15.42:54396) 2024-05-27T10:20:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 -- 172.21.15.42:0/543487491 learned_addr learned my addr 172.21.15.42:0/543487491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedde00700 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c49610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 -- 172.21.15.42:0/543487491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 msgr2=0x564689c2e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 -- 172.21.15.42:0/543487491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 msgr2=0x564689c49610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c49610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.756+0000 7feedd400700 1 -- 172.21.15.42:0/543487491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564689c30030 con 0x564689c322c0 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.757+0000 7feedd400700 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c49b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x564689c2a210 tx=0x7feecc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2dd1330b4ce888 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.757+0000 7feecaa00700 1 -- 172.21.15.42:0/543487491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc004830 con 0x564689c322c0 2024-05-27T10:20:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.757+0000 7feecaa00700 1 -- 172.21.15.42:0/543487491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feecc005020 con 0x564689c322c0 2024-05-27T10:20:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.757+0000 7feecaa00700 1 -- 172.21.15.42:0/543487491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0225d0 con 0x564689c322c0 2024-05-27T10:20:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 -- 172.21.15.42:0/543487491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 msgr2=0x564689c49b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c49b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x564689c2a210 tx=0x7feecc0047c0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 -- 172.21.15.42:0/543487491 shutdown_connections 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689c322c0 0x564689c49b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689aa8e40 0x564689c2e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 --2- 172.21.15.42:0/543487491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689b949f0 0x564689c49610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 -- 172.21.15.42:0/543487491 >> 172.21.15.42:0/543487491 conn(0x564689b42390 msgr2=0x564689c2db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 -- 172.21.15.42:0/543487491 shutdown_connections 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 -- 172.21.15.42:0/543487491 wait complete. 2024-05-27T10:20:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.759+0000 7feef39f8b80 1 -- start start 2024-05-27T10:20:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.760+0000 7feef39f8b80 1 Processor -- start 2024-05-27T10:20:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689aa8e40 0x564689c36460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689b949f0 0x564689c369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feedd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689aa8e40 0x564689c36460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689cab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feee01b3730 con 0x564689aa8e40 2024-05-27T10:20:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feee01b4ba0 con 0x564689c322c0 2024-05-27T10:20:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feee01b3e00 con 0x564689b949f0 2024-05-27T10:20:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feef39f8b80 10 client.0 _ll_register_callbacks cb 0x564689aefc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feedd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689aa8e40 0x564689c36460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54406/0 (socket says 172.21.15.42:54406) 2024-05-27T10:20:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.761+0000 7feedd400700 1 -- 172.21.15.42:0/407961775 learned_addr learned my addr 172.21.15.42:0/407961775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.762+0000 7feedde00700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689b949f0 0x564689c369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.762+0000 7feedd400700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689cab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.762+0000 7feedde00700 1 -- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 msgr2=0x564689cab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feedde00700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689cab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feedde00700 1 -- 172.21.15.42:0/407961775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689aa8e40 msgr2=0x564689c36460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feedde00700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689aa8e40 0x564689c36460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feedde00700 1 -- 172.21.15.42:0/407961775 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564689caba20 con 0x564689b949f0 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feedd400700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689cab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feedde00700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689b949f0 0x564689c369a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feed000bdb0 tx=0x7feed002ddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adfd1d76d18b88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.763+0000 7feebfe00700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:20:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.764+0000 7feebfe00700 1 -- 172.21.15.42:0/407961775 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0003cf0 con 0x564689b949f0 2024-05-27T10:20:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.764+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc005020 con 0x564689b949f0 2024-05-27T10:20:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.764+0000 7feebfe00700 1 -- 172.21.15.42:0/407961775 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7feed0003e90 con 0x564689b949f0 2024-05-27T10:20:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.764+0000 7feebfe00700 1 -- 172.21.15.42:0/407961775 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feed00044c0 con 0x564689b949f0 2024-05-27T10:20:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.764+0000 7feef39f8b80 10 client.25429 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.764+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564689cacb50 con 0x564689b949f0 2024-05-27T10:20:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.765+0000 7feebfe00700 1 -- 172.21.15.42:0/407961775 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feecc005020 con 0x564689b949f0 2024-05-27T10:20:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feebfe00700 1 -- 172.21.15.42:0/407961775 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7feed0035070 con 0x564689b949f0 2024-05-27T10:20:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feef39f8b80 10 client.25429 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feef39f8b80 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:20:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feef39f8b80 20 client.25429 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feef39f8b80 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689cad1f0 0x564689caf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feef39f8b80 4 client.25429 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:20:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.766+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564689b51570 con 0x564689cad1f0 2024-05-27T10:20:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.767+0000 7feedd400700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689cad1f0 0x564689caf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.768+0000 7feebfe00700 10 client.25429 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.768+0000 7feedd400700 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689cad1f0 0x564689caf610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8663afefde4aec server_cookie=886e8abc80f4e77a in_seq=0 out_seq=0 2024-05-27T10:20:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.768+0000 7feebfe00700 1 -- 172.21.15.42:0/407961775 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564689b51570 con 0x564689cad1f0 2024-05-27T10:20:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.768+0000 7feebfe00700 10 client.25429 handle_command_reply: tid=0 2024-05-27T10:20:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.768+0000 7feef39f8b80 1 client.25429 shutdown 2024-05-27T10:20:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689b949f0 msgr2=0x564689c369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689b949f0 0x564689c369a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feed000bdb0 tx=0x7feed002ddf0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 shutdown_connections 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564689cad1f0 0x564689caf610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564689c322c0 0x564689cab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564689aa8e40 0x564689c36460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 --2- 172.21.15.42:0/407961775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564689b949f0 0x564689c369a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feebfe00700 0 client.25429 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 >> 172.21.15.42:0/407961775 conn(0x564689b42390 msgr2=0x564689c236c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 shutdown_connections 2024-05-27T10:20:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 1 -- 172.21.15.42:0/407961775 wait complete. 2024-05-27T10:20:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.769+0000 7feef39f8b80 20 client.25429 trim_cache size 0 max 16384 2024-05-27T10:20:54.771 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:20:54.771 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-05-27T10:20:54.771 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-05-27T10:20:54.771 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.770+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feec40b2a70 msgr2=0x7feec40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.770+0000 7feedf200700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feec40b2a70 0x7feec40b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feed80aa250 tx=0x7feed8009040 comp rx=0 tx=0).stop 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.770+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 msgr2=0x7feee0106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.770+0000 7feedf200700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0106b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feed800b070 tx=0x7feed80094f0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 shutdown_connections 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7feec40b2a70 0x7feec40b4f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee010e6f0 0x7feee0106b70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feee0105280 0x7feee01060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 --2- 172.21.15.42:0/1057882890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feee0105bd0 0x7feee0106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 >> 172.21.15.42:0/1057882890 conn(0x7feee0078ab0 msgr2=0x7feee0102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 shutdown_connections 2024-05-27T10:20:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:54.771+0000 7feedf200700 1 -- 172.21.15.42:0/1057882890 wait complete. 2024-05-27T10:20:55.035 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:54 smithi042 ceph-mon[31661]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-27T10:20:55.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:54 smithi143 ceph-mon[35545]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-27T10:20:55.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:54 smithi104 ceph-mon[34812]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-27T10:20:56.106 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T10:20:56.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:20:56.893 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:20:57.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:56 smithi042 ceph-mon[31661]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:57.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:56 smithi143 ceph-mon[35545]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:57.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:56 smithi104 ceph-mon[34812]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:20:57.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.805+0000 7fe69a400700 1 -- 172.21.15.42:0/3727752423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694108f00 msgr2=0x7fe694109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.805+0000 7fe69a400700 1 --2- 172.21.15.42:0/3727752423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694108f00 0x7fe694109310 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe6800099c0 tx=0x7fe680009cd0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 -- 172.21.15.42:0/3727752423 shutdown_connections 2024-05-27T10:20:57.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 --2- 172.21.15.42:0/3727752423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe694109850 0x7fe6941111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 --2- 172.21.15.42:0/3727752423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 0x7fe694104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 --2- 172.21.15.42:0/3727752423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694108f00 0x7fe694109310 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 -- 172.21.15.42:0/3727752423 >> 172.21.15.42:0/3727752423 conn(0x7fe694100090 msgr2=0x7fe6941024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:57.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 -- 172.21.15.42:0/3727752423 shutdown_connections 2024-05-27T10:20:57.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.806+0000 7fe69a400700 1 -- 172.21.15.42:0/3727752423 wait complete. 2024-05-27T10:20:57.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.808+0000 7fe69a400700 1 Processor -- start 2024-05-27T10:20:57.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.808+0000 7fe69a400700 1 -- start start 2024-05-27T10:20:57.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe69a400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 0x7fe694075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe69a400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe694108f00 0x7fe694075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe69a400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 0x7fe694076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe69a400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe694114360 con 0x7fe694109850 2024-05-27T10:20:57.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe69a400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941141e0 con 0x7fe694104620 2024-05-27T10:20:57.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe69a400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941144e0 con 0x7fe694108f00 2024-05-27T10:20:57.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.809+0000 7fe693e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 0x7fe694076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 0x7fe694076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54428/0 (socket says 172.21.15.42:54428) 2024-05-27T10:20:57.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693e00700 1 -- 172.21.15.42:0/666044126 learned_addr learned my addr 172.21.15.42:0/666044126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:57.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 0x7fe694075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe692a00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe694108f00 0x7fe694075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693e00700 1 -- 172.21.15.42:0/666044126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe694108f00 msgr2=0x7fe694075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693e00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe694108f00 0x7fe694075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693e00700 1 -- 172.21.15.42:0/666044126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 msgr2=0x7fe694075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.810+0000 7fe693e00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 0x7fe694075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.811+0000 7fe693e00700 1 -- 172.21.15.42:0/666044126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6941a91b0 con 0x7fe694109850 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.811+0000 7fe693400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 0x7fe694075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.811+0000 7fe693e00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 0x7fe694076530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe68c007ee0 tx=0x7fe68c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.812+0000 7fe68be00700 1 -- 172.21.15.42:0/666044126 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004bb0 con 0x7fe694109850 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.812+0000 7fe68be00700 1 -- 172.21.15.42:0/666044126 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe68c004d50 con 0x7fe694109850 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.812+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941a94a0 con 0x7fe694109850 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.812+0000 7fe68be00700 1 -- 172.21.15.42:0/666044126 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0058b0 con 0x7fe694109850 2024-05-27T10:20:57.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.812+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a99b0 con 0x7fe694109850 2024-05-27T10:20:57.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.818+0000 7fe68be00700 1 -- 172.21.15.42:0/666044126 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe68c01a020 con 0x7fe694109850 2024-05-27T10:20:57.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.819+0000 7fe6a8f60b80 1 Processor -- start 2024-05-27T10:20:57.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.819+0000 7fe6a8f60b80 1 -- start start 2024-05-27T10:20:57.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.819+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e710db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.819+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e7112f0 0x55e81e70af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.819+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.820+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e81e638f40 con 0x55e81e58bf40 2024-05-27T10:20:57.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.820+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e81e628ae0 con 0x55e81e70b440 2024-05-27T10:20:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.820+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e81e62cb20 con 0x55e81e7112f0 2024-05-27T10:20:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.820+0000 7fe692a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e710db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.820+0000 7fe692a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e710db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54438/0 (socket says 172.21.15.42:54438) 2024-05-27T10:20:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.820+0000 7fe692a00700 1 -- 172.21.15.42:0/1373238942 learned_addr learned my addr 172.21.15.42:0/1373238942 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe68be00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe6780b2b40 0x7fe6780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe693400700 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e7112f0 0x55e81e70af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe692a00700 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe68be00700 1 -- 172.21.15.42:0/666044126 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe68c0e1bd0 con 0x7fe694109850 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe692a00700 1 -- 172.21.15.42:0/1373238942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e7112f0 msgr2=0x55e81e70af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe692a00700 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e7112f0 0x55e81e70af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe692a00700 1 -- 172.21.15.42:0/1373238942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 msgr2=0x55e81e70b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe692a00700 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.821+0000 7fe692a00700 1 -- 172.21.15.42:0/1373238942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81e5f1520 con 0x55e81e58bf40 2024-05-27T10:20:57.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.822+0000 7fe692a00700 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e710db0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe6800099c0 tx=0x7fe680009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a94bb1d0abfb35d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:57.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.822+0000 7fe688c00700 1 -- 172.21.15.42:0/1373238942 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680005350 con 0x55e81e58bf40 2024-05-27T10:20:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.822+0000 7fe688c00700 1 -- 172.21.15.42:0/1373238942 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6800054f0 con 0x55e81e58bf40 2024-05-27T10:20:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.822+0000 7fe688c00700 1 -- 172.21.15.42:0/1373238942 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800039e0 con 0x55e81e58bf40 2024-05-27T10:20:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.823+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/1373238942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 msgr2=0x55e81e710db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.823+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e710db0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe6800099c0 tx=0x7fe680009f10 comp rx=0 tx=0).stop 2024-05-27T10:20:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe693e00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe6780b2b40 0x7fe6780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/1373238942 shutdown_connections 2024-05-27T10:20:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e710db0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/1373238942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e7112f0 0x55e81e70af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/1373238942 >> 172.21.15.42:0/1373238942 conn(0x55e81e5ec0c0 msgr2=0x55e81e706100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.824+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/1373238942 shutdown_connections 2024-05-27T10:20:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.825+0000 7fe693e00700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe6780b2b40 0x7fe6780b5050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe68c007fd0 tx=0x7fe68c00b040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.825+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/1373238942 wait complete. 2024-05-27T10:20:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.825+0000 7fe6a8f60b80 1 -- start start 2024-05-27T10:20:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.825+0000 7fe6a8f60b80 1 Processor -- start 2024-05-27T10:20:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e714af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e70b440 0x55e81e715030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e7112f0 0x55e81e715570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e81e52f0a0 con 0x55e81e58bf40 2024-05-27T10:20:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e81e5c9860 con 0x55e81e7112f0 2024-05-27T10:20:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e81e717d60 con 0x55e81e70b440 2024-05-27T10:20:57.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.826+0000 7fe6a8f60b80 10 client.0 _ll_register_callbacks cb 0x55e81e5d2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:57.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e714af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e714af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54442/0 (socket says 172.21.15.42:54442) 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 -- 172.21.15.42:0/2773491741 learned_addr learned my addr 172.21.15.42:0/2773491741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e7112f0 0x55e81e715570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe693400700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e70b440 0x55e81e715030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 -- 172.21.15.42:0/2773491741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e70b440 msgr2=0x55e81e715030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e70b440 0x55e81e715030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 -- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e7112f0 msgr2=0x55e81e715570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.827+0000 7fe692a00700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e7112f0 0x55e81e715570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.828+0000 7fe692a00700 1 -- 172.21.15.42:0/2773491741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81e5789d0 con 0x55e81e58bf40 2024-05-27T10:20:57.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.828+0000 7fe692a00700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e714af0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe680009990 tx=0x7fe68000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1f1d07c2d9d9c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:57.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.828+0000 7fe67d600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.828+0000 7fe67d600700 1 -- 172.21.15.42:0/2773491741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680003890 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.828+0000 7fe67d600700 1 -- 172.21.15.42:0/2773491741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe680003a30 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.828+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6800054f0 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.829+0000 7fe67d600700 1 -- 172.21.15.42:0/2773491741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe680038030 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.829+0000 7fe6a8f60b80 10 client.15771 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.829+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e81e5f9160 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.829+0000 7fe67d600700 1 -- 172.21.15.42:0/2773491741 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6800054f0 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe67d600700 1 -- 172.21.15.42:0/2773491741 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe6800364d0 con 0x55e81e58bf40 2024-05-27T10:20:57.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe6a8f60b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe6a8f60b80 10 client.15771 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:20:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe6a8f60b80 20 client.15771 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e724b10 0x55e81e726f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe6a8f60b80 4 client.15771 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:20:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.830+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e81e63ae90 con 0x55e81e724b10 2024-05-27T10:20:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.831+0000 7fe693400700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e724b10 0x55e81e726f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.832+0000 7fe693400700 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e724b10 0x55e81e726f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1263a5824eb9e85 server_cookie=1434b16cce07af3 in_seq=0 out_seq=0 2024-05-27T10:20:57.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.832+0000 7fe67d600700 10 client.15771 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:57.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.834+0000 7fe67d600700 1 -- 172.21.15.42:0/2773491741 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e81e63ae90 con 0x55e81e724b10 2024-05-27T10:20:57.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.834+0000 7fe67d600700 10 client.15771 handle_command_reply: tid=0 2024-05-27T10:20:57.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.834+0000 7fe6a8f60b80 1 client.15771 shutdown 2024-05-27T10:20:57.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 msgr2=0x55e81e714af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e714af0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe680009990 tx=0x7fe68000bfd0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 shutdown_connections 2024-05-27T10:20:57.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e724b10 0x55e81e726f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e7112f0 0x55e81e715570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e58bf40 0x55e81e714af0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/2773491741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e70b440 0x55e81e715030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.835+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 >> 172.21.15.42:0/2773491741 conn(0x55e81e5ec0c0 msgr2=0x55e81e709d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:57.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.836+0000 7fe67d600700 0 client.15771 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:57.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.836+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 shutdown_connections 2024-05-27T10:20:57.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.836+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/2773491741 wait complete. 2024-05-27T10:20:57.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.836+0000 7fe6a8f60b80 20 client.15771 trim_cache size 0 max 16384 2024-05-27T10:20:57.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 Processor -- start 2024-05-27T10:20:57.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 -- start start 2024-05-27T10:20:57.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e716f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e70ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941b3700 con 0x55e81e7112f0 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941b0f90 con 0x55e81e70b440 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.858+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941aede0 con 0x55e81e720fc0 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.859+0000 7fe693400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e716f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.859+0000 7fe693400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e716f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54444/0 (socket says 172.21.15.42:54444) 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.859+0000 7fe693400700 1 -- 172.21.15.42:0/3102750311 learned_addr learned my addr 172.21.15.42:0/3102750311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.859+0000 7fe692a00700 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.859+0000 7fe692a00700 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e70ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe693400700 1 -- 172.21.15.42:0/3102750311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 msgr2=0x55e81e70ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe693400700 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e70ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe693400700 1 -- 172.21.15.42:0/3102750311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 msgr2=0x55e81e70a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe693400700 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe693400700 1 -- 172.21.15.42:0/3102750311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81e7006f0 con 0x55e81e7112f0 2024-05-27T10:20:57.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe693400700 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e716f90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe684005170 tx=0x7fe68400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0ea36aa3ddcd0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:57.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.860+0000 7fe67fe00700 1 -- 172.21.15.42:0/3102750311 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684013310 con 0x55e81e7112f0 2024-05-27T10:20:57.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.861+0000 7fe67fe00700 1 -- 172.21.15.42:0/3102750311 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe684004df0 con 0x55e81e7112f0 2024-05-27T10:20:57.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.861+0000 7fe67fe00700 1 -- 172.21.15.42:0/3102750311 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684003a20 con 0x55e81e7112f0 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/3102750311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 msgr2=0x55e81e716f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e716f90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe684005170 tx=0x7fe68400b410 comp rx=0 tx=0).stop 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/3102750311 shutdown_connections 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e70ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e70a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/3102750311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e716f90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/3102750311 >> 172.21.15.42:0/3102750311 conn(0x55e81e5ec0c0 msgr2=0x55e81e70e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/3102750311 shutdown_connections 2024-05-27T10:20:57.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/3102750311 wait complete. 2024-05-27T10:20:57.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.862+0000 7fe6a8f60b80 1 -- start start 2024-05-27T10:20:57.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.863+0000 7fe6a8f60b80 1 Processor -- start 2024-05-27T10:20:57.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.863+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e72a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e728500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe6a8f60b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e728a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941af450 con 0x55e81e7112f0 2024-05-27T10:20:57.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941b08c0 con 0x55e81e70b440 2024-05-27T10:20:57.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe6a8f60b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941afb20 con 0x55e81e720fc0 2024-05-27T10:20:57.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe6a8f60b80 10 client.0 _ll_register_callbacks cb 0x55e81e5d2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:20:57.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe692a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e72a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe693400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e728500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe692a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e72a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:47476/0 (socket says 172.21.15.42:47476) 2024-05-27T10:20:57.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.864+0000 7fe692a00700 1 -- 172.21.15.42:0/739184049 learned_addr learned my addr 172.21.15.42:0/739184049 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:20:57.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e728a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 -- 172.21.15.42:0/739184049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 msgr2=0x55e81e728a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e728a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 -- 172.21.15.42:0/739184049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 msgr2=0x55e81e728500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e728500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 -- 172.21.15.42:0/739184049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81e7158e0 con 0x55e81e70b440 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe692a00700 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e72a170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe68000a010 tx=0x7fe68000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d3f0b95b764c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.865+0000 7fe67d600700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.866+0000 7fe67d600700 1 -- 172.21.15.42:0/739184049 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680042040 con 0x55e81e70b440 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.866+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684004df0 con 0x55e81e70b440 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.866+0000 7fe67d600700 1 -- 172.21.15.42:0/739184049 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe680039070 con 0x55e81e70b440 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.866+0000 7fe67d600700 1 -- 172.21.15.42:0/739184049 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe680036cf0 con 0x55e81e70b440 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.866+0000 7fe6a8f60b80 10 client.35411 fetch_fsmap learned FSMap version 26 2024-05-27T10:20:57.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.866+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e81e798890 con 0x55e81e70b440 2024-05-27T10:20:57.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe67d600700 1 -- 172.21.15.42:0/739184049 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe684004df0 con 0x55e81e70b440 2024-05-27T10:20:57.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe67d600700 1 -- 172.21.15.42:0/739184049 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe68000f690 con 0x55e81e70b440 2024-05-27T10:20:57.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe6a8f60b80 10 client.35411 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:20:57.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe6a8f60b80 10 client.35411 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:20:57.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe6a8f60b80 20 client.35411 populate_metadata read hostname 'smithi042' 2024-05-27T10:20:57.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e798e90 0x55e81e79b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:20:57.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe6a8f60b80 4 client.35411 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "91f1c12c", "release": true, "await": true}] 2024-05-27T10:20:57.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.867+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "91f1c12c", "release": true, "await": true}) -- 0x55e81e634670 con 0x55e81e798e90 2024-05-27T10:20:57.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.868+0000 7fe693400700 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e798e90 0x55e81e79b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:20:57.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.869+0000 7fe693400700 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e798e90 0x55e81e79b2b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c6c16e3670d300 server_cookie=96e362b5793fcf48 in_seq=0 out_seq=0 2024-05-27T10:20:57.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.869+0000 7fe67d600700 10 client.35411 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:57.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe67d600700 1 -- 172.21.15.42:0/739184049 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e81e634670 con 0x55e81e798e90 2024-05-27T10:20:57.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe67d600700 10 client.35411 handle_command_reply: tid=0 2024-05-27T10:20:57.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 client.35411 shutdown 2024-05-27T10:20:57.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 msgr2=0x55e81e72a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e72a170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe68000a010 tx=0x7fe68000bb20 comp rx=0 tx=0).stop 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 shutdown_connections 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e81e798e90 0x55e81e79b2b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e81e720fc0 0x55e81e728a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e81e70b440 0x55e81e72a170 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe68000a010 tx=0x7fe68000bb20 comp rx=0 tx=0).stop 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 --2- 172.21.15.42:0/739184049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e81e7112f0 0x55e81e728500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 >> 172.21.15.42:0/739184049 conn(0x55e81e5ec0c0 msgr2=0x55e81e70e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.875+0000 7fe67d600700 0 client.35411 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.876+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 shutdown_connections 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.876+0000 7fe6a8f60b80 1 -- 172.21.15.42:0/739184049 wait complete. 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.876+0000 7fe6a8f60b80 20 client.35411 trim_cache size 0 max 16384 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 26, 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:20:57.878 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 4, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "91f1c12c": { 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 4, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 149.4, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:20:57.879 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:20:57.880 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe6780b2b40 msgr2=0x7fe6780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe6780b2b40 0x7fe6780b5050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe68c007fd0 tx=0x7fe68c00b040 comp rx=0 tx=0).stop 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 msgr2=0x7fe694076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 0x7fe694076530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe68c007ee0 tx=0x7fe68c00d3b0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 shutdown_connections 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe6780b2b40 0x7fe6780b5050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe694109850 0x7fe694076530 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe694104620 0x7fe694075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 --2- 172.21.15.42:0/666044126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe694108f00 0x7fe694075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.878+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 >> 172.21.15.42:0/666044126 conn(0x7fe694100090 msgr2=0x7fe6941086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.879+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 shutdown_connections 2024-05-27T10:20:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:20:57.879+0000 7fe69a400700 1 -- 172.21.15.42:0/666044126 wait complete. 2024-05-27T10:20:58.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:20:58 smithi042 ceph-mon[31661]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:59.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:20:58 smithi143 ceph-mon[35545]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:59.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:20:58 smithi104 ceph-mon[34812]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:20:59.823 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '91f1c12c', total seconds elapsed: 42.0 2024-05-27T10:21:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 -- 172.21.15.42:0/2962852070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec8107de0 msgr2=0x7efec81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 --2- 172.21.15.42:0/2962852070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec8107de0 0x7efec81081f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efeb00099c0 tx=0x7efeb0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:21:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 -- 172.21.15.42:0/2962852070 shutdown_connections 2024-05-27T10:21:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 --2- 172.21.15.42:0/2962852070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8108730 0x7efec8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 --2- 172.21.15.42:0/2962852070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec81034b0 0x7efec81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 --2- 172.21.15.42:0/2962852070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec8107de0 0x7efec81081f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 -- 172.21.15.42:0/2962852070 >> 172.21.15.42:0/2962852070 conn(0x7efec80fedb0 msgr2=0x7efec81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 -- 172.21.15.42:0/2962852070 shutdown_connections 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.607+0000 7efece400700 1 -- 172.21.15.42:0/2962852070 wait complete. 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.609+0000 7efece400700 1 Processor -- start 2024-05-27T10:21:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 -- start start 2024-05-27T10:21:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec81034b0 0x7efec8075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 0x7efec806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8108730 0x7efec806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:00.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efec81121c0 con 0x7efec8108730 2024-05-27T10:21:00.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efec8112040 con 0x7efec8107de0 2024-05-27T10:21:00.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.610+0000 7efece400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efec8112340 con 0x7efec81034b0 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.611+0000 7efec7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8108730 0x7efec806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.611+0000 7efec7400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec81034b0 0x7efec8075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.611+0000 7efebea00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 0x7efec806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.611+0000 7efebea00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 0x7efec806e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:47496/0 (socket says 172.21.15.42:47496) 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.611+0000 7efebea00700 1 -- 172.21.15.42:0/1507007776 learned_addr learned my addr 172.21.15.42:0/1507007776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.612+0000 7efebea00700 1 -- 172.21.15.42:0/1507007776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec81034b0 msgr2=0x7efec8075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.612+0000 7efebea00700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec81034b0 0x7efec8075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.612+0000 7efebea00700 1 -- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8108730 msgr2=0x7efec806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.612+0000 7efebea00700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8108730 0x7efec806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.612+0000 7efebea00700 1 -- 172.21.15.42:0/1507007776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec8072f50 con 0x7efec8107de0 2024-05-27T10:21:00.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.613+0000 7efebea00700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 0x7efec806e5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efeb0010720 tx=0x7efeb000bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:00.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.614+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0009e60 con 0x7efec8107de0 2024-05-27T10:21:00.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.614+0000 7efece400700 1 -- 172.21.15.42:0/1507007776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec80731e0 con 0x7efec8107de0 2024-05-27T10:21:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.614+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb0011e50 con 0x7efec8107de0 2024-05-27T10:21:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.614+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0022450 con 0x7efec8107de0 2024-05-27T10:21:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.614+0000 7efece400700 1 -- 172.21.15.42:0/1507007776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec8073770 con 0x7efec8107de0 2024-05-27T10:21:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.617+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec8104960 con 0x7efec8107de0 2024-05-27T10:21:00.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.621+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efeb00037a0 con 0x7efec8107de0 2024-05-27T10:21:00.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.624+0000 7efec4c00700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efeac0b2a70 0x7efeac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:00.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.624+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7efeb00e8900 con 0x7efec8107de0 2024-05-27T10:21:00.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.625+0000 7efec7400700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efeac0b2a70 0x7efeac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:00.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.626+0000 7efec7400700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efeac0b2a70 0x7efeac0b4f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efeb800bfd0 tx=0x7efeb800b040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:00.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:00.636+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efeb00b0470 con 0x7efec8107de0 2024-05-27T10:21:00.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:00 smithi042 ceph-mon[31661]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:21:01.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:00 smithi143 ceph-mon[35545]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:21:01.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.083+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efec806fa80 con 0x7efec8107de0 2024-05-27T10:21:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:21:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:21:01.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.085+0000 7efec4c00700 1 -- 172.21.15.42:0/1507007776 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7efeb00afd70 con 0x7efec8107de0 2024-05-27T10:21:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.089+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efeac0b2a70 msgr2=0x7efeac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.089+0000 7efebd600700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efeac0b2a70 0x7efeac0b4f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efeb800bfd0 tx=0x7efeb800b040 comp rx=0 tx=0).stop 2024-05-27T10:21:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.089+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 msgr2=0x7efec806e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.089+0000 7efebd600700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 0x7efec806e5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efeb0010720 tx=0x7efeb000bee0 comp rx=0 tx=0).stop 2024-05-27T10:21:01.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 shutdown_connections 2024-05-27T10:21:01.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7efeac0b2a70 0x7efeac0b4f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:01.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8108730 0x7efec806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:01.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec81034b0 0x7efec8075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:01.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 --2- 172.21.15.42:0/1507007776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efec8107de0 0x7efec806e5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:01.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 >> 172.21.15.42:0/1507007776 conn(0x7efec80fedb0 msgr2=0x7efec81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:01.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 shutdown_connections 2024-05-27T10:21:01.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:01.091+0000 7efebd600700 1 -- 172.21.15.42:0/1507007776 wait complete. 2024-05-27T10:21:01.094 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:21:01.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:00 smithi104 ceph-mon[34812]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T10:21:02.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:01 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1507007776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:21:02.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:01 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1507007776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:21:02.347 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:01 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1507007776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:21:02.348 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T10:21:02.986 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:03.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:02 smithi143 ceph-mon[35545]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:03.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:02 smithi042 ceph-mon[31661]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:03.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:02 smithi104 ceph-mon[34812]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:04.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:04 smithi042 ceph-mon[31661]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:05.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:04 smithi143 ceph-mon[35545]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:05.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:04 smithi104 ceph-mon[34812]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 -- 172.21.15.42:0/208197547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 msgr2=0x7f2fc0116f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 --2- 172.21.15.42:0/208197547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 0x7f2fc0116f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009ca0 tx=0x7f2fb802a8e0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 -- 172.21.15.42:0/208197547 shutdown_connections 2024-05-27T10:21:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 --2- 172.21.15.42:0/208197547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 0x7f2fc0116f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 --2- 172.21.15.42:0/208197547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 --2- 172.21.15.42:0/208197547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 0x7f2fc0077a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 -- 172.21.15.42:0/208197547 >> 172.21.15.42:0/208197547 conn(0x7f2fc0101530 msgr2=0x7f2fc0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 -- 172.21.15.42:0/208197547 shutdown_connections 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.491+0000 7f2fc6000640 1 -- 172.21.15.42:0/208197547 wait complete. 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.493+0000 7f2fc6000640 1 Processor -- start 2024-05-27T10:21:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.493+0000 7f2fc6000640 1 -- start start 2024-05-27T10:21:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.494+0000 7f2fc6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 0x7f2fc01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fc6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fc6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 0x7f2fc01a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fc6000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc011a340 con 0x7f2fc0077610 2024-05-27T10:21:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fc6000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc011a1c0 con 0x7f2fc0114b00 2024-05-27T10:21:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fc6000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc011a4c0 con 0x7f2fc01121a0 2024-05-27T10:21:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 0x7f2fc01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fbea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fbea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:57282/0 (socket says 172.21.15.42:57282) 2024-05-27T10:21:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.495+0000 7f2fbea00640 1 -- 172.21.15.42:0/3775880531 learned_addr learned my addr 172.21.15.42:0/3775880531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.496+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 0x7f2fc01a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.496+0000 7f2fbea00640 1 -- 172.21.15.42:0/3775880531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 msgr2=0x7f2fc01a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.496+0000 7f2fbea00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 0x7f2fc01a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.496+0000 7f2fbea00640 1 -- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 msgr2=0x7f2fc01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.497+0000 7f2fbea00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 0x7f2fc01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.497+0000 7f2fbea00640 1 -- 172.21.15.42:0/3775880531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc01a5be0 con 0x7f2fc01121a0 2024-05-27T10:21:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.497+0000 7f2fbea00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01a12e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00e970 tx=0x7f2fac00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.497+0000 7f2fbf400640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 0x7f2fc01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.498+0000 7f2fb7e00640 1 -- 172.21.15.42:0/3775880531 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00ccc0 con 0x7f2fc01121a0 2024-05-27T10:21:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.498+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc01a5ed0 con 0x7f2fc01121a0 2024-05-27T10:21:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.498+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc01ad850 con 0x7f2fc01121a0 2024-05-27T10:21:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.498+0000 7f2fb7e00640 1 -- 172.21.15.42:0/3775880531 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fac0218d0 con 0x7f2fc01121a0 2024-05-27T10:21:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.498+0000 7f2fb7e00640 1 -- 172.21.15.42:0/3775880531 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac010690 con 0x7f2fc01121a0 2024-05-27T10:21:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.501+0000 7f2fccff4740 1 Processor -- start 2024-05-27T10:21:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.501+0000 7f2fccff4740 1 -- start start 2024-05-27T10:21:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 0x55645f8ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f8ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fccff4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55645f83fb90 con 0x55645f5032f0 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fccff4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55645f82f730 con 0x55645f52c1b0 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fccff4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55645f833770 con 0x55645f50b4d0 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 0x55645f8ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 0x55645f8ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:51672/0 (socket says 172.21.15.42:51672) 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.502+0000 7f2fbfe00640 1 -- 172.21.15.42:0/1254877274 learned_addr learned my addr 172.21.15.42:0/1254877274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbf400640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f8ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbea00640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbfe00640 1 -- 172.21.15.42:0/1254877274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 msgr2=0x55645f8cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbfe00640 1 -- 172.21.15.42:0/1254877274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 msgr2=0x55645f8ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f8ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbfe00640 1 -- 172.21.15.42:0/1254877274 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645f7c88b0 con 0x55645f52c1b0 2024-05-27T10:21:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.503+0000 7f2fbf400640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f8ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.504+0000 7f2fbea00640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.504+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 0x55645f8ce860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8035000 tx=0x7f2fb8031520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92db8722c9d57dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.504+0000 7f2fabe00640 1 -- 172.21.15.42:0/1254877274 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80319d0 con 0x55645f52c1b0 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.505+0000 7f2fabe00640 1 -- 172.21.15.42:0/1254877274 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fb8036040 con 0x55645f52c1b0 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.505+0000 7f2fabe00640 1 -- 172.21.15.42:0/1254877274 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80045c0 con 0x55645f52c1b0 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 -- 172.21.15.42:0/1254877274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 msgr2=0x55645f8ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 0x55645f8ce860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8035000 tx=0x7f2fb8031520 comp rx=0 tx=0).stop 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 -- 172.21.15.42:0/1254877274 shutdown_connections 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f8ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 --2- 172.21.15.42:0/1254877274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f52c1b0 0x55645f8ce860 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.506+0000 7f2fccff4740 1 -- 172.21.15.42:0/1254877274 >> 172.21.15.42:0/1254877274 conn(0x55645f520300 msgr2=0x55645f8cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.508+0000 7f2fb7e00640 1 -- 172.21.15.42:0/3775880531 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2fac021440 con 0x7f2fc01121a0 2024-05-27T10:21:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.508+0000 7f2fccff4740 1 -- 172.21.15.42:0/1254877274 shutdown_connections 2024-05-27T10:21:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.509+0000 7f2fccff4740 1 -- 172.21.15.42:0/1254877274 wait complete. 2024-05-27T10:21:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.510+0000 7f2fccff4740 1 -- start start 2024-05-27T10:21:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.511+0000 7f2fccff4740 1 Processor -- start 2024-05-27T10:21:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.511+0000 7f2fb7e00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2fa40b2930 0x7f2fa40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.511+0000 7f2fb7e00640 1 -- 172.21.15.42:0/3775880531 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f2fac0e2810 con 0x7f2fc01121a0 2024-05-27T10:21:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.511+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2fa40b2930 0x7f2fa40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.513+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 0x55645f8d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2fa40b2930 0x7f2fa40b4e40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009620 tx=0x7f2fb8005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 0x55645f8d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 0x55645f8d89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:57292/0 (socket says 172.21.15.42:57292) 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f8d8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fbf400640 1 -- 172.21.15.42:0/864068095 learned_addr learned my addr 172.21.15.42:0/864068095 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55645f515790 con 0x55645f52c1b0 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55645f7d7bc0 con 0x55645f50b4d0 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55645f8db930 con 0x55645f5032f0 2024-05-27T10:21:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.514+0000 7f2fccff4740 10 client.0 _ll_register_callbacks cb 0x55645f7b5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbf400640 1 -- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 msgr2=0x55645f8d8ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbf400640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f8d8ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbf400640 1 -- 172.21.15.42:0/864068095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 msgr2=0x55645f8e4cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbf400640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbf400640 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645f544210 con 0x55645f5032f0 2024-05-27T10:21:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.515+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.516+0000 7f2fbf400640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 0x55645f8d89b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012780 tx=0x7f2fb0012c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a243a3c984602861 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.516+0000 7f2fa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:21:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.516+0000 7f2fa8c00640 1 -- 172.21.15.42:0/864068095 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0010e80 con 0x55645f5032f0 2024-05-27T10:21:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.516+0000 7f2fa8c00640 1 -- 172.21.15.42:0/864068095 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f2fb0005020 con 0x55645f5032f0 2024-05-27T10:21:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.517+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb8036040 con 0x55645f5032f0 2024-05-27T10:21:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.517+0000 7f2fa8c00640 1 -- 172.21.15.42:0/864068095 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2fb0005260 con 0x55645f5032f0 2024-05-27T10:21:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.517+0000 7f2fccff4740 10 client.25459 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.517+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55645f610620 con 0x55645f5032f0 2024-05-27T10:21:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.518+0000 7f2fa8c00640 1 -- 172.21.15.42:0/864068095 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fb8036040 con 0x55645f5032f0 2024-05-27T10:21:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.518+0000 7f2fa8c00640 1 -- 172.21.15.42:0/864068095 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2fb001c490 con 0x55645f5032f0 2024-05-27T10:21:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.518+0000 7f2fccff4740 10 client.25459 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.518+0000 7f2fccff4740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.518+0000 7f2fccff4740 20 client.25459 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.518+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4ead80 0x55645f8eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.519+0000 7f2fccff4740 4 client.25459 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.519+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55645f841ae0 con 0x55645f4ead80 2024-05-27T10:21:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.519+0000 7f2fbea00640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4ead80 0x55645f8eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.520+0000 7f2fbea00640 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4ead80 0x55645f8eadf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21195fdf8d91944 server_cookie=84d6c8beced1d65e in_seq=0 out_seq=0 2024-05-27T10:21:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.520+0000 7f2fa8c00640 10 client.25459 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.522+0000 7f2fa8c00640 1 -- 172.21.15.42:0/864068095 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55645f841ae0 con 0x55645f4ead80 2024-05-27T10:21:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.522+0000 7f2fa8c00640 10 client.25459 handle_command_reply: tid=0 2024-05-27T10:21:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.522+0000 7f2fccff4740 1 client.25459 shutdown 2024-05-27T10:21:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 msgr2=0x55645f8d89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 0x55645f8d89b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012780 tx=0x7f2fb0012c40 comp rx=0 tx=0).stop 2024-05-27T10:21:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 shutdown_connections 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f8d8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4ead80 0x55645f8eadf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 --2- 172.21.15.42:0/864068095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f5032f0 0x55645f8d89b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 >> 172.21.15.42:0/864068095 conn(0x55645f520300 msgr2=0x55645f8ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.523+0000 7f2fa8c00640 0 client.25459 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.524+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 shutdown_connections 2024-05-27T10:21:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.524+0000 7f2fccff4740 1 -- 172.21.15.42:0/864068095 wait complete. 2024-05-27T10:21:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.524+0000 7f2fccff4740 20 client.25459 trim_cache size 0 max 16384 2024-05-27T10:21:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.557+0000 7f2fccff4740 1 Processor -- start 2024-05-27T10:21:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.557+0000 7f2fccff4740 1 -- start start 2024-05-27T10:21:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.558+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 0x55645f8d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.558+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.558+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.558+0000 7f2fccff4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0010f10 con 0x55645f52c1b0 2024-05-27T10:21:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.558+0000 7f2fccff4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa000e7e0 con 0x55645f5032f0 2024-05-27T10:21:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.558+0000 7f2fccff4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa000c630 con 0x55645f50b4d0 2024-05-27T10:21:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 0x55645f8d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 0x55645f8d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:51682/0 (socket says 172.21.15.42:51682) 2024-05-27T10:21:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 -- 172.21.15.42:0/3313346014 learned_addr learned my addr 172.21.15.42:0/3313346014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbea00640 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 -- 172.21.15.42:0/3313346014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 msgr2=0x55645f8e4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8e4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 -- 172.21.15.42:0/3313346014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 msgr2=0x55645f8e4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.559+0000 7f2fbf400640 1 -- 172.21.15.42:0/3313346014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645f8c6080 con 0x55645f5032f0 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.560+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.560+0000 7f2fbf400640 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 0x55645f8d41f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2fb001ca60 tx=0x7f2fb00108d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ad73e528720401 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.561+0000 7f2fab400640 1 -- 172.21.15.42:0/3313346014 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0029070 con 0x55645f5032f0 2024-05-27T10:21:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.561+0000 7f2fab400640 1 -- 172.21.15.42:0/3313346014 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fb0010e80 con 0x55645f5032f0 2024-05-27T10:21:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.561+0000 7f2fab400640 1 -- 172.21.15.42:0/3313346014 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00258c0 con 0x55645f5032f0 2024-05-27T10:21:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.562+0000 7f2fccff4740 1 -- 172.21.15.42:0/3313346014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 msgr2=0x55645f8d41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.562+0000 7f2fccff4740 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 0x55645f8d41f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2fb001ca60 tx=0x7f2fb00108d0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 -- 172.21.15.42:0/3313346014 shutdown_connections 2024-05-27T10:21:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f52c1b0 0x55645f8e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f50b4d0 0x55645f8e4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 --2- 172.21.15.42:0/3313346014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f5032f0 0x55645f8d41f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 -- 172.21.15.42:0/3313346014 >> 172.21.15.42:0/3313346014 conn(0x55645f520300 msgr2=0x55645f8d8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 -- 172.21.15.42:0/3313346014 shutdown_connections 2024-05-27T10:21:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.563+0000 7f2fccff4740 1 -- 172.21.15.42:0/3313346014 wait complete. 2024-05-27T10:21:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.564+0000 7f2fccff4740 1 -- start start 2024-05-27T10:21:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.564+0000 7f2fccff4740 1 Processor -- start 2024-05-27T10:21:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f94bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f94c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f52c1b0 0x55645f8dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa000cca0 con 0x55645f5032f0 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa000e110 con 0x55645f50b4d0 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa000d370 con 0x55645f52c1b0 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.565+0000 7f2fccff4740 10 client.0 _ll_register_callbacks cb 0x7f2fb000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.566+0000 7f2fbea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f94c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.566+0000 7f2fbea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f94c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:51696/0 (socket says 172.21.15.42:51696) 2024-05-27T10:21:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.566+0000 7f2fbea00640 1 -- 172.21.15.42:0/2733315337 learned_addr learned my addr 172.21.15.42:0/2733315337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.566+0000 7f2fbf400640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f94bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f52c1b0 0x55645f8dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbea00640 1 -- 172.21.15.42:0/2733315337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f52c1b0 msgr2=0x55645f8dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbea00640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f52c1b0 0x55645f8dfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbea00640 1 -- 172.21.15.42:0/2733315337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 msgr2=0x55645f94bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbea00640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f94bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbea00640 1 -- 172.21.15.42:0/2733315337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645f8e0190 con 0x55645f50b4d0 2024-05-27T10:21:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbfe00640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f52c1b0 0x55645f8dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.567+0000 7f2fbf400640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f94bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.568+0000 7f2fbea00640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f94c000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0a97e0 tx=0x7f2fac0a5f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20655a223a6c185b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.568+0000 7f2fa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:21:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.568+0000 7f2fa8c00640 1 -- 172.21.15.42:0/2733315337 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a5040 con 0x55645f50b4d0 2024-05-27T10:21:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.568+0000 7f2fa8c00640 1 -- 172.21.15.42:0/2733315337 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f2fac0b24b0 con 0x55645f50b4d0 2024-05-27T10:21:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.568+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55645f8cf130 con 0x55645f50b4d0 2024-05-27T10:21:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.569+0000 7f2fa8c00640 1 -- 172.21.15.42:0/2733315337 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2fac0aa1e0 con 0x55645f50b4d0 2024-05-27T10:21:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.569+0000 7f2fccff4740 10 client.35432 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.569+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55645f8e1480 con 0x55645f50b4d0 2024-05-27T10:21:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.569+0000 7f2fa8c00640 1 -- 172.21.15.42:0/2733315337 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55645f8cf130 con 0x55645f50b4d0 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fa8c00640 1 -- 172.21.15.42:0/2733315337 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2fac0a7a30 con 0x55645f50b4d0 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fccff4740 10 client.35432 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fccff4740 10 client.35432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fccff4740 20 client.35432 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fccff4740 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4c01c0 0x55645f9665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fccff4740 4 client.35432 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.570+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55645f83b2c0 con 0x55645f4c01c0 2024-05-27T10:21:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.571+0000 7f2fbf400640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4c01c0 0x55645f9665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.572+0000 7f2fbf400640 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4c01c0 0x55645f9665d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1abebad354f60a0 server_cookie=7c580f4e814b5ffb in_seq=0 out_seq=0 2024-05-27T10:21:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.572+0000 7f2fa8c00640 10 client.35432 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.587+0000 7f2fa8c00640 1 -- 172.21.15.42:0/2733315337 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55645f83b2c0 con 0x55645f4c01c0 2024-05-27T10:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.587+0000 7f2fa8c00640 10 client.35432 handle_command_reply: tid=0 2024-05-27T10:21:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 client.35432 shutdown 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 msgr2=0x55645f94c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f94c000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0a97e0 tx=0x7f2fac0a5f10 comp rx=0 tx=0).stop 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 shutdown_connections 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55645f4c01c0 0x55645f9665d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55645f52c1b0 0x55645f8dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55645f50b4d0 0x55645f94c000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 --2- 172.21.15.42:0/2733315337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55645f5032f0 0x55645f94bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 >> 172.21.15.42:0/2733315337 conn(0x55645f520300 msgr2=0x55645f8d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fa8c00640 0 client.35432 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 shutdown_connections 2024-05-27T10:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.588+0000 7f2fccff4740 1 -- 172.21.15.42:0/2733315337 wait complete. 2024-05-27T10:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.589+0000 7f2fccff4740 20 client.35432 trim_cache size 0 max 16384 2024-05-27T10:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:21:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2fa40b2930 msgr2=0x7f2fa40b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2fa40b2930 0x7f2fa40b4e40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009620 tx=0x7f2fb8005fd0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 msgr2=0x7f2fc01a12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01a12e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00e970 tx=0x7f2fac00ee30 comp rx=0 tx=0).stop 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 shutdown_connections 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2fa40b2930 0x7f2fa40b4e40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fc0114b00 0x7f2fc01a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fc01121a0 0x7f2fc01a12e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 --2- 172.21.15.42:0/3775880531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fc0077610 0x7f2fc01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.590+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 >> 172.21.15.42:0/3775880531 conn(0x7f2fc0101530 msgr2=0x7f2fc01134c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.591+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 shutdown_connections 2024-05-27T10:21:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:05.591+0000 7f2fc6000640 1 -- 172.21.15.42:0/3775880531 wait complete. 2024-05-27T10:21:05.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 -- 172.21.15.42:0/1569704091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea8108730 msgr2=0x7f3ea8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 --2- 172.21.15.42:0/1569704091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea8108730 0x7f3ea8110090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b0d0 tx=0x7f3ea000b3e0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 -- 172.21.15.42:0/1569704091 shutdown_connections 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 --2- 172.21.15.42:0/1569704091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea8108730 0x7f3ea8110090 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 --2- 172.21.15.42:0/1569704091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ea81034b0 0x7f3ea81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 --2- 172.21.15.42:0/1569704091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 0x7f3ea81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 -- 172.21.15.42:0/1569704091 >> 172.21.15.42:0/1569704091 conn(0x7f3ea80fedb0 msgr2=0x7f3ea81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 -- 172.21.15.42:0/1569704091 shutdown_connections 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.686+0000 7f3eb0200700 1 -- 172.21.15.42:0/1569704091 wait complete. 2024-05-27T10:21:05.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.688+0000 7f3eb0200700 1 Processor -- start 2024-05-27T10:21:05.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.688+0000 7f3eb0200700 1 -- start start 2024-05-27T10:21:05.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.690+0000 7f3eb0200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 0x7f3ea81a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.690+0000 7f3eb0200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 0x7f3ea81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.690+0000 7f3ead800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 0x7f3ea81a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.690+0000 7f3eb0200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ea8108730 0x7f3ea81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.690+0000 7f3ead800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 0x7f3ea81a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49454/0 (socket says 172.21.15.42:49454) 2024-05-27T10:21:05.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.691+0000 7f3ead800700 1 -- 172.21.15.42:0/3549554574 learned_addr learned my addr 172.21.15.42:0/3549554574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.691+0000 7f3eace00700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 0x7f3ea81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.691+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81121c0 con 0x7f3ea81034b0 2024-05-27T10:21:05.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.691+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8112040 con 0x7f3ea8107de0 2024-05-27T10:21:05.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.691+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8112340 con 0x7f3ea8108730 2024-05-27T10:21:05.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.692+0000 7f3eae200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ea8108730 0x7f3ea81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.693+0000 7f3eace00700 1 -- 172.21.15.42:0/3549554574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ea8108730 msgr2=0x7f3ea81a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.693+0000 7f3eace00700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ea8108730 0x7f3ea81a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.693+0000 7f3eace00700 1 -- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 msgr2=0x7f3ea81a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.693+0000 7f3eace00700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 0x7f3ea81a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.693+0000 7f3eace00700 1 -- 172.21.15.42:0/3549554574 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81a61b0 con 0x7f3ea8107de0 2024-05-27T10:21:05.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.693+0000 7f3ead800700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 0x7f3ea81a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:05.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.694+0000 7f3eace00700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 0x7f3ea81a1880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400e9c0 tx=0x7f3e9400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.694+0000 7f3ea6000700 1 -- 172.21.15.42:0/3549554574 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400c9c0 con 0x7f3ea8107de0 2024-05-27T10:21:05.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.694+0000 7f3ea6000700 1 -- 172.21.15.42:0/3549554574 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e94010430 con 0x7f3ea8107de0 2024-05-27T10:21:05.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.695+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81a64a0 con 0x7f3ea8107de0 2024-05-27T10:21:05.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.695+0000 7f3ea6000700 1 -- 172.21.15.42:0/3549554574 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94018680 con 0x7f3ea8107de0 2024-05-27T10:21:05.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.696+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81addc0 con 0x7f3ea8107de0 2024-05-27T10:21:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.701+0000 7f3ea6000700 1 -- 172.21.15.42:0/3549554574 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3e940108e0 con 0x7f3ea8107de0 2024-05-27T10:21:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.703+0000 7f3ebed96b80 1 Processor -- start 2024-05-27T10:21:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.704+0000 7f3ea6000700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3e900b2a60 0x7f3e900b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.704+0000 7f3ebed96b80 1 -- start start 2024-05-27T10:21:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.704+0000 7f3ea6000700 1 -- 172.21.15.42:0/3549554574 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3e940e2760 con 0x7f3ea8107de0 2024-05-27T10:21:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.705+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b5747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.705+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b56e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.705+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b56f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.705+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c23b49ce40 con 0x55c23b3efe40 2024-05-27T10:21:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.705+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c23b48c9e0 con 0x55c23b574d30 2024-05-27T10:21:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.705+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c23b490a20 con 0x55c23b56edb0 2024-05-27T10:21:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.706+0000 7f3eae200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b56f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.706+0000 7f3ead800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b5747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.706+0000 7f3eae200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3e900b2a60 0x7f3e900b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.706+0000 7f3ead800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b5747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49456/0 (socket says 172.21.15.42:49456) 2024-05-27T10:21:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.706+0000 7f3ead800700 1 -- 172.21.15.42:0/1862156909 learned_addr learned my addr 172.21.15.42:0/1862156909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3ead800700 1 -- 172.21.15.42:0/1862156909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 msgr2=0x55c23b56f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3eace00700 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b56e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3ead800700 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b56f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3ead800700 1 -- 172.21.15.42:0/1862156909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 msgr2=0x55c23b56e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3ead800700 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b56e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3ead800700 1 -- 172.21.15.42:0/1862156909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23b455420 con 0x55c23b3efe40 2024-05-27T10:21:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3eace00700 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b56e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.707+0000 7f3eae200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3e900b2a60 0x7f3e900b4f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009e60 tx=0x7f3ea0009820 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.708+0000 7f3ead800700 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b5747f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0107a0 tx=0x7f3e9c010ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3543d6df9c91350a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.709+0000 7f3e9aa00700 1 -- 172.21.15.42:0/1862156909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c011700 con 0x55c23b3efe40 2024-05-27T10:21:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.710+0000 7f3e9aa00700 1 -- 172.21.15.42:0/1862156909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c011d30 con 0x55c23b3efe40 2024-05-27T10:21:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.710+0000 7f3e9aa00700 1 -- 172.21.15.42:0/1862156909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c019cf0 con 0x55c23b3efe40 2024-05-27T10:21:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1862156909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 msgr2=0x55c23b5747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b5747f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0107a0 tx=0x7f3e9c010ab0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1862156909 shutdown_connections 2024-05-27T10:21:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b56f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b5747f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1862156909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b56e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.711+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1862156909 >> 172.21.15.42:0/1862156909 conn(0x55c23b44ffc0 msgr2=0x55c23b5698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.712+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1862156909 shutdown_connections 2024-05-27T10:21:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.712+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1862156909 wait complete. 2024-05-27T10:21:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.713+0000 7f3ebed96b80 1 -- start start 2024-05-27T10:21:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.713+0000 7f3ebed96b80 1 Processor -- start 2024-05-27T10:21:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b578480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b5789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b578f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c23b392fa0 con 0x55c23b3efe40 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c23b42d760 con 0x55c23b574d30 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c23b57b800 con 0x55c23b56edb0 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.714+0000 7f3ebed96b80 10 client.0 _ll_register_callbacks cb 0x55c23b436c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b578480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3eace00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b5789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b578480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49460/0 (socket says 172.21.15.42:49460) 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 -- 172.21.15.42:0/1388560677 learned_addr learned my addr 172.21.15.42:0/1388560677 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3eae200700 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b578f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 -- 172.21.15.42:0/1388560677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 msgr2=0x55c23b5789c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b5789c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 -- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 msgr2=0x55c23b578f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b578f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.715+0000 7f3ead800700 1 -- 172.21.15.42:0/1388560677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23b3dc8d0 con 0x55c23b3efe40 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.716+0000 7f3ead800700 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b578480 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c000c00 tx=0x7f3e9c019470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e7bd978104aec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.716+0000 7f3e8f400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.717+0000 7f3e8f400700 1 -- 172.21.15.42:0/1388560677 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c01f040 con 0x55c23b3efe40 2024-05-27T10:21:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.717+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9c011d30 con 0x55c23b3efe40 2024-05-27T10:21:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.717+0000 7f3e8f400700 1 -- 172.21.15.42:0/1388560677 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3e9c019c30 con 0x55c23b3efe40 2024-05-27T10:21:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.718+0000 7f3e8f400700 1 -- 172.21.15.42:0/1388560677 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c019e50 con 0x55c23b3efe40 2024-05-27T10:21:05.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.719+0000 7f3ebed96b80 10 client.15810 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:05.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.719+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c23b45d060 con 0x55c23b3efe40 2024-05-27T10:21:05.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.720+0000 7f3e8f400700 1 -- 172.21.15.42:0/1388560677 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c011d30 con 0x55c23b3efe40 2024-05-27T10:21:05.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.720+0000 7f3e8f400700 1 -- 172.21.15.42:0/1388560677 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3e9c022030 con 0x55c23b3efe40 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3ebed96b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3ebed96b80 10 client.15810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3ebed96b80 20 client.15810 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b588580 0x55c23b58a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3ebed96b80 4 client.15810 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c23b49ed90 con 0x55c23b588580 2024-05-27T10:21:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.721+0000 7f3eace00700 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b588580 0x55c23b58a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.722+0000 7f3e8f400700 10 client.15810 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.726+0000 7f3eace00700 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b588580 0x55c23b58a9a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57216c9d4a3f5e00 server_cookie=774fac074211534c in_seq=0 out_seq=0 2024-05-27T10:21:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.727+0000 7f3e8f400700 1 -- 172.21.15.42:0/1388560677 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c23b49ed90 con 0x55c23b588580 2024-05-27T10:21:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.727+0000 7f3e8f400700 10 client.15810 handle_command_reply: tid=0 2024-05-27T10:21:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.727+0000 7f3ebed96b80 1 client.15810 shutdown 2024-05-27T10:21:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 msgr2=0x55c23b578480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b578480 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c000c00 tx=0x7f3e9c019470 comp rx=0 tx=0).stop 2024-05-27T10:21:05.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 shutdown_connections 2024-05-27T10:21:05.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b588580 0x55c23b58a9a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3e8f400700 0 client.15810 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:05.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b574d30 0x55c23b578f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b3efe40 0x55c23b578480 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/1388560677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b5789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.728+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 >> 172.21.15.42:0/1388560677 conn(0x55c23b44ffc0 msgr2=0x55c23b569580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.730+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 shutdown_connections 2024-05-27T10:21:05.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.730+0000 7f3ebed96b80 1 -- 172.21.15.42:0/1388560677 wait complete. 2024-05-27T10:21:05.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.730+0000 7f3ebed96b80 20 client.15810 trim_cache size 0 max 16384 2024-05-27T10:21:05.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 Processor -- start 2024-05-27T10:21:05.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 -- start start 2024-05-27T10:21:05.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b56e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b57a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b574d30 0x55c23b5848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b7c20 con 0x55c23b574d30 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b54b0 con 0x55c23b3efe40 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.765+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b3320 con 0x55c23b56edb0 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.766+0000 7f3ead800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b56e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.766+0000 7f3eae200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b574d30 0x55c23b5848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.766+0000 7f3ead800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b56e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:51736/0 (socket says 172.21.15.42:51736) 2024-05-27T10:21:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.766+0000 7f3ead800700 1 -- 172.21.15.42:0/8636771 learned_addr learned my addr 172.21.15.42:0/8636771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.766+0000 7f3eace00700 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b57a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3ead800700 1 -- 172.21.15.42:0/8636771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 msgr2=0x55c23b57a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3ead800700 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b57a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3ead800700 1 -- 172.21.15.42:0/8636771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b574d30 msgr2=0x55c23b5848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3ead800700 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b574d30 0x55c23b5848a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3ead800700 1 -- 172.21.15.42:0/8636771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23b579270 con 0x55c23b3efe40 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3eace00700 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b57a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3eae200700 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b574d30 0x55c23b5848a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:05.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.767+0000 7f3ead800700 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b56e870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0106c0 tx=0x7f3e9c00b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74ba42290ded8dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.768+0000 7f3e9a000700 1 -- 172.21.15.42:0/8636771 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c019560 con 0x55c23b3efe40 2024-05-27T10:21:05.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.768+0000 7f3e9a000700 1 -- 172.21.15.42:0/8636771 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c03a070 con 0x55c23b3efe40 2024-05-27T10:21:05.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.768+0000 7f3e9a000700 1 -- 172.21.15.42:0/8636771 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c003680 con 0x55c23b3efe40 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.769+0000 7f3ebed96b80 1 -- 172.21.15.42:0/8636771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 msgr2=0x55c23b56e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.769+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b56e870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0106c0 tx=0x7f3e9c00b650 comp rx=0 tx=0).stop 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 -- 172.21.15.42:0/8636771 shutdown_connections 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b574d30 0x55c23b5848a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b56e870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/8636771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b56edb0 0x55c23b57a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 -- 172.21.15.42:0/8636771 >> 172.21.15.42:0/8636771 conn(0x55c23b44ffc0 msgr2=0x55c23b576c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 -- 172.21.15.42:0/8636771 shutdown_connections 2024-05-27T10:21:05.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.770+0000 7f3ebed96b80 1 -- 172.21.15.42:0/8636771 wait complete. 2024-05-27T10:21:05.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.771+0000 7f3ebed96b80 1 -- start start 2024-05-27T10:21:05.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.771+0000 7f3ebed96b80 1 Processor -- start 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b580e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b56edb0 0x55c23b58f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b574d30 0x55c23b58fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b3970 con 0x55c23b56edb0 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b4de0 con 0x55c23b3efe40 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b4040 con 0x55c23b574d30 2024-05-27T10:21:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.772+0000 7f3ebed96b80 10 client.0 _ll_register_callbacks cb 0x55c23b436c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:05.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.773+0000 7f3ead800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b580e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.773+0000 7f3eae200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b574d30 0x55c23b58fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.773+0000 7f3eae200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b574d30 0x55c23b58fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:57348/0 (socket says 172.21.15.42:57348) 2024-05-27T10:21:05.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.773+0000 7f3eae200700 1 -- 172.21.15.42:0/812982340 learned_addr learned my addr 172.21.15.42:0/812982340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:05.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3ead800700 1 -- 172.21.15.42:0/812982340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b574d30 msgr2=0x55c23b58fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3ead800700 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b574d30 0x55c23b58fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3ead800700 1 -- 172.21.15.42:0/812982340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b56edb0 msgr2=0x55c23b58f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:05.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3ead800700 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b56edb0 0x55c23b58f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3ead800700 1 -- 172.21.15.42:0/812982340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23b58ff90 con 0x55c23b3efe40 2024-05-27T10:21:05.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3ead800700 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b580e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c011570 tx=0x7f3e9c0118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ef95d6b8f3a13 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:05.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.774+0000 7f3e8f400700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:21:05.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.775+0000 7f3e8f400700 1 -- 172.21.15.42:0/812982340 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c0051f0 con 0x55c23b3efe40 2024-05-27T10:21:05.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.777+0000 7f3e8f400700 1 -- 172.21.15.42:0/812982340 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3e9c005390 con 0x55c23b3efe40 2024-05-27T10:21:05.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.777+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9c03a070 con 0x55c23b3efe40 2024-05-27T10:21:05.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.797+0000 7f3e8f400700 1 -- 172.21.15.42:0/812982340 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c019a30 con 0x55c23b3efe40 2024-05-27T10:21:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.798+0000 7f3ebed96b80 10 client.35447 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.798+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c23b5f29b0 con 0x55c23b3efe40 2024-05-27T10:21:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.798+0000 7f3e8f400700 1 -- 172.21.15.42:0/812982340 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c03a070 con 0x55c23b3efe40 2024-05-27T10:21:05.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.799+0000 7f3e8f400700 1 -- 172.21.15.42:0/812982340 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3e9c022040 con 0x55c23b3efe40 2024-05-27T10:21:05.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.800+0000 7f3ebed96b80 10 client.35447 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:05.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.800+0000 7f3ebed96b80 10 client.35447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:21:05.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.800+0000 7f3ebed96b80 20 client.35447 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:05.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.800+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b5f2d70 0x55c23b5f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:05.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.800+0000 7f3ebed96b80 4 client.35447 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:21:05.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.800+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c23b498570 con 0x55c23b5f2d70 2024-05-27T10:21:05.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.801+0000 7f3eace00700 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b5f2d70 0x55c23b5f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:05.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.802+0000 7f3eace00700 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b5f2d70 0x55c23b5f5190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dacf3f4f4d516c6a server_cookie=bf6c8281dc29111f in_seq=0 out_seq=0 2024-05-27T10:21:05.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.802+0000 7f3e8f400700 10 client.35447 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.802+0000 7f3e8f400700 1 -- 172.21.15.42:0/812982340 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c23b498570 con 0x55c23b5f2d70 2024-05-27T10:21:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.802+0000 7f3e8f400700 10 client.35447 handle_command_reply: tid=0 2024-05-27T10:21:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.802+0000 7f3ebed96b80 1 client.35447 shutdown 2024-05-27T10:21:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 msgr2=0x55c23b580e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b580e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c011570 tx=0x7f3e9c0118e0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 shutdown_connections 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55c23b5f2d70 0x55c23b5f5190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c23b574d30 0x55c23b58fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3e8f400700 0 client.35447 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c23b3efe40 0x55c23b580e70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 --2- 172.21.15.42:0/812982340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c23b56edb0 0x55c23b58f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 >> 172.21.15.42:0/812982340 conn(0x55c23b44ffc0 msgr2=0x55c23b576c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 shutdown_connections 2024-05-27T10:21:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.803+0000 7f3ebed96b80 1 -- 172.21.15.42:0/812982340 wait complete. 2024-05-27T10:21:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.804+0000 7f3ebed96b80 20 client.35447 trim_cache size 0 max 16384 2024-05-27T10:21:05.805 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:21:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.805+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3e900b2a60 msgr2=0x7f3e900b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.805+0000 7f3eb0200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3e900b2a60 0x7f3e900b4f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009e60 tx=0x7f3ea0009820 comp rx=0 tx=0).stop 2024-05-27T10:21:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.805+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 msgr2=0x7f3ea81a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.805+0000 7f3eb0200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 0x7f3ea81a1880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400e9c0 tx=0x7f3e9400ecd0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 shutdown_connections 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3e900b2a60 0x7f3e900b4f70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ea8108730 0x7f3ea81a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ea81034b0 0x7f3ea81a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 --2- 172.21.15.42:0/3549554574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea8107de0 0x7f3ea81a1880 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 >> 172.21.15.42:0/3549554574 conn(0x7f3ea80fedb0 msgr2=0x7f3ea8109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:05.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 shutdown_connections 2024-05-27T10:21:05.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:05.806+0000 7f3eb0200700 1 -- 172.21.15.42:0/3549554574 wait complete. 2024-05-27T10:21:06.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:05 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:21:06.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:05 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:21:06.099 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:05 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:21:06.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T10:21:07.059 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:06 smithi143 ceph-mon[35545]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:21:07.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:06 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:21:07.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:06 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:21:07.061 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:06 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:21:07.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:06 smithi042 ceph-mon[31661]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:21:07.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:06 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:21:07.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:06 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:21:07.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:06 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:21:07.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:06 smithi104 ceph-mon[34812]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:21:07.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:06 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:21:07.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:06 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:21:07.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:06 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:21:07.309 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:08.795 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:08 smithi042 ceph-mon[31661]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-27T10:21:09.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:08 smithi143 ceph-mon[35545]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-27T10:21:09.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:08 smithi104 ceph-mon[34812]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-27T10:21:10.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 -- 172.21.15.42:0/2386750170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81030f0 msgr2=0x7fe4b8103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 --2- 172.21.15.42:0/2386750170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81030f0 0x7fe4b8103500 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009990 tx=0x7fe4a8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 -- 172.21.15.42:0/2386750170 shutdown_connections 2024-05-27T10:21:10.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 --2- 172.21.15.42:0/2386750170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b81130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 --2- 172.21.15.42:0/2386750170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81030f0 0x7fe4b8103500 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 --2- 172.21.15.42:0/2386750170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8103a40 0x7fe4b8110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 -- 172.21.15.42:0/2386750170 >> 172.21.15.42:0/2386750170 conn(0x7fe4b80fedb0 msgr2=0x7fe4b81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 -- 172.21.15.42:0/2386750170 shutdown_connections 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:09.998+0000 7fe4bee00700 1 -- 172.21.15.42:0/2386750170 wait complete. 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.000+0000 7fe4bee00700 1 Processor -- start 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.000+0000 7fe4bee00700 1 -- start start 2024-05-27T10:21:10.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4bee00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b8103a40 0x7fe4b819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4bee00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4bee00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b81a1880 0x7fe4b81a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4bee00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81165a0 con 0x7fe4b8103a40 2024-05-27T10:21:10.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4bee00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8116420 con 0x7fe4b81a1880 2024-05-27T10:21:10.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4bee00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8116720 con 0x7fe4b8110ca0 2024-05-27T10:21:10.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.001+0000 7fe4b7400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:57366/0 (socket says 172.21.15.42:57366) 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 -- 172.21.15.42:0/3630728460 learned_addr learned my addr 172.21.15.42:0/3630728460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4bce00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b81a1880 0x7fe4b81a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b8103a40 0x7fe4b819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 -- 172.21.15.42:0/3630728460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b81a1880 msgr2=0x7fe4b81a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b81a1880 0x7fe4b81a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 -- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b8103a40 msgr2=0x7fe4b819d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b8103a40 0x7fe4b819d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4b7400700 1 -- 172.21.15.42:0/3630728460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81a23d0 con 0x7fe4b8110ca0 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.002+0000 7fe4bce00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b81a1880 0x7fe4b81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.003+0000 7fe4b7400700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b819d6b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400d980 tx=0x7fe4a400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.003+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b8103a40 0x7fe4b819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.003+0000 7fe4b4c00700 1 -- 172.21.15.42:0/3630728460 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4016070 con 0x7fe4b8110ca0 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.004+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81a26c0 con 0x7fe4b8110ca0 2024-05-27T10:21:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.004+0000 7fe4b4c00700 1 -- 172.21.15.42:0/3630728460 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a400bde0 con 0x7fe4b8110ca0 2024-05-27T10:21:10.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.004+0000 7fe4b4c00700 1 -- 172.21.15.42:0/3630728460 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400f3c0 con 0x7fe4b8110ca0 2024-05-27T10:21:10.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.004+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81a9ef0 con 0x7fe4b8110ca0 2024-05-27T10:21:10.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.010+0000 7fe4b4c00700 1 -- 172.21.15.42:0/3630728460 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe4a4019070 con 0x7fe4b8110ca0 2024-05-27T10:21:10.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.011+0000 7fe4cd8b0b80 1 Processor -- start 2024-05-27T10:21:10.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.011+0000 7fe4b4c00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.011+0000 7fe4cd8b0b80 1 -- start start 2024-05-27T10:21:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4b4c00700 1 -- 172.21.15.42:0/3630728460 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe4a40e2390 con 0x7fe4b8110ca0 2024-05-27T10:21:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b2fb2fee40 con 0x55b2fb3d0db0 2024-05-27T10:21:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2fb2ee9e0 con 0x55b2fb251e40 2024-05-27T10:21:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.012+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b2fb2f2a20 con 0x55b2fb3d6d30 2024-05-27T10:21:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.013+0000 7fe4b7400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.013+0000 7fe4b7400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:57368/0 (socket says 172.21.15.42:57368) 2024-05-27T10:21:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.013+0000 7fe4b7400700 1 -- 172.21.15.42:0/2258559445 learned_addr learned my addr 172.21.15.42:0/2258559445 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.013+0000 7fe4bce00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7400700 1 -- 172.21.15.42:0/2258559445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 msgr2=0x55b2fb3d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7400700 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7400700 1 -- 172.21.15.42:0/2258559445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 msgr2=0x55b2fb3d11c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7400700 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3d11c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7400700 1 -- 172.21.15.42:0/2258559445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2fb2b7420 con 0x55b2fb3d6d30 2024-05-27T10:21:10.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:10.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.014+0000 7fe4b7400700 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d0870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40e0130 tx=0x7fe4a40a8f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f156735680066431 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:10.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.015+0000 7fe4ad600700 1 -- 172.21.15.42:0/2258559445 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40ada70 con 0x55b2fb3d6d30 2024-05-27T10:21:10.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.015+0000 7fe4ad600700 1 -- 172.21.15.42:0/2258559445 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a40b2520 con 0x55b2fb3d6d30 2024-05-27T10:21:10.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.015+0000 7fe4ad600700 1 -- 172.21.15.42:0/2258559445 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a408fe40 con 0x55b2fb3d6d30 2024-05-27T10:21:10.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2258559445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 msgr2=0x55b2fb3d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4bce00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00076f0 tx=0x7fe4b0009100 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:10.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d0870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40e0130 tx=0x7fe4a40a8f00 comp rx=0 tx=0).stop 2024-05-27T10:21:10.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2258559445 shutdown_connections 2024-05-27T10:21:10.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2258559445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d0870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.016+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2258559445 >> 172.21.15.42:0/2258559445 conn(0x55b2fb2b1fc0 msgr2=0x55b2fb3cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:10.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.017+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2258559445 shutdown_connections 2024-05-27T10:21:10.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.017+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2258559445 wait complete. 2024-05-27T10:21:10.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.017+0000 7fe4cd8b0b80 1 -- start start 2024-05-27T10:21:10.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.018+0000 7fe4cd8b0b80 1 Processor -- start 2024-05-27T10:21:10.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb251e40 0x55b2fb3da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b2fb1f4fa0 con 0x55b2fb3d6d30 2024-05-27T10:21:10.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2fb28f760 con 0x55b2fb3d0db0 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b2fb3dd800 con 0x55b2fb251e40 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4cd8b0b80 10 client.0 _ll_register_callbacks cb 0x55b2fb298c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4bce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4bce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3db010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49516/0 (socket says 172.21.15.42:49516) 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4bce00700 1 -- 172.21.15.42:0/1293275849 learned_addr learned my addr 172.21.15.42:0/1293275849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.019+0000 7fe4b7400700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb251e40 0x55b2fb3da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4bce00700 1 -- 172.21.15.42:0/1293275849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb251e40 msgr2=0x55b2fb3da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:10.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4bce00700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb251e40 0x55b2fb3da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4bce00700 1 -- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 msgr2=0x55b2fb3daad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4bce00700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3daad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4bce00700 1 -- 172.21.15.42:0/1293275849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2fb23e8d0 con 0x55b2fb3d6d30 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.020+0000 7fe4b7400700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.021+0000 7fe4bce00700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3db010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0015f60 tx=0x7fe4b0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a504a5741404e5fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.021+0000 7fe4a2000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.021+0000 7fe4a2000700 1 -- 172.21.15.42:0/1293275849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b001c050 con 0x55b2fb3d6d30 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.021+0000 7fe4a2000700 1 -- 172.21.15.42:0/1293275849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe4b0021640 con 0x55b2fb3d6d30 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.021+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a40b2520 con 0x55b2fb3d6d30 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.022+0000 7fe4a2000700 1 -- 172.21.15.42:0/1293275849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4b002f050 con 0x55b2fb3d6d30 2024-05-27T10:21:10.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.022+0000 7fe4cd8b0b80 10 client.15819 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.022+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2fb2bf060 con 0x55b2fb3d6d30 2024-05-27T10:21:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.022+0000 7fe4a2000700 1 -- 172.21.15.42:0/1293275849 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a40b2520 con 0x55b2fb3d6d30 2024-05-27T10:21:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.022+0000 7fe4a2000700 1 -- 172.21.15.42:0/1293275849 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe4b002a740 con 0x55b2fb3d6d30 2024-05-27T10:21:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.023+0000 7fe4cd8b0b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.023+0000 7fe4cd8b0b80 10 client.15819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:21:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.023+0000 7fe4cd8b0b80 20 client.15819 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.023+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb3ea7b0 0x55b2fb3ecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.023+0000 7fe4cd8b0b80 4 client.15819 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.023+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2fb300d90 con 0x55b2fb3ea7b0 2024-05-27T10:21:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.024+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb3ea7b0 0x55b2fb3ecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.025+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb3ea7b0 0x55b2fb3ecbf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74762d4ba3587a07 server_cookie=4992411b9b993683 in_seq=0 out_seq=0 2024-05-27T10:21:10.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.025+0000 7fe4a2000700 10 client.15819 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.027+0000 7fe4a2000700 1 -- 172.21.15.42:0/1293275849 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b2fb300d90 con 0x55b2fb3ea7b0 2024-05-27T10:21:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.027+0000 7fe4a2000700 10 client.15819 handle_command_reply: tid=0 2024-05-27T10:21:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.027+0000 7fe4cd8b0b80 1 client.15819 shutdown 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 msgr2=0x55b2fb3db010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3db010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0015f60 tx=0x7fe4b0004b40 comp rx=0 tx=0).stop 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 shutdown_connections 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb3ea7b0 0x55b2fb3ecbf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3db010 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb251e40 0x55b2fb3da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/1293275849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 >> 172.21.15.42:0/1293275849 conn(0x55b2fb2b1fc0 msgr2=0x55b2fb3cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4a2000700 0 client.15819 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:10.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.028+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 shutdown_connections 2024-05-27T10:21:10.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.029+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/1293275849 wait complete. 2024-05-27T10:21:10.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.029+0000 7fe4cd8b0b80 20 client.15819 trim_cache size 0 max 16384 2024-05-27T10:21:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.045+0000 7fe4cd8b0b80 1 Processor -- start 2024-05-27T10:21:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 -- start start 2024-05-27T10:21:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81afb00 con 0x55b2fb3d0db0 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81ad390 con 0x55b2fb251e40 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.046+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81ab2d0 con 0x55b2fb3d6d30 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.047+0000 7fe4b7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.047+0000 7fe4b7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.047+0000 7fe4b7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:51796/0 (socket says 172.21.15.42:51796) 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.047+0000 7fe4b7e00700 1 -- 172.21.15.42:0/2227087030 learned_addr learned my addr 172.21.15.42:0/2227087030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.047+0000 7fe4bce00700 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7400700 1 -- 172.21.15.42:0/2227087030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 msgr2=0x55b2fb3d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7400700 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7400700 1 -- 172.21.15.42:0/2227087030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 msgr2=0x55b2fb3d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7400700 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7400700 1 -- 172.21.15.42:0/2227087030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2fb3c7cc0 con 0x55b2fb3d0db0 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.048+0000 7fe4b7400700 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3dca30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40e0130 tx=0x7fe4a400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c64da5071094da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.049+0000 7fe4acc00700 1 -- 172.21.15.42:0/2227087030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40ada70 con 0x55b2fb3d0db0 2024-05-27T10:21:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.049+0000 7fe4acc00700 1 -- 172.21.15.42:0/2227087030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a4010610 con 0x55b2fb3d0db0 2024-05-27T10:21:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.049+0000 7fe4acc00700 1 -- 172.21.15.42:0/2227087030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40a9b80 con 0x55b2fb3d0db0 2024-05-27T10:21:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2227087030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 msgr2=0x55b2fb3dca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3dca30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40e0130 tx=0x7fe4a400bf90 comp rx=0 tx=0).stop 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2227087030 shutdown_connections 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb251e40 0x55b2fb3d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/2227087030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3dca30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.050+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2227087030 >> 172.21.15.42:0/2227087030 conn(0x55b2fb2b1fc0 msgr2=0x55b2fb3d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.051+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2227087030 shutdown_connections 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.051+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/2227087030 wait complete. 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.051+0000 7fe4cd8b0b80 1 -- start start 2024-05-27T10:21:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 Processor -- start 2024-05-27T10:21:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 0x55b2fb3e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3dba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81ab850 con 0x55b2fb251e40 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81accc0 con 0x55b2fb3d0db0 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81abf20 con 0x55b2fb3d6d30 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.052+0000 7fe4cd8b0b80 10 client.0 _ll_register_callbacks cb 0x55b2fb298c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 0x55b2fb3e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 0x55b2fb3e2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49544/0 (socket says 172.21.15.42:49544) 2024-05-27T10:21:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 -- 172.21.15.42:0/975356135 learned_addr learned my addr 172.21.15.42:0/975356135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4bce00700 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 -- 172.21.15.42:0/975356135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 msgr2=0x55b2fb3dba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3dba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 -- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 msgr2=0x55b2fb3e3190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3e3190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.053+0000 7fe4b7e00700 1 -- 172.21.15.42:0/975356135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2fb3dc090 con 0x55b2fb251e40 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.054+0000 7fe4b7e00700 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 0x55b2fb3e2c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009990 tx=0x7fe4a80050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506f02cc0e868c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.054+0000 7fe4a2000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.054+0000 7fe4a2000700 1 -- 172.21.15.42:0/975356135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80056e0 con 0x55b2fb251e40 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.054+0000 7fe4a2000700 1 -- 172.21.15.42:0/975356135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe4a80037c0 con 0x55b2fb251e40 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.054+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4010610 con 0x55b2fb251e40 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.055+0000 7fe4a2000700 1 -- 172.21.15.42:0/975356135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4a8034050 con 0x55b2fb251e40 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.055+0000 7fe4cd8b0b80 10 client.15825 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.055+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2fb454b80 con 0x55b2fb251e40 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.055+0000 7fe4a2000700 1 -- 172.21.15.42:0/975356135 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a4010610 con 0x55b2fb251e40 2024-05-27T10:21:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4a2000700 1 -- 172.21.15.42:0/975356135 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe4a800f460 con 0x55b2fb251e40 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4cd8b0b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4cd8b0b80 10 client.15825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4cd8b0b80 20 client.15825 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb455340 0x55b2fb457760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4cd8b0b80 4 client.15825 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.056+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2fb2fa570 con 0x55b2fb455340 2024-05-27T10:21:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.057+0000 7fe4b7400700 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb455340 0x55b2fb457760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.058+0000 7fe4b7400700 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb455340 0x55b2fb457760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae25d6f5b3f28349 server_cookie=1ca8ed88ba6223a7 in_seq=0 out_seq=0 2024-05-27T10:21:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.058+0000 7fe4a2000700 10 client.15825 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.058+0000 7fe4a2000700 1 -- 172.21.15.42:0/975356135 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2fb2fa570 con 0x55b2fb455340 2024-05-27T10:21:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.058+0000 7fe4a2000700 10 client.15825 handle_command_reply: tid=0 2024-05-27T10:21:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.058+0000 7fe4cd8b0b80 1 client.15825 shutdown 2024-05-27T10:21:10.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 msgr2=0x55b2fb3e2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 0x55b2fb3e2c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009990 tx=0x7fe4a80050b0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 shutdown_connections 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x55b2fb455340 0x55b2fb457760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b2fb3d6d30 0x55b2fb3dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2fb251e40 0x55b2fb3e2c50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 --2- 172.21.15.42:0/975356135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2fb3d0db0 0x55b2fb3e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 >> 172.21.15.42:0/975356135 conn(0x55b2fb2b1fc0 msgr2=0x55b2fb3d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:10.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4a2000700 0 client.15825 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:10.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 shutdown_connections 2024-05-27T10:21:10.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.059+0000 7fe4cd8b0b80 1 -- 172.21.15.42:0/975356135 wait complete. 2024-05-27T10:21:10.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.060+0000 7fe4cd8b0b80 20 client.15825 trim_cache size 0 max 16384 2024-05-27T10:21:10.062 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.060+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 msgr2=0x7fe49c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.060+0000 7fe4bee00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00076f0 tx=0x7fe4b0009100 comp rx=0 tx=0).stop 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.060+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 msgr2=0x7fe4b819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.060+0000 7fe4bee00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b819d6b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400d980 tx=0x7fe4a400dc90 comp rx=0 tx=0).stop 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 shutdown_connections 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b81a1880 0x7fe4b81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b8103a40 0x7fe4b819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 --2- 172.21.15.42:0/3630728460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8110ca0 0x7fe4b819d6b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 >> 172.21.15.42:0/3630728460 conn(0x7fe4b80fedb0 msgr2=0x7fe4b8101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 shutdown_connections 2024-05-27T10:21:10.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:10.061+0000 7fe4bee00700 1 -- 172.21.15.42:0/3630728460 wait complete. 2024-05-27T10:21:11.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:10 smithi143 ceph-mon[35545]: pgmap v76: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:21:11.080 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:10 smithi042 ceph-mon[31661]: pgmap v76: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:21:11.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:10 smithi104 ceph-mon[34812]: pgmap v76: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:21:11.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T10:21:11.639 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:13.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:12 smithi143 ceph-mon[35545]: pgmap v77: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-27T10:21:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:12 smithi042 ceph-mon[31661]: pgmap v77: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-27T10:21:13.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:12 smithi104 ceph-mon[34812]: pgmap v77: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-27T10:21:14.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:14 smithi042 ceph-mon[31661]: pgmap v78: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-27T10:21:15.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:14 smithi104 ceph-mon[34812]: pgmap v78: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-27T10:21:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:14 smithi143 ceph-mon[35545]: pgmap v78: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-27T10:21:15.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 -- 172.21.15.42:0/3537364914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8108d20 msgr2=0x7f22e8109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 --2- 172.21.15.42:0/3537364914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8108d20 0x7f22e8109130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f22d40099c0 tx=0x7f22d4009cd0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 -- 172.21.15.42:0/3537364914 shutdown_connections 2024-05-27T10:21:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 --2- 172.21.15.42:0/3537364914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e8109670 0x7f22e8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 --2- 172.21.15.42:0/3537364914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e81043f0 0x7f22e8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 --2- 172.21.15.42:0/3537364914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8108d20 0x7f22e8109130 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 -- 172.21.15.42:0/3537364914 >> 172.21.15.42:0/3537364914 conn(0x7f22e8079d00 msgr2=0x7f22e807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.909+0000 7f22ee200700 1 -- 172.21.15.42:0/3537364914 shutdown_connections 2024-05-27T10:21:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.910+0000 7f22ee200700 1 -- 172.21.15.42:0/3537364914 wait complete. 2024-05-27T10:21:15.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 Processor -- start 2024-05-27T10:21:15.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 -- start start 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 0x7f22e81a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e8108d20 0x7f22e81a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8109670 0x7f22e819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8114180 con 0x7f22e8109670 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8114000 con 0x7f22e81043f0 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.912+0000 7f22ee200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8114300 con 0x7f22e8108d20 2024-05-27T10:21:15.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.913+0000 7f22ece00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 0x7f22e81a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.913+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8109670 0x7f22e819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.913+0000 7f22ece00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 0x7f22e81a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:53310/0 (socket says 172.21.15.42:53310) 2024-05-27T10:21:15.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.913+0000 7f22ece00700 1 -- 172.21.15.42:0/2622388704 learned_addr learned my addr 172.21.15.42:0/2622388704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:15.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.913+0000 7f22e7e00700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e8108d20 0x7f22e81a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.914+0000 7f22ece00700 1 -- 172.21.15.42:0/2622388704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e8108d20 msgr2=0x7f22e81a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.914+0000 7f22ece00700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e8108d20 0x7f22e81a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.914+0000 7f22ece00700 1 -- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8109670 msgr2=0x7f22e819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.914+0000 7f22ece00700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8109670 0x7f22e819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.914+0000 7f22ece00700 1 -- 172.21.15.42:0/2622388704 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e819b0c0 con 0x7f22e81043f0 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.914+0000 7f22ece00700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 0x7f22e81a0860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c9b0 tx=0x7f22dc00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.915+0000 7f22e5600700 1 -- 172.21.15.42:0/2622388704 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007b00 con 0x7f22e81043f0 2024-05-27T10:21:15.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.915+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e819b3b0 con 0x7f22e81043f0 2024-05-27T10:21:15.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.915+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e80759e0 con 0x7f22e81043f0 2024-05-27T10:21:15.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.915+0000 7f22e5600700 1 -- 172.21.15.42:0/2622388704 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22dc00ce50 con 0x7f22e81043f0 2024-05-27T10:21:15.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.915+0000 7f22e5600700 1 -- 172.21.15.42:0/2622388704 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc018660 con 0x7f22e81043f0 2024-05-27T10:21:15.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.920+0000 7f22fe38cb80 1 Processor -- start 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.920+0000 7f22fe38cb80 1 -- start start 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6af12c0 0x55f1b6af36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f1b6a14e40 con 0x55f1b6a539f0 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f1b6a049e0 con 0x55f1b6af12c0 2024-05-27T10:21:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f1b6a08a20 con 0x55f1b6967e40 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40012/0 (socket says 172.21.15.42:40012) 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.921+0000 7f22ed800700 1 -- 172.21.15.42:0/3729846721 learned_addr learned my addr 172.21.15.42:0/3729846721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.922+0000 7f22ed800700 1 -- 172.21.15.42:0/3729846721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 msgr2=0x55f1b6ae7c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.922+0000 7f22ed800700 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae7c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.922+0000 7f22ed800700 1 -- 172.21.15.42:0/3729846721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6af12c0 msgr2=0x55f1b6af36e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.922+0000 7f22ed800700 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6af12c0 0x55f1b6af36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.922+0000 7f22ed800700 1 -- 172.21.15.42:0/3729846721 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b69cd420 con 0x55f1b6a539f0 2024-05-27T10:21:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.922+0000 7f22ed800700 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae7750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22e0003fa0 tx=0x7f22e0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e4883c6dc8c063 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:15.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.923+0000 7f22e5600700 1 -- 172.21.15.42:0/2622388704 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f22dc00f450 con 0x7f22e81043f0 2024-05-27T10:21:15.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.923+0000 7f22da000700 1 -- 172.21.15.42:0/3729846721 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0004bb0 con 0x55f1b6a539f0 2024-05-27T10:21:15.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.923+0000 7f22da000700 1 -- 172.21.15.42:0/3729846721 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e0004d50 con 0x55f1b6a539f0 2024-05-27T10:21:15.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.923+0000 7f22da000700 1 -- 172.21.15.42:0/3729846721 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0007d00 con 0x55f1b6a539f0 2024-05-27T10:21:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.924+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3729846721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 msgr2=0x55f1b6ae7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.924+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae7750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22e0003fa0 tx=0x7f22e0009ba0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.925+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3729846721 shutdown_connections 2024-05-27T10:21:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.925+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6af12c0 0x55f1b6af36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.925+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae7750 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.925+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3729846721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.925+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3729846721 >> 172.21.15.42:0/3729846721 conn(0x55f1b6a01390 msgr2=0x55f1b6ae2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.926+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3729846721 shutdown_connections 2024-05-27T10:21:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.926+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3729846721 wait complete. 2024-05-27T10:21:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.927+0000 7f22fe38cb80 1 -- start start 2024-05-27T10:21:15.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.927+0000 7f22fe38cb80 1 Processor -- start 2024-05-27T10:21:15.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22e5600700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f22d00b2a00 0x7f22d00b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 0x55f1b6aee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6a539f0 0x55f1b6aee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6aeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f1b690afa0 con 0x55f1b6af12c0 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f1b69a5760 con 0x55f1b6967e40 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f1b6af3e00 con 0x55f1b6a539f0 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.928+0000 7f22fe38cb80 10 client.0 _ll_register_callbacks cb 0x55f1b69aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.929+0000 7f22ece00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 0x55f1b6aee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.929+0000 7f22e5600700 1 -- 172.21.15.42:0/2622388704 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f22dc0e24a0 con 0x7f22e81043f0 2024-05-27T10:21:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.929+0000 7f22ece00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 0x55f1b6aee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:53322/0 (socket says 172.21.15.42:53322) 2024-05-27T10:21:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.929+0000 7f22ece00700 1 -- 172.21.15.42:0/102294657 learned_addr learned my addr 172.21.15.42:0/102294657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.930+0000 7f22ece00700 1 -- 172.21.15.42:0/102294657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6a539f0 msgr2=0x55f1b6aee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.930+0000 7f22ece00700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6a539f0 0x55f1b6aee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.930+0000 7f22ece00700 1 -- 172.21.15.42:0/102294657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 msgr2=0x55f1b6aeece0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:21:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.930+0000 7f22ed800700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6aeece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.930+0000 7f22ece00700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6aeece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.930+0000 7f22ece00700 1 -- 172.21.15.42:0/102294657 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b69548d0 con 0x55f1b6967e40 2024-05-27T10:21:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22ed800700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6aeece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22ed800700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f22d00b2a00 0x7f22d00b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22ece00700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 0x55f1b6aee260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0a9470 tx=0x7f22dc0148a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb0c900b7f97199 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22cea00700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:21:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22cea00700 1 -- 172.21.15.42:0/102294657 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a9770 con 0x55f1b6967e40 2024-05-27T10:21:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22cea00700 1 -- 172.21.15.42:0/102294657 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f22dc0cd400 con 0x55f1b6967e40 2024-05-27T10:21:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.931+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0004d50 con 0x55f1b6967e40 2024-05-27T10:21:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.932+0000 7f22cea00700 1 -- 172.21.15.42:0/102294657 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22dc0adb20 con 0x55f1b6967e40 2024-05-27T10:21:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.932+0000 7f22fe38cb80 10 client.35468 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.932+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1b69d5060 con 0x55f1b6967e40 2024-05-27T10:21:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.932+0000 7f22cea00700 1 -- 172.21.15.42:0/102294657 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e0004d50 con 0x55f1b6967e40 2024-05-27T10:21:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22cea00700 1 -- 172.21.15.42:0/102294657 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f22dc007ca0 con 0x55f1b6967e40 2024-05-27T10:21:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22fe38cb80 10 client.35468 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22fe38cb80 10 client.35468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22fe38cb80 20 client.35468 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b00d80 0x55f1b6b03200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22fe38cb80 4 client.35468 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.933+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b6a16d90 con 0x55f1b6b00d80 2024-05-27T10:21:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.934+0000 7f22ed800700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f22d00b2a00 0x7f22d00b4f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22e0003ba0 tx=0x7f22e0003ad0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.934+0000 7f22e7e00700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b00d80 0x55f1b6b03200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.935+0000 7f22cea00700 10 client.35468 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:15.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.935+0000 7f22e7e00700 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b00d80 0x55f1b6b03200 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4f302ae08aac8a server_cookie=c1cae0ef55c2a95b in_seq=0 out_seq=0 2024-05-27T10:21:15.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.937+0000 7f22cea00700 1 -- 172.21.15.42:0/102294657 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f1b6a16d90 con 0x55f1b6b00d80 2024-05-27T10:21:15.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.937+0000 7f22cea00700 10 client.35468 handle_command_reply: tid=0 2024-05-27T10:21:15.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.937+0000 7f22fe38cb80 1 client.35468 shutdown 2024-05-27T10:21:15.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.939+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 msgr2=0x55f1b6aee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.939+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 0x55f1b6aee260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0a9470 tx=0x7f22dc0148a0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 shutdown_connections 2024-05-27T10:21:15.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b00d80 0x55f1b6b03200 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6aeece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6967e40 0x55f1b6aee260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/102294657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6a539f0 0x55f1b6aee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 >> 172.21.15.42:0/102294657 conn(0x55f1b6a01390 msgr2=0x55f1b6ae2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22cea00700 0 client.35468 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 shutdown_connections 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 1 -- 172.21.15.42:0/102294657 wait complete. 2024-05-27T10:21:15.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.940+0000 7f22fe38cb80 20 client.35468 trim_cache size 0 max 16384 2024-05-27T10:21:15.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 Processor -- start 2024-05-27T10:21:15.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 -- start start 2024-05-27T10:21:15.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6b08a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800fd00 con 0x55f1b6af12c0 2024-05-27T10:21:15.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800d590 con 0x55f1b6a539f0 2024-05-27T10:21:15.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.957+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800b3e0 con 0x55f1b6967e40 2024-05-27T10:21:15.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.958+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6b08a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.958+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6b08a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40038/0 (socket says 172.21.15.42:40038) 2024-05-27T10:21:15.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.958+0000 7f22ed800700 1 -- 172.21.15.42:0/3758321379 learned_addr learned my addr 172.21.15.42:0/3758321379 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:15.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.958+0000 7f22ece00700 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.958+0000 7f22e7e00700 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.959+0000 7f22ece00700 1 -- 172.21.15.42:0/3758321379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 msgr2=0x55f1b6ae97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.959+0000 7f22ece00700 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.959+0000 7f22ece00700 1 -- 172.21.15.42:0/3758321379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 msgr2=0x55f1b6b08a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.959+0000 7f22ece00700 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6b08a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.959+0000 7f22ece00700 1 -- 172.21.15.42:0/3758321379 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b6aed580 con 0x55f1b6967e40 2024-05-27T10:21:15.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.959+0000 7f22ece00700 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae2740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00afd0 tx=0x7f22dc0a91f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08fbc82d706ed4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:15.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.960+0000 7f22d9600700 1 -- 172.21.15.42:0/3758321379 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0cc0b0 con 0x55f1b6967e40 2024-05-27T10:21:15.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.960+0000 7f22d9600700 1 -- 172.21.15.42:0/3758321379 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22dc0a9770 con 0x55f1b6967e40 2024-05-27T10:21:15.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.960+0000 7f22d9600700 1 -- 172.21.15.42:0/3758321379 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00f740 con 0x55f1b6967e40 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3758321379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 msgr2=0x55f1b6ae2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae2740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00afd0 tx=0x7f22dc0a91f0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3758321379 shutdown_connections 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6b08a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6ae2740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/3758321379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6ae97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.961+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3758321379 >> 172.21.15.42:0/3758321379 conn(0x55f1b6a01390 msgr2=0x55f1b6ae90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:15.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.962+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3758321379 shutdown_connections 2024-05-27T10:21:15.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.962+0000 7f22fe38cb80 1 -- 172.21.15.42:0/3758321379 wait complete. 2024-05-27T10:21:15.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.962+0000 7f22fe38cb80 1 -- start start 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 Processor -- start 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6af40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6af45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6af4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800ba50 con 0x55f1b6af12c0 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800cec0 con 0x55f1b6a539f0 2024-05-27T10:21:15.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.963+0000 7f22fe38cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800c120 con 0x55f1b6967e40 2024-05-27T10:21:15.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.964+0000 7f22fe38cb80 10 client.0 _ll_register_callbacks cb 0x55f1b69aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:15.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.964+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6af4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.964+0000 7f22ed800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6af4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40044/0 (socket says 172.21.15.42:40044) 2024-05-27T10:21:15.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.964+0000 7f22ed800700 1 -- 172.21.15.42:0/2155466644 learned_addr learned my addr 172.21.15.42:0/2155466644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:15.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.964+0000 7f22ece00700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6af40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.964+0000 7f22e7e00700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6af45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.965+0000 7f22ed800700 1 -- 172.21.15.42:0/2155466644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 msgr2=0x55f1b6af40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.965+0000 7f22ed800700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6af40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.965+0000 7f22ed800700 1 -- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 msgr2=0x55f1b6af45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.965+0000 7f22ed800700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6af45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.965+0000 7f22ed800700 1 -- 172.21.15.42:0/2155466644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b6b699f0 con 0x55f1b6af12c0 2024-05-27T10:21:15.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.966+0000 7f22ed800700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6af4b30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f22e0007720 tx=0x7f22e0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391d54f784f5f3ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:15.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.966+0000 7f22cea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:15.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.966+0000 7f22cea00700 1 -- 172.21.15.42:0/2155466644 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0028040 con 0x55f1b6af12c0 2024-05-27T10:21:15.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.966+0000 7f22cea00700 1 -- 172.21.15.42:0/2155466644 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f22e002e690 con 0x55f1b6af12c0 2024-05-27T10:21:15.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.967+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22dc0a9770 con 0x55f1b6af12c0 2024-05-27T10:21:15.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.967+0000 7f22cea00700 1 -- 172.21.15.42:0/2155466644 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22e003c050 con 0x55f1b6af12c0 2024-05-27T10:21:15.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.967+0000 7f22fe38cb80 10 client.15840 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:15.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.967+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1b6b73460 con 0x55f1b6af12c0 2024-05-27T10:21:15.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22cea00700 1 -- 172.21.15.42:0/2155466644 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22dc0a9770 con 0x55f1b6af12c0 2024-05-27T10:21:15.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22cea00700 1 -- 172.21.15.42:0/2155466644 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f22e0015990 con 0x55f1b6af12c0 2024-05-27T10:21:15.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22fe38cb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:15.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22fe38cb80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:15.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22fe38cb80 20 client.15840 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:15.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b73910 0x55f1b6b75d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:15.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22fe38cb80 4 client.15840 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:21:15.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.968+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1b6a10570 con 0x55f1b6b73910 2024-05-27T10:21:15.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.969+0000 7f22ece00700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b73910 0x55f1b6b75d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:15.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22ece00700 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b73910 0x55f1b6b75d30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb1b6b2f9ddc116 server_cookie=34f6dd49a2994d5f in_seq=0 out_seq=0 2024-05-27T10:21:15.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22cea00700 10 client.15840 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:15.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22cea00700 1 -- 172.21.15.42:0/2155466644 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1b6a10570 con 0x55f1b6b73910 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22cea00700 10 client.15840 handle_command_reply: tid=0 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 client.15840 shutdown 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 msgr2=0x55f1b6af4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6af4b30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f22e0007720 tx=0x7f22e0009540 comp rx=0 tx=0).stop 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 shutdown_connections 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f1b6b73910 0x55f1b6b75d30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1b6af12c0 0x55f1b6af4b30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f1b6967e40 0x55f1b6af40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 --2- 172.21.15.42:0/2155466644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1b6a539f0 0x55f1b6af45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.970+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 >> 172.21.15.42:0/2155466644 conn(0x55f1b6a01390 msgr2=0x55f1b6ae6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:15.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.971+0000 7f22cea00700 0 client.15840 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:15.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.971+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 shutdown_connections 2024-05-27T10:21:15.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.971+0000 7f22fe38cb80 1 -- 172.21.15.42:0/2155466644 wait complete. 2024-05-27T10:21:15.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.971+0000 7f22fe38cb80 20 client.15840 trim_cache size 0 max 16384 2024-05-27T10:21:15.973 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.972+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f22d00b2a00 msgr2=0x7f22d00b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.972+0000 7f22ee200700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f22d00b2a00 0x7f22d00b4f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22e0003ba0 tx=0x7f22e0003ad0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.972+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 msgr2=0x7f22e81a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.972+0000 7f22ee200700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 0x7f22e81a0860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c9b0 tx=0x7f22dc00ccc0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 shutdown_connections 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f22d00b2a00 0x7f22d00b4f10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22e8109670 0x7f22e819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22e81043f0 0x7f22e81a0860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 --2- 172.21.15.42:0/2622388704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e8108d20 0x7f22e81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:15.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 >> 172.21.15.42:0/2622388704 conn(0x7f22e8079d00 msgr2=0x7f22e8101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:15.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 shutdown_connections 2024-05-27T10:21:15.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:15.973+0000 7f22ee200700 1 -- 172.21.15.42:0/2622388704 wait complete. 2024-05-27T10:21:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:16 smithi042 ceph-mon[31661]: pgmap v79: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:21:17.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:16 smithi104 ceph-mon[34812]: pgmap v79: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:21:17.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:16 smithi143 ceph-mon[35545]: pgmap v79: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-27T10:21:18.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T10:21:18.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T10:21:18.729 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 -- 172.21.15.42:0/473152193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810e000 msgr2=0x7f91381104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 --2- 172.21.15.42:0/473152193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810e000 0x7f91381104f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9130009860 tx=0x7f913002a340 comp rx=0 tx=0).stop 2024-05-27T10:21:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 -- 172.21.15.42:0/473152193 shutdown_connections 2024-05-27T10:21:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 --2- 172.21.15.42:0/473152193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810e000 0x7f91381104f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 --2- 172.21.15.42:0/473152193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f913810b610 0x7f913810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 --2- 172.21.15.42:0/473152193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9138104bb0 0x7f9138104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.849+0000 7f913e000640 1 -- 172.21.15.42:0/473152193 >> 172.21.15.42:0/473152193 conn(0x7f91380fe210 msgr2=0x7f9138100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.850+0000 7f913e000640 1 -- 172.21.15.42:0/473152193 shutdown_connections 2024-05-27T10:21:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.850+0000 7f913e000640 1 -- 172.21.15.42:0/473152193 wait complete. 2024-05-27T10:21:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.851+0000 7f913e000640 1 Processor -- start 2024-05-27T10:21:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 -- start start 2024-05-27T10:21:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9138104bb0 0x7f9138104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 0x7f9138102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f913810e000 0x7f9138102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9138113b10 con 0x7f913810b610 2024-05-27T10:21:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9138113990 con 0x7f913810e000 2024-05-27T10:21:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.852+0000 7f913e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9138113c90 con 0x7f9138104bb0 2024-05-27T10:21:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.853+0000 7f912fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 0x7f9138102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.853+0000 7f913cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9138104bb0 0x7f9138104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.853+0000 7f912fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 0x7f9138102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40070/0 (socket says 172.21.15.42:40070) 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.853+0000 7f912fe00640 1 -- 172.21.15.42:0/2239465991 learned_addr learned my addr 172.21.15.42:0/2239465991 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.853+0000 7f913d600640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f913810e000 0x7f9138102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.854+0000 7f912fe00640 1 -- 172.21.15.42:0/2239465991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9138104bb0 msgr2=0x7f9138104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.854+0000 7f912fe00640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9138104bb0 0x7f9138104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.854+0000 7f912fe00640 1 -- 172.21.15.42:0/2239465991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f913810e000 msgr2=0x7f9138102a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.854+0000 7f912fe00640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f913810e000 0x7f9138102a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.854+0000 7f912fe00640 1 -- 172.21.15.42:0/2239465991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9138103260 con 0x7f913810b610 2024-05-27T10:21:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.855+0000 7f912fe00640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 0x7f9138102470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f912800efb0 tx=0x7f912800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.855+0000 7f9136000640 1 -- 172.21.15.42:0/2239465991 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912801b070 con 0x7f913810b610 2024-05-27T10:21:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.855+0000 7f9136000640 1 -- 172.21.15.42:0/2239465991 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f912800ba00 con 0x7f913810b610 2024-05-27T10:21:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.855+0000 7f9136000640 1 -- 172.21.15.42:0/2239465991 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128016ac0 con 0x7f913810b610 2024-05-27T10:21:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.856+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91381acfb0 con 0x7f913810b610 2024-05-27T10:21:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.856+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91381ad420 con 0x7f913810b610 2024-05-27T10:21:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.861+0000 7f9146773740 1 Processor -- start 2024-05-27T10:21:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.861+0000 7f9146773740 1 -- start start 2024-05-27T10:21:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.862+0000 7f9146773740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e72aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.862+0000 7f9146773740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e72afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.862+0000 7f9146773740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.862+0000 7f9146773740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55724e69bb90 con 0x55724e3674d0 2024-05-27T10:21:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.862+0000 7f9146773740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55724e68b730 con 0x55724e3881b0 2024-05-27T10:21:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.862+0000 7f9146773740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55724e68f770 con 0x55724e35f2f0 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f9136000640 1 -- 172.21.15.42:0/2239465991 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f912801e070 con 0x7f913810b610 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e72aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e72aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:53364/0 (socket says 172.21.15.42:53364) 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 -- 172.21.15.42:0/1857833231 learned_addr learned my addr 172.21.15.42:0/1857833231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913cc00640 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e72afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 -- 172.21.15.42:0/1857833231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 msgr2=0x55724e72afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e72afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 -- 172.21.15.42:0/1857833231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 msgr2=0x55724e72b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.863+0000 7f913d600640 1 -- 172.21.15.42:0/1857833231 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724e6248b0 con 0x55724e3881b0 2024-05-27T10:21:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.864+0000 7f913d600640 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e72aa60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9130009860 tx=0x7f9130009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c8c7bf5642ccee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.864+0000 7f912dc00640 1 -- 172.21.15.42:0/1857833231 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130043070 con 0x55724e3881b0 2024-05-27T10:21:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.865+0000 7f912dc00640 1 -- 172.21.15.42:0/1857833231 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91300044d0 con 0x55724e3881b0 2024-05-27T10:21:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.865+0000 7f912dc00640 1 -- 172.21.15.42:0/1857833231 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130037040 con 0x55724e3881b0 2024-05-27T10:21:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.865+0000 7f9136000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f911c0b2aa0 0x7f911c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.865+0000 7f9136000640 1 -- 172.21.15.42:0/2239465991 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f91280e23d0 con 0x7f913810b610 2024-05-27T10:21:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.866+0000 7f913cc00640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f911c0b2aa0 0x7f911c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.866+0000 7f9146773740 1 -- 172.21.15.42:0/1857833231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 msgr2=0x55724e72aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.866+0000 7f9146773740 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e72aa60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9130009860 tx=0x7f9130009490 comp rx=0 tx=0).stop 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.866+0000 7f913cc00640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f911c0b2aa0 0x7f911c0b4fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55724e72be10 tx=0x7f9124005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 -- 172.21.15.42:0/1857833231 shutdown_connections 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e72afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 --2- 172.21.15.42:0/1857833231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e72aa60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 -- 172.21.15.42:0/1857833231 >> 172.21.15.42:0/1857833231 conn(0x55724e37c300 msgr2=0x55724e727210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 -- 172.21.15.42:0/1857833231 shutdown_connections 2024-05-27T10:21:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 -- 172.21.15.42:0/1857833231 wait complete. 2024-05-27T10:21:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 -- start start 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.867+0000 7f9146773740 1 Processor -- start 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 0x55724e734c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3674d0 0x55724e7351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e3881b0 0x55724e740e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55724e371790 con 0x55724e35f2f0 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55724e633bc0 con 0x55724e3674d0 2024-05-27T10:21:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55724e737b30 con 0x55724e3881b0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f9146773740 10 client.0 _ll_register_callbacks cb 0x55724e611400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.868+0000 7f913cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 0x55724e734c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 0x55724e734c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40090/0 (socket says 172.21.15.42:40090) 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 -- 172.21.15.42:0/1274860329 learned_addr learned my addr 172.21.15.42:0/1274860329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913d600640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e3881b0 0x55724e740e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f912fe00640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3674d0 0x55724e7351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 -- 172.21.15.42:0/1274860329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e3881b0 msgr2=0x55724e740e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e3881b0 0x55724e740e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 -- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3674d0 msgr2=0x55724e7351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3674d0 0x55724e7351c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.869+0000 7f913cc00640 1 -- 172.21.15.42:0/1274860329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724e3a0210 con 0x55724e35f2f0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.870+0000 7f913cc00640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 0x55724e734c80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9124004650 tx=0x7f9124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c7958c033b38ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.870+0000 7f9122000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.870+0000 7f9122000640 1 -- 172.21.15.42:0/1274860329 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124009db0 con 0x55724e35f2f0 2024-05-27T10:21:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.870+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91300044d0 con 0x55724e35f2f0 2024-05-27T10:21:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.870+0000 7f9122000640 1 -- 172.21.15.42:0/1274860329 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9124018030 con 0x55724e35f2f0 2024-05-27T10:21:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9122000640 1 -- 172.21.15.42:0/1274860329 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91240047a0 con 0x55724e35f2f0 2024-05-27T10:21:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9146773740 10 client.15849 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55724e46c620 con 0x55724e35f2f0 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9122000640 1 -- 172.21.15.42:0/1274860329 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91300044d0 con 0x55724e35f2f0 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9122000640 1 -- 172.21.15.42:0/1274860329 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f912400f450 con 0x55724e35f2f0 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9146773740 10 client.15849 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9146773740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.871+0000 7f9146773740 20 client.15849 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.872+0000 7f9146773740 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e346d80 0x55724e746f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.872+0000 7f9146773740 4 client.15849 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.872+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55724e69dae0 con 0x55724e346d80 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.872+0000 7f912fe00640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e346d80 0x55724e746f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.873+0000 7f912fe00640 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e346d80 0x55724e746f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d113806f6a78f8e server_cookie=df06d94d7930ee29 in_seq=0 out_seq=0 2024-05-27T10:21:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.873+0000 7f9122000640 10 client.15849 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.875+0000 7f9122000640 1 -- 172.21.15.42:0/1274860329 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55724e69dae0 con 0x55724e346d80 2024-05-27T10:21:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.875+0000 7f9122000640 10 client.15849 handle_command_reply: tid=0 2024-05-27T10:21:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.875+0000 7f9146773740 1 client.15849 shutdown 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 msgr2=0x55724e734c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 0x55724e734c80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9124004650 tx=0x7f9124004730 comp rx=0 tx=0).stop 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 shutdown_connections 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e346d80 0x55724e746f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e3881b0 0x55724e740e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3674d0 0x55724e7351c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 --2- 172.21.15.42:0/1274860329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e35f2f0 0x55724e734c80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9122000640 0 client.15849 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 >> 172.21.15.42:0/1274860329 conn(0x55724e37c300 msgr2=0x55724e72c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 shutdown_connections 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 1 -- 172.21.15.42:0/1274860329 wait complete. 2024-05-27T10:21:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.876+0000 7f9146773740 20 client.15849 trim_cache size 0 max 16384 2024-05-27T10:21:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.893+0000 7f9146773740 1 Processor -- start 2024-05-27T10:21:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.893+0000 7f9146773740 1 -- start start 2024-05-27T10:21:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f9146773740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e7373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f9146773740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f9146773740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e740c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f9146773740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b8540 con 0x55724e3674d0 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f9146773740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b5e10 con 0x55724e3881b0 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f9146773740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b3c60 con 0x55724e35f2f0 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f912fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f912fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40102/0 (socket says 172.21.15.42:40102) 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f912fe00640 1 -- 172.21.15.42:0/1659562539 learned_addr learned my addr 172.21.15.42:0/1659562539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.894+0000 7f913cc00640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e7373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f913d600640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e740c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f912fe00640 1 -- 172.21.15.42:0/1659562539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 msgr2=0x55724e7373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f912fe00640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e7373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f912fe00640 1 -- 172.21.15.42:0/1659562539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 msgr2=0x55724e740c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f912fe00640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e740c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f912fe00640 1 -- 172.21.15.42:0/1659562539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724e72be40 con 0x55724e3674d0 2024-05-27T10:21:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f913d600640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e740c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f913cc00640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e7373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.895+0000 7f912fe00640 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72a900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f912800bfd0 tx=0x7f91280cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c7553f023f3ec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.896+0000 7f912d200640 1 -- 172.21.15.42:0/1659562539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91280c3070 con 0x55724e3674d0 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.896+0000 7f912d200640 1 -- 172.21.15.42:0/1659562539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91280ada50 con 0x55724e3674d0 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.896+0000 7f912d200640 1 -- 172.21.15.42:0/1659562539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91280a89b0 con 0x55724e3674d0 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 -- 172.21.15.42:0/1659562539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 msgr2=0x55724e72a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72a900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f912800bfd0 tx=0x7f91280cc040 comp rx=0 tx=0).stop 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 -- 172.21.15.42:0/1659562539 shutdown_connections 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e740c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e3674d0 0x55724e72a900 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 --2- 172.21.15.42:0/1659562539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e7373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 -- 172.21.15.42:0/1659562539 >> 172.21.15.42:0/1659562539 conn(0x55724e37c300 msgr2=0x55724e734f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 -- 172.21.15.42:0/1659562539 shutdown_connections 2024-05-27T10:21:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.897+0000 7f9146773740 1 -- 172.21.15.42:0/1659562539 wait complete. 2024-05-27T10:21:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.898+0000 7f9146773740 1 -- start start 2024-05-27T10:21:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.898+0000 7f9146773740 1 Processor -- start 2024-05-27T10:21:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e73cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e73d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 0x55724e7383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b42d0 con 0x55724e31c1c0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b5740 con 0x55724e3881b0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f91381b49a0 con 0x55724e35f2f0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f9146773740 10 client.0 _ll_register_callbacks cb 0x7f9124009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f913d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 0x55724e7383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f913d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 0x55724e7383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40110/0 (socket says 172.21.15.42:40110) 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f913d600640 1 -- 172.21.15.42:0/393965535 learned_addr learned my addr 172.21.15.42:0/393965535 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.899+0000 7f913cc00640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e73cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f912fe00640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e73d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913d600640 1 -- 172.21.15.42:0/393965535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 msgr2=0x55724e73cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913d600640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e73cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913d600640 1 -- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 msgr2=0x55724e73d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913d600640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e73d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913d600640 1 -- 172.21.15.42:0/393965535 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724e738a70 con 0x55724e31c1c0 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913cc00640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e73cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f912fe00640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e73d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f913d600640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 0x55724e7383b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55724e727bd0 tx=0x7f9130005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85dc36ba61e400f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f9122000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.900+0000 7f9122000640 1 -- 172.21.15.42:0/393965535 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130037040 con 0x55724e31c1c0 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.901+0000 7f9122000640 1 -- 172.21.15.42:0/393965535 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9130030550 con 0x55724e31c1c0 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.901+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91280ada50 con 0x55724e31c1c0 2024-05-27T10:21:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.901+0000 7f9122000640 1 -- 172.21.15.42:0/393965535 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9130043040 con 0x55724e31c1c0 2024-05-27T10:21:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.901+0000 7f9146773740 10 client.15855 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.901+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55724e7aef90 con 0x55724e31c1c0 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.901+0000 7f9122000640 1 -- 172.21.15.42:0/393965535 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91280ada50 con 0x55724e31c1c0 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9122000640 1 -- 172.21.15.42:0/393965535 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9130003c10 con 0x55724e31c1c0 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9146773740 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9146773740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9146773740 20 client.15855 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9146773740 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e7af2d0 0x55724e7b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9146773740 4 client.15855 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55724e6972c0 con 0x55724e7af2d0 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.902+0000 7f913cc00640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e7af2d0 0x55724e7b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.903+0000 7f9122000640 10 client.15855 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.903+0000 7f913cc00640 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e7af2d0 0x55724e7b16f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50243221b052866 server_cookie=d95050dcea9be648 in_seq=0 out_seq=0 2024-05-27T10:21:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.913+0000 7f9122000640 1 -- 172.21.15.42:0/393965535 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55724e6972c0 con 0x55724e7af2d0 2024-05-27T10:21:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.913+0000 7f9122000640 10 client.15855 handle_command_reply: tid=0 2024-05-27T10:21:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.913+0000 7f9146773740 1 client.15855 shutdown 2024-05-27T10:21:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 msgr2=0x55724e7383b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 0x55724e7383b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55724e727bd0 tx=0x7f9130005dc0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 shutdown_connections 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55724e31c1c0 0x55724e7383b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55724e3881b0 0x55724e73d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55724e7af2d0 0x55724e7b16f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 --2- 172.21.15.42:0/393965535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55724e35f2f0 0x55724e73cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 >> 172.21.15.42:0/393965535 conn(0x55724e37c300 msgr2=0x55724e734ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9122000640 0 client.15855 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 shutdown_connections 2024-05-27T10:21:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.914+0000 7f9146773740 1 -- 172.21.15.42:0/393965535 wait complete. 2024-05-27T10:21:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.915+0000 7f9146773740 20 client.15855 trim_cache size 0 max 16384 2024-05-27T10:21:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:21:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:21:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:21:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f911c0b2aa0 msgr2=0x7f911c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f911c0b2aa0 0x7f911c0b4fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55724e72be10 tx=0x7f9124005f90 comp rx=0 tx=0).stop 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 msgr2=0x7f9138102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 0x7f9138102470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f912800efb0 tx=0x7f912800c800 comp rx=0 tx=0).stop 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 shutdown_connections 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f911c0b2aa0 0x7f911c0b4fb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f913810e000 0x7f9138102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f913810b610 0x7f9138102470 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 --2- 172.21.15.42:0/2239465991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9138104bb0 0x7f9138104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 >> 172.21.15.42:0/2239465991 conn(0x7f91380fe210 msgr2=0x7f913810e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 shutdown_connections 2024-05-27T10:21:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:18.916+0000 7f913e000640 1 -- 172.21.15.42:0/2239465991 wait complete. 2024-05-27T10:21:19.077 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:18 smithi042 ceph-mon[31661]: pgmap v80: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-27T10:21:19.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:18 smithi104 ceph-mon[34812]: pgmap v80: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-27T10:21:19.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:18 smithi143 ceph-mon[35545]: pgmap v80: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-27T10:21:20.988 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:20 smithi042 ceph-mon[31661]: pgmap v81: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T10:21:21.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:20 smithi104 ceph-mon[34812]: pgmap v81: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T10:21:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:20 smithi143 ceph-mon[35545]: pgmap v81: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 -- 172.21.15.42:0/3790513052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df810e6f0 msgr2=0x7f4df8110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 --2- 172.21.15.42:0/3790513052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df810e6f0 0x7f4df8110bd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4df400b0a0 tx=0x7f4df400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 -- 172.21.15.42:0/3790513052 shutdown_connections 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 --2- 172.21.15.42:0/3790513052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df810e6f0 0x7f4df8110bd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 --2- 172.21.15.42:0/3790513052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df8105280 0x7f4df8105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 --2- 172.21.15.42:0/3790513052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8105bd0 0x7f4df810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 -- 172.21.15.42:0/3790513052 >> 172.21.15.42:0/3790513052 conn(0x7f4df8078ab0 msgr2=0x7f4df8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 -- 172.21.15.42:0/3790513052 shutdown_connections 2024-05-27T10:21:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.309+0000 7f4dff200700 1 -- 172.21.15.42:0/3790513052 wait complete. 2024-05-27T10:21:21.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.310+0000 7f4dff200700 1 Processor -- start 2024-05-27T10:21:21.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.310+0000 7f4dff200700 1 -- start start 2024-05-27T10:21:21.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dff200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 0x7f4df81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dff200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df8105bd0 0x7f4df8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dff200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df810e6f0 0x7f4df81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dff200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8114150 con 0x7f4df8105280 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dff200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8113fd0 con 0x7f4df810e6f0 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dff200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81142d0 con 0x7f4df8105bd0 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 0x7f4df81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 0x7f4df81997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40120/0 (socket says 172.21.15.42:40120) 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.311+0000 7f4dfde00700 1 -- 172.21.15.42:0/1924351855 learned_addr learned my addr 172.21.15.42:0/1924351855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfd400700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df8105bd0 0x7f4df8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df810e6f0 0x7f4df81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfde00700 1 -- 172.21.15.42:0/1924351855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df8105bd0 msgr2=0x7f4df8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df8105bd0 0x7f4df8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfde00700 1 -- 172.21.15.42:0/1924351855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df810e6f0 msgr2=0x7f4df81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df810e6f0 0x7f4df81a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfde00700 1 -- 172.21.15.42:0/1924351855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df81a4be0 con 0x7f4df8105280 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.312+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 0x7f4df81997f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d950 tx=0x7f4dec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:21.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.313+0000 7f4df2a00700 1 -- 172.21.15.42:0/1924351855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec016070 con 0x7f4df8105280 2024-05-27T10:21:21.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.313+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81a4ed0 con 0x7f4df8105280 2024-05-27T10:21:21.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.313+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a5460 con 0x7f4df8105280 2024-05-27T10:21:21.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.313+0000 7f4df2a00700 1 -- 172.21.15.42:0/1924351855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dec010460 con 0x7f4df8105280 2024-05-27T10:21:21.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.313+0000 7f4df2a00700 1 -- 172.21.15.42:0/1924351855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec020060 con 0x7f4df8105280 2024-05-27T10:21:21.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.318+0000 7f4df2a00700 1 -- 172.21.15.42:0/1924351855 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4dec019070 con 0x7f4df8105280 2024-05-27T10:21:21.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.319+0000 7f4e0f394b80 1 Processor -- start 2024-05-27T10:21:21.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.319+0000 7f4df2a00700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4de00f6a80 0x7f4de00f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.319+0000 7f4e0f394b80 1 -- start start 2024-05-27T10:21:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4df2a00700 1 -- 172.21.15.42:0/1924351855 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4dec0e2180 con 0x7f4df8105280 2024-05-27T10:21:21.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 0x558ed8d20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d218a0 0x558ed8d2e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558ed8c4fec0 con 0x558ed8d21490 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558ed8c3fa60 con 0x558ed8ba2ec0 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558ed8c43aa0 con 0x558ed8d218a0 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.320+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4de00f6a80 0x7f4de00f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.321+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 0x558ed8d20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.321+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 0x558ed8d20f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:53412/0 (socket says 172.21.15.42:53412) 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.321+0000 7f4dfde00700 1 -- 172.21.15.42:0/955050894 learned_addr learned my addr 172.21.15.42:0/955050894 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.321+0000 7f4dfe800700 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d218a0 0x558ed8d2e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.321+0000 7f4dfd400700 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d2bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4de00f6a80 0x7f4de00f8f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4df40062a0 tx=0x7f4df40090f0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfde00700 1 -- 172.21.15.42:0/955050894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d218a0 msgr2=0x558ed8d2e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfde00700 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d218a0 0x558ed8d2e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfde00700 1 -- 172.21.15.42:0/955050894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 msgr2=0x558ed8d2bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfde00700 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d2bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfde00700 1 -- 172.21.15.42:0/955050894 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed8c084a0 con 0x558ed8ba2ec0 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfd400700 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d2bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:21:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.322+0000 7f4dfde00700 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 0x558ed8d20f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0a9150 tx=0x7f4dec0a9b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee76cb39b895fa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.323+0000 7f4de7400700 1 -- 172.21.15.42:0/955050894 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0ad800 con 0x558ed8ba2ec0 2024-05-27T10:21:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.323+0000 7f4de7400700 1 -- 172.21.15.42:0/955050894 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dec0b22b0 con 0x558ed8ba2ec0 2024-05-27T10:21:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.323+0000 7f4de7400700 1 -- 172.21.15.42:0/955050894 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0b0de0 con 0x558ed8ba2ec0 2024-05-27T10:21:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 -- 172.21.15.42:0/955050894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 msgr2=0x558ed8d20f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 0x558ed8d20f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0a9150 tx=0x7f4dec0a9b50 comp rx=0 tx=0).stop 2024-05-27T10:21:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 -- 172.21.15.42:0/955050894 shutdown_connections 2024-05-27T10:21:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d218a0 0x558ed8d2e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8ba2ec0 0x558ed8d20f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/955050894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d2bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.324+0000 7f4e0f394b80 1 -- 172.21.15.42:0/955050894 >> 172.21.15.42:0/955050894 conn(0x558ed8c03040 msgr2=0x558ed8d1cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.325+0000 7f4e0f394b80 1 -- 172.21.15.42:0/955050894 shutdown_connections 2024-05-27T10:21:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.325+0000 7f4e0f394b80 1 -- 172.21.15.42:0/955050894 wait complete. 2024-05-27T10:21:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.326+0000 7f4e0f394b80 1 -- start start 2024-05-27T10:21:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.326+0000 7f4e0f394b80 1 Processor -- start 2024-05-27T10:21:21.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d21490 0x558ed8d2bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 0x558ed8d2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558ed8b46020 con 0x558ed8d218a0 2024-05-27T10:21:21.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558ed8be07e0 con 0x558ed8d21490 2024-05-27T10:21:21.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558ed8d2ea10 con 0x558ed8ba2ec0 2024-05-27T10:21:21.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4e0f394b80 10 client.0 _ll_register_callbacks cb 0x558ed8be9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:21.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4dfe800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 0x558ed8d2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.327+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 0x558ed8d2c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40128/0 (socket says 172.21.15.42:40128) 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 -- 172.21.15.42:0/1397271707 learned_addr learned my addr 172.21.15.42:0/1397271707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfd400700 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d21490 0x558ed8d2bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 -- 172.21.15.42:0/1397271707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 msgr2=0x558ed8d2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 -- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d21490 msgr2=0x558ed8d2bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d21490 0x558ed8d2bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.328+0000 7f4dfe800700 1 -- 172.21.15.42:0/1397271707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed8b8f950 con 0x558ed8d218a0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.329+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 0x558ed8d2c120 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4df400b070 tx=0x7f4df400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a58f712243eddd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.329+0000 7f4ddbe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.329+0000 7f4ddbe00700 1 -- 172.21.15.42:0/1397271707 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4030060 con 0x558ed8d218a0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.329+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec0b22b0 con 0x558ed8d218a0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.329+0000 7f4ddbe00700 1 -- 172.21.15.42:0/1397271707 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4df4009390 con 0x558ed8d218a0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.330+0000 7f4ddbe00700 1 -- 172.21.15.42:0/1397271707 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4df4030060 con 0x558ed8d218a0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.330+0000 7f4e0f394b80 10 client.15867 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.330+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ed8c100e0 con 0x558ed8d218a0 2024-05-27T10:21:21.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.330+0000 7f4ddbe00700 1 -- 172.21.15.42:0/1397271707 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dec0b22b0 con 0x558ed8d218a0 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.331+0000 7f4ddbe00700 1 -- 172.21.15.42:0/1397271707 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4df4028070 con 0x558ed8d218a0 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4e0f394b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4e0f394b80 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4e0f394b80 20 client.15867 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8d3baa0 0x558ed8d3def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4e0f394b80 4 client.15867 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ed8c51e10 con 0x558ed8d3baa0 2024-05-27T10:21:21.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.332+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8d3baa0 0x558ed8d3def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.333+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8d3baa0 0x558ed8d3def0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e661f116b90b0b server_cookie=a12e7a7fe02159e0 in_seq=0 out_seq=0 2024-05-27T10:21:21.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.333+0000 7f4ddbe00700 10 client.15867 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.335+0000 7f4ddbe00700 1 -- 172.21.15.42:0/1397271707 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558ed8c51e10 con 0x558ed8d3baa0 2024-05-27T10:21:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.335+0000 7f4ddbe00700 10 client.15867 handle_command_reply: tid=0 2024-05-27T10:21:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.335+0000 7f4e0f394b80 1 client.15867 shutdown 2024-05-27T10:21:21.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 msgr2=0x558ed8d2c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 0x558ed8d2c120 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4df400b070 tx=0x7f4df400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 shutdown_connections 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8d3baa0 0x558ed8d3def0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d218a0 0x558ed8d2c120 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1397271707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d21490 0x558ed8d2bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 >> 172.21.15.42:0/1397271707 conn(0x558ed8c03040 msgr2=0x558ed8d1c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:21.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.336+0000 7f4ddbe00700 0 client.15867 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:21.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.337+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 shutdown_connections 2024-05-27T10:21:21.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.337+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1397271707 wait complete. 2024-05-27T10:21:21.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.337+0000 7f4e0f394b80 20 client.15867 trim_cache size 0 max 16384 2024-05-27T10:21:21.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.380+0000 7f4e0f394b80 1 Processor -- start 2024-05-27T10:21:21.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.380+0000 7f4e0f394b80 1 -- start start 2024-05-27T10:21:21.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d21490 0x558ed8d2db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00fd50 con 0x558ed8ba2ec0 2024-05-27T10:21:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00d5e0 con 0x558ed8d218a0 2024-05-27T10:21:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00b430 con 0x558ed8d21490 2024-05-27T10:21:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4dfde00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40140/0 (socket says 172.21.15.42:40140) 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4dfde00700 1 -- 172.21.15.42:0/1730197174 learned_addr learned my addr 172.21.15.42:0/1730197174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4dfd400700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d21490 0x558ed8d2db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.381+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfde00700 1 -- 172.21.15.42:0/1730197174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d21490 msgr2=0x558ed8d2db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d21490 0x558ed8d2db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfde00700 1 -- 172.21.15.42:0/1730197174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 msgr2=0x558ed8d37ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d37ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfde00700 1 -- 172.21.15.42:0/1730197174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed8d1e3d0 con 0x558ed8ba2ec0 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfe800700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d37ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfd400700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d21490 0x558ed8d2db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.382+0000 7f4dfde00700 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2a600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b2f0 tx=0x7f4dec0a9cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c519336ef24b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.383+0000 7f4de6a00700 1 -- 172.21.15.42:0/1730197174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0a98c0 con 0x558ed8ba2ec0 2024-05-27T10:21:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.383+0000 7f4de6a00700 1 -- 172.21.15.42:0/1730197174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dec0b1e20 con 0x558ed8ba2ec0 2024-05-27T10:21:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.383+0000 7f4de6a00700 1 -- 172.21.15.42:0/1730197174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec009b40 con 0x558ed8ba2ec0 2024-05-27T10:21:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1730197174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 msgr2=0x558ed8d2a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2a600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b2f0 tx=0x7f4dec0a9cc0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1730197174 shutdown_connections 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d37ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2a600 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/1730197174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8d21490 0x558ed8d2db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1730197174 >> 172.21.15.42:0/1730197174 conn(0x558ed8c03040 msgr2=0x558ed8d1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1730197174 shutdown_connections 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.384+0000 7f4e0f394b80 1 -- 172.21.15.42:0/1730197174 wait complete. 2024-05-27T10:21:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.385+0000 7f4e0f394b80 1 -- start start 2024-05-27T10:21:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.385+0000 7f4e0f394b80 1 Processor -- start 2024-05-27T10:21:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d30010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d30550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00baa0 con 0x558ed8d21490 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00cf10 con 0x558ed8d218a0 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc00c170 con 0x558ed8ba2ec0 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4e0f394b80 10 client.0 _ll_register_callbacks cb 0x558ed8be9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4dfd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d30010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4dfd400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d30010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40146/0 (socket says 172.21.15.42:40146) 2024-05-27T10:21:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4dfd400700 1 -- 172.21.15.42:0/4234881878 learned_addr learned my addr 172.21.15.42:0/4234881878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.386+0000 7f4dfde00700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfe800700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d30550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfd400700 1 -- 172.21.15.42:0/4234881878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 msgr2=0x558ed8d2fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfd400700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfd400700 1 -- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 msgr2=0x558ed8d30550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfd400700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d30550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfd400700 1 -- 172.21.15.42:0/4234881878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed8da55b0 con 0x558ed8d21490 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4dfd400700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d30010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4de8012700 tx=0x7f4de8012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df136fd36df1526 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.387+0000 7f4ddbe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.388+0000 7f4ddbe00700 1 -- 172.21.15.42:0/4234881878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8021060 con 0x558ed8d21490 2024-05-27T10:21:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.388+0000 7f4ddbe00700 1 -- 172.21.15.42:0/4234881878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4de8004230 con 0x558ed8d21490 2024-05-27T10:21:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.388+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec0b1e20 con 0x558ed8d21490 2024-05-27T10:21:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.388+0000 7f4ddbe00700 1 -- 172.21.15.42:0/4234881878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4de8004430 con 0x558ed8d21490 2024-05-27T10:21:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.388+0000 7f4e0f394b80 10 client.15873 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.388+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ed8da6820 con 0x558ed8d21490 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4ddbe00700 1 -- 172.21.15.42:0/4234881878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dec0b1e20 con 0x558ed8d21490 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4ddbe00700 1 -- 172.21.15.42:0/4234881878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4de8003aa0 con 0x558ed8d21490 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4e0f394b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4e0f394b80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4e0f394b80 20 client.15873 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8da6f90 0x558ed8da93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4e0f394b80 4 client.15873 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T10:21:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.389+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ed8c4b5f0 con 0x558ed8da6f90 2024-05-27T10:21:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.390+0000 7f4dfde00700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8da6f90 0x558ed8da93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.391+0000 7f4dfde00700 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8da6f90 0x558ed8da93b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451984898833f9b9 server_cookie=4a39033a2122ef7f in_seq=0 out_seq=0 2024-05-27T10:21:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.391+0000 7f4ddbe00700 10 client.15873 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:21.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.394+0000 7f4ddbe00700 1 -- 172.21.15.42:0/4234881878 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ed8c4b5f0 con 0x558ed8da6f90 2024-05-27T10:21:21.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.394+0000 7f4ddbe00700 10 client.15873 handle_command_reply: tid=0 2024-05-27T10:21:21.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.394+0000 7f4e0f394b80 1 client.15873 shutdown 2024-05-27T10:21:21.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 msgr2=0x558ed8d30010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d30010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4de8012700 tx=0x7f4de8012ac0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 shutdown_connections 2024-05-27T10:21:21.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558ed8da6f90 0x558ed8da93b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ed8d218a0 0x558ed8d30550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ed8ba2ec0 0x558ed8d2fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 --2- 172.21.15.42:0/4234881878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ed8d21490 0x558ed8d30010 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 >> 172.21.15.42:0/4234881878 conn(0x558ed8c03040 msgr2=0x558ed8d1c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:21.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.395+0000 7f4ddbe00700 0 client.15873 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:21.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.396+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 shutdown_connections 2024-05-27T10:21:21.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.396+0000 7f4e0f394b80 1 -- 172.21.15.42:0/4234881878 wait complete. 2024-05-27T10:21:21.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.396+0000 7f4e0f394b80 20 client.15873 trim_cache size 0 max 16384 2024-05-27T10:21:21.399 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:21:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-05-27T10:21:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "9dc9dcde-4203-4d42-b88b-1d156896c44d", 2024-05-27T10:21:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-05-27T10:21:21.400 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:21:21.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.397+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4de00f6a80 msgr2=0x7f4de00f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.397+0000 7f4dff200700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4de00f6a80 0x7f4de00f8f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4df40062a0 tx=0x7f4df40090f0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.397+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 msgr2=0x7f4df81997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:21.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.397+0000 7f4dff200700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 0x7f4df81997f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d950 tx=0x7f4dec00dd10 comp rx=0 tx=0).stop 2024-05-27T10:21:21.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 shutdown_connections 2024-05-27T10:21:21.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4de00f6a80 0x7f4de00f8f90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df810e6f0 0x7f4df81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4df8105280 0x7f4df81997f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 --2- 172.21.15.42:0/1924351855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df8105bd0 0x7f4df8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:21.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 >> 172.21.15.42:0/1924351855 conn(0x7f4df8078ab0 msgr2=0x7f4df8103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:21.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 shutdown_connections 2024-05-27T10:21:21.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:21.398+0000 7f4dff200700 1 -- 172.21.15.42:0/1924351855 wait complete. 2024-05-27T10:21:22.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:21:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:22 smithi042 ceph-mon[31661]: pgmap v82: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:22 smithi042 ceph-mon[31661]: scrub queued for path: / 2024-05-27T10:21:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:22 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:22 smithi042 ceph-mon[31661]: scrub summary: active paths [/] 2024-05-27T10:21:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:22 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:22 smithi104 ceph-mon[34812]: pgmap v82: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:22 smithi104 ceph-mon[34812]: scrub queued for path: / 2024-05-27T10:21:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:22 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:22 smithi104 ceph-mon[34812]: scrub summary: active paths [/] 2024-05-27T10:21:23.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:22 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:23.302 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:23.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:22 smithi143 ceph-mon[35545]: pgmap v82: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:23.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:22 smithi143 ceph-mon[35545]: scrub queued for path: / 2024-05-27T10:21:23.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:22 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:23.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:22 smithi143 ceph-mon[35545]: scrub summary: active paths [/] 2024-05-27T10:21:23.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:22 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:24 smithi042 ceph-mon[31661]: pgmap v83: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:25.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:24 smithi104 ceph-mon[34812]: pgmap v83: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:25.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:24 smithi143 ceph-mon[35545]: pgmap v83: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T10:21:26.056 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:25 smithi042 ceph-mon[31661]: scrub summary: idle 2024-05-27T10:21:26.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:25 smithi104 ceph-mon[34812]: scrub summary: idle 2024-05-27T10:21:26.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:25 smithi143 ceph-mon[35545]: scrub summary: idle 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.483+0000 7f5b88c00700 1 -- 172.21.15.42:0/2309503499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84110c80 msgr2=0x7f5b841130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 --2- 172.21.15.42:0/2309503499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84110c80 0x7f5b841130a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b0a0 tx=0x7f5b7c00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 -- 172.21.15.42:0/2309503499 shutdown_connections 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 --2- 172.21.15.42:0/2309503499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84110c80 0x7f5b841130a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 --2- 172.21.15.42:0/2309503499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b841030d0 0x7f5b841034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 --2- 172.21.15.42:0/2309503499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84103a20 0x7f5b84110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 -- 172.21.15.42:0/2309503499 >> 172.21.15.42:0/2309503499 conn(0x7f5b840fedb0 msgr2=0x7f5b841011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 -- 172.21.15.42:0/2309503499 shutdown_connections 2024-05-27T10:21:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.484+0000 7f5b88c00700 1 -- 172.21.15.42:0/2309503499 wait complete. 2024-05-27T10:21:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.486+0000 7f5b88c00700 1 Processor -- start 2024-05-27T10:21:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.486+0000 7f5b88c00700 1 -- start start 2024-05-27T10:21:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.486+0000 7f5b88c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b841030d0 0x7f5b8419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.486+0000 7f5b88c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 0x7f5b8419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b88c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84110c80 0x7f5b841a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b88c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84116570 con 0x7f5b84103a20 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b88c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841163f0 con 0x7f5b84110c80 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b88c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841166f0 con 0x7f5b841030d0 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 0x7f5b8419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 0x7f5b8419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38602/0 (socket says 172.21.15.42:38602) 2024-05-27T10:21:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 -- 172.21.15.42:0/1349255705 learned_addr learned my addr 172.21.15.42:0/1349255705 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81e00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b841030d0 0x7f5b8419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b82800700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84110c80 0x7f5b841a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 -- 172.21.15.42:0/1349255705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b841030d0 msgr2=0x7f5b8419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b841030d0 0x7f5b8419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 -- 172.21.15.42:0/1349255705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84110c80 msgr2=0x7f5b841a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84110c80 0x7f5b841a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.487+0000 7f5b81400700 1 -- 172.21.15.42:0/1349255705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b841a1cc0 con 0x7f5b84103a20 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.488+0000 7f5b81400700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 0x7f5b8419d440 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c9e0 tx=0x7f5b7000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.488+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1349255705 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70007aa0 con 0x7f5b84103a20 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.488+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1349255705 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b70007c40 con 0x7f5b84103a20 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.488+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841a1fb0 con 0x7f5b84103a20 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.488+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1349255705 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70018660 con 0x7f5b84103a20 2024-05-27T10:21:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.488+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b841a24f0 con 0x7f5b84103a20 2024-05-27T10:21:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.493+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1349255705 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5b70018800 con 0x7f5b84103a20 2024-05-27T10:21:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.493+0000 7f5b97745b80 1 Processor -- start 2024-05-27T10:21:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 -- start start 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee74330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee74870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 0x56323ee81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56323eda2e40 con 0x56323ee74e70 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56323ed929e0 con 0x56323ecf5e40 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.494+0000 7f5b97745b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56323ed96a20 con 0x56323ede19f0 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 0x56323ee81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 0x56323ee81110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38616/0 (socket says 172.21.15.42:38616) 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81e00700 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee74870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 -- 172.21.15.42:0/2262875868 learned_addr learned my addr 172.21.15.42:0/2262875868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b82800700 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee74330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 -- 172.21.15.42:0/2262875868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 msgr2=0x56323ee74330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee74330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 -- 172.21.15.42:0/2262875868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 msgr2=0x56323ee74870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee74870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.495+0000 7f5b81400700 1 -- 172.21.15.42:0/2262875868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323ed5b420 con 0x56323ee74e70 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b7aa00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5b680b29f0 0x7f5b680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b81400700 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 0x56323ee81110 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b700dfbf0 tx=0x7f5b700e86b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a80a2e6dc8da563 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b7aa00700 1 -- 172.21.15.42:0/1349255705 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5b700e2570 con 0x7f5b84103a20 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b82800700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5b680b29f0 0x7f5b680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b6f400700 1 -- 172.21.15.42:0/2262875868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700a5700 con 0x56323ee74e70 2024-05-27T10:21:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b6f400700 1 -- 172.21.15.42:0/2262875868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b700a9350 con 0x56323ee74e70 2024-05-27T10:21:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.496+0000 7f5b6f400700 1 -- 172.21.15.42:0/2262875868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700a8fb0 con 0x56323ee74e70 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 -- 172.21.15.42:0/2262875868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 msgr2=0x56323ee81110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b82800700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5b680b29f0 0x7f5b680b4f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009900 tx=0x7f5b7c009b80 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 0x56323ee81110 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b700dfbf0 tx=0x7f5b700e86b0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 -- 172.21.15.42:0/2262875868 shutdown_connections 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ee74e70 0x56323ee81110 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee74330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 --2- 172.21.15.42:0/2262875868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee74870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 -- 172.21.15.42:0/2262875868 >> 172.21.15.42:0/2262875868 conn(0x56323ed8f390 msgr2=0x56323ee704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 -- 172.21.15.42:0/2262875868 shutdown_connections 2024-05-27T10:21:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.498+0000 7f5b97745b80 1 -- 172.21.15.42:0/2262875868 wait complete. 2024-05-27T10:21:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.499+0000 7f5b97745b80 1 -- start start 2024-05-27T10:21:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.499+0000 7f5b97745b80 1 Processor -- start 2024-05-27T10:21:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.499+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.499+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.499+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b97745b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56323ec98fa0 con 0x56323ede19f0 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b97745b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56323ed33760 con 0x56323ecf5e40 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b97745b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56323ee81b20 con 0x56323ee74e70 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b97745b80 10 client.0 _ll_register_callbacks cb 0x56323ed3cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee7ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38628/0 (socket says 172.21.15.42:38628) 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 -- 172.21.15.42:0/34031044 learned_addr learned my addr 172.21.15.42:0/34031044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81e00700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b82800700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 -- 172.21.15.42:0/34031044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 msgr2=0x56323ee7f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 -- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 msgr2=0x56323ee7e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee7e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.500+0000 7f5b81400700 1 -- 172.21.15.42:0/34031044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323ece28d0 con 0x56323ede19f0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.501+0000 7f5b82800700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.501+0000 7f5b81400700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee7ebc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5b700a97d0 tx=0x7f5b700cc2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150d869a01ca0dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.501+0000 7f5b63e00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.501+0000 7f5b63e00700 1 -- 172.21.15.42:0/34031044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700a9190 con 0x56323ede19f0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.501+0000 7f5b63e00700 1 -- 172.21.15.42:0/34031044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5b700a9aa0 con 0x56323ede19f0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.501+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b700a9350 con 0x56323ede19f0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b63e00700 1 -- 172.21.15.42:0/34031044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b700a9610 con 0x56323ede19f0 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 10 client.15885 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56323ed63060 con 0x56323ede19f0 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b63e00700 1 -- 172.21.15.42:0/34031044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b700a9350 con 0x56323ede19f0 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b63e00700 1 -- 172.21.15.42:0/34031044 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5b70020030 con 0x56323ede19f0 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 20 client.15885 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323ee8e940 0x56323ee90d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 4 client.15885 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.502+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56323eda4d90 con 0x56323ee8e940 2024-05-27T10:21:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.503+0000 7f5b81e00700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323ee8e940 0x56323ee90d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.504+0000 7f5b81e00700 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323ee8e940 0x56323ee90d60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7d0bfecd4955a3 server_cookie=795206380b09df in_seq=0 out_seq=0 2024-05-27T10:21:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.504+0000 7f5b63e00700 10 client.15885 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b63e00700 1 -- 172.21.15.42:0/34031044 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56323eda4d90 con 0x56323ee8e940 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b63e00700 10 client.15885 handle_command_reply: tid=0 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 client.15885 shutdown 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 msgr2=0x56323ee7ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee7ebc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5b700a97d0 tx=0x7f5b700cc2d0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 shutdown_connections 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323ee8e940 0x56323ee90d60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee7e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 --2- 172.21.15.42:0/34031044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee7ebc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 >> 172.21.15.42:0/34031044 conn(0x56323ed8f390 msgr2=0x56323ee74370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.506+0000 7f5b63e00700 0 client.15885 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.507+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 shutdown_connections 2024-05-27T10:21:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.507+0000 7f5b97745b80 1 -- 172.21.15.42:0/34031044 wait complete. 2024-05-27T10:21:26.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.507+0000 7f5b97745b80 20 client.15885 trim_cache size 0 max 16384 2024-05-27T10:21:26.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 Processor -- start 2024-05-27T10:21:26.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 -- start start 2024-05-27T10:21:26.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee78030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee78570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b36c0 con 0x56323ede19f0 2024-05-27T10:21:26.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b0f50 con 0x56323ecf5e40 2024-05-27T10:21:26.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.530+0000 7f5b97745b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841aeda0 con 0x56323ee74e70 2024-05-27T10:21:26.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee78570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee78570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38640/0 (socket says 172.21.15.42:38640) 2024-05-27T10:21:26.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 -- 172.21.15.42:0/880898073 learned_addr learned my addr 172.21.15.42:0/880898073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81e00700 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee78030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b82800700 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 -- 172.21.15.42:0/880898073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 msgr2=0x56323ee7fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 -- 172.21.15.42:0/880898073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 msgr2=0x56323ee78030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee78030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.531+0000 7f5b81400700 1 -- 172.21.15.42:0/880898073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323ee81480 con 0x56323ede19f0 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.532+0000 7f5b81400700 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee78570 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b700a6ab0 tx=0x7f5b700e8a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa63e1105d1635f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.532+0000 7f5b6ea00700 1 -- 172.21.15.42:0/880898073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700a8fb0 con 0x56323ede19f0 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.532+0000 7f5b6ea00700 1 -- 172.21.15.42:0/880898073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b700a9350 con 0x56323ede19f0 2024-05-27T10:21:26.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.532+0000 7f5b6ea00700 1 -- 172.21.15.42:0/880898073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700a5700 con 0x56323ede19f0 2024-05-27T10:21:26.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 -- 172.21.15.42:0/880898073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 msgr2=0x56323ee78570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee78570 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b700a6ab0 tx=0x7f5b700e8a00 comp rx=0 tx=0).stop 2024-05-27T10:21:26.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 -- 172.21.15.42:0/880898073 shutdown_connections 2024-05-27T10:21:26.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ee74e70 0x56323ee7fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ecf5e40 0x56323ee78030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 --2- 172.21.15.42:0/880898073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ede19f0 0x56323ee78570 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 -- 172.21.15.42:0/880898073 >> 172.21.15.42:0/880898073 conn(0x56323ed8f390 msgr2=0x56323ee7d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:26.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.533+0000 7f5b97745b80 1 -- 172.21.15.42:0/880898073 shutdown_connections 2024-05-27T10:21:26.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.534+0000 7f5b97745b80 1 -- 172.21.15.42:0/880898073 wait complete. 2024-05-27T10:21:26.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.534+0000 7f5b97745b80 1 -- start start 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.534+0000 7f5b97745b80 1 Processor -- start 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 0x56323ee8f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee8fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ee74e70 0x56323ee90140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841af410 con 0x56323ecf5e40 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b0880 con 0x56323ee74e70 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841afae0 con 0x56323ede19f0 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b97745b80 10 client.0 _ll_register_callbacks cb 0x56323ed3cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:26.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b81e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 0x56323ee8f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b81e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 0x56323ee8f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38652/0 (socket says 172.21.15.42:38652) 2024-05-27T10:21:26.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.535+0000 7f5b81e00700 1 -- 172.21.15.42:0/3252457568 learned_addr learned my addr 172.21.15.42:0/3252457568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:26.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b82800700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ee74e70 0x56323ee90140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81400700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee8fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81e00700 1 -- 172.21.15.42:0/3252457568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 msgr2=0x56323ee8fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81e00700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee8fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81e00700 1 -- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ee74e70 msgr2=0x56323ee90140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81e00700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ee74e70 0x56323ee90140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81e00700 1 -- 172.21.15.42:0/3252457568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323ee7f470 con 0x56323ecf5e40 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b81e00700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 0x56323ee8f6c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5b74004790 tx=0x7f5b74012a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e92ef3266e1534b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.536+0000 7f5b63e00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b63e00700 1 -- 172.21.15.42:0/3252457568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74004a30 con 0x56323ecf5e40 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b63e00700 1 -- 172.21.15.42:0/3252457568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5b74004bd0 con 0x56323ecf5e40 2024-05-27T10:21:26.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b700a9350 con 0x56323ecf5e40 2024-05-27T10:21:26.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b63e00700 1 -- 172.21.15.42:0/3252457568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b7402a050 con 0x56323ecf5e40 2024-05-27T10:21:26.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b97745b80 10 client.15891 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:26.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56323ee6c240 con 0x56323ecf5e40 2024-05-27T10:21:26.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.537+0000 7f5b63e00700 1 -- 172.21.15.42:0/3252457568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b700a9350 con 0x56323ecf5e40 2024-05-27T10:21:26.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b63e00700 1 -- 172.21.15.42:0/3252457568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5b7401c5b0 con 0x56323ecf5e40 2024-05-27T10:21:26.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b97745b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b97745b80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b97745b80 20 client.15891 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b97745b80 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323eef9760 0x56323eefbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b97745b80 4 client.15891 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.538+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56323ed9e570 con 0x56323eef9760 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.539+0000 7f5b81400700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323eef9760 0x56323eefbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.539+0000 7f5b81400700 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323eef9760 0x56323eefbb80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3cd8ca828686cc7 server_cookie=fcf41117034f8b40 in_seq=0 out_seq=0 2024-05-27T10:21:26.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.539+0000 7f5b63e00700 10 client.15891 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b63e00700 1 -- 172.21.15.42:0/3252457568 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56323ed9e570 con 0x56323eef9760 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b63e00700 10 client.15891 handle_command_reply: tid=0 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 client.15891 shutdown 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 msgr2=0x56323ee8f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 0x56323ee8f6c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5b74004790 tx=0x7f5b74012a60 comp rx=0 tx=0).stop 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 shutdown_connections 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56323eef9760 0x56323eefbb80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56323ee74e70 0x56323ee90140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56323ecf5e40 0x56323ee8f6c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 --2- 172.21.15.42:0/3252457568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56323ede19f0 0x56323ee8fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 >> 172.21.15.42:0/3252457568 conn(0x56323ed8f390 msgr2=0x56323ee89db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b63e00700 0 client.15891 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 shutdown_connections 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.540+0000 7f5b97745b80 1 -- 172.21.15.42:0/3252457568 wait complete. 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.541+0000 7f5b97745b80 20 client.15891 trim_cache size 0 max 16384 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:21:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.541+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5b680b29f0 msgr2=0x7f5b680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.541+0000 7f5b88c00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5b680b29f0 0x7f5b680b4f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009900 tx=0x7f5b7c009b80 comp rx=0 tx=0).stop 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.541+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 msgr2=0x7f5b8419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.541+0000 7f5b88c00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 0x7f5b8419d440 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c9e0 tx=0x7f5b7000cda0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 shutdown_connections 2024-05-27T10:21:26.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5b680b29f0 0x7f5b680b4f00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84110c80 0x7f5b841a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b841030d0 0x7f5b8419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 --2- 172.21.15.42:0/1349255705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84103a20 0x7f5b8419d440 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:26.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 >> 172.21.15.42:0/1349255705 conn(0x7f5b840fedb0 msgr2=0x7f5b84112650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:26.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 shutdown_connections 2024-05-27T10:21:26.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:26.542+0000 7f5b88c00700 1 -- 172.21.15.42:0/1349255705 wait complete. 2024-05-27T10:21:27.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:26 smithi104 ceph-mon[34812]: pgmap v84: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:21:27.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:26 smithi042 ceph-mon[31661]: pgmap v84: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:21:27.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:26 smithi143 ceph-mon[35545]: pgmap v84: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:21:27.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T10:21:27.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:21:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:28.560 INFO:tasks.workunit.client.0.smithi042.stderr:Updating files: 52% (6400/12271) Updating files: 53% (6504/12271) Updating files: 54% (6627/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 87% (10779/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-27T10:21:29.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:28 smithi042 ceph-mon[31661]: pgmap v85: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T10:21:29.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:28 smithi143 ceph-mon[35545]: pgmap v85: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T10:21:29.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:28 smithi104 ceph-mon[34812]: pgmap v85: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T10:21:29.513 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'a6507388dd5057528934822c0163b0c347ef1d5d'. 2024-05-27T10:21:29.513 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.513 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T10:21:29.513 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T10:21:29.513 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T10:21:29.513 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:29.514 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at a6507388dd5 Merge PR #53755 into wip-vshankar-testing-20240521.105740-debug 2024-05-27T10:21:29.528 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-27T10:21:29.645 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T10:21:29.654 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T10:21:29.654 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 -- 172.21.15.42:0/326390307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 msgr2=0x7f0f70109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/326390307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f70109150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0099c0 tx=0x7f0f5c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 -- 172.21.15.42:0/326390307 shutdown_connections 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/326390307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f70109690 0x7f0f70110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/326390307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f70104410 0x7f0f70104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/326390307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f70109150 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 -- 172.21.15.42:0/326390307 >> 172.21.15.42:0/326390307 conn(0x7f0f7007aca0 msgr2=0x7f0f7007b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:30.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 -- 172.21.15.42:0/326390307 shutdown_connections 2024-05-27T10:21:30.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.989+0000 7f0f6fe00700 1 -- 172.21.15.42:0/326390307 wait complete. 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.990+0000 7f0f6fe00700 1 Processor -- start 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.990+0000 7f0f6fe00700 1 -- start start 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6fe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f70104410 0x7f0f7019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f7019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6fe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f70109690 0x7f0f701a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701141c0 con 0x7f0f70108d40 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6fe00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70114040 con 0x7f0f70104410 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6fe00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70114340 con 0x7f0f70109690 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f7019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f7019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38666/0 (socket says 172.21.15.42:38666) 2024-05-27T10:21:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6e000700 1 -- 172.21.15.42:0/2848562955 learned_addr learned my addr 172.21.15.42:0/2848562955 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.991+0000 7f0f6ea00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f70104410 0x7f0f7019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6f400700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f70109690 0x7f0f701a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6e000700 1 -- 172.21.15.42:0/2848562955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f70109690 msgr2=0x7f0f701a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6e000700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f70109690 0x7f0f701a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6e000700 1 -- 172.21.15.42:0/2848562955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f70104410 msgr2=0x7f0f7019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6e000700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f70104410 0x7f0f7019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6e000700 1 -- 172.21.15.42:0/2848562955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f701a1c80 con 0x7f0f70108d40 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.992+0000 7f0f6e000700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f7019d350 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00b580 tx=0x7f0f5c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.993+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c027070 con 0x7f0f70108d40 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.993+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f5c00fd30 con 0x7f0f70108d40 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.993+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f701a1f10 con 0x7f0f70108d40 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.993+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c022e50 con 0x7f0f70108d40 2024-05-27T10:21:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.993+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f701a23f0 con 0x7f0f70108d40 2024-05-27T10:21:30.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.994+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f700512b0 con 0x7f0f70108d40 2024-05-27T10:21:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.996+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0f5c02a070 con 0x7f0f70108d40 2024-05-27T10:21:30.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.998+0000 7f0f67400700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0f540b2b90 0x7f0f540b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:30.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.998+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0f5c0e3600 con 0x7f0f70108d40 2024-05-27T10:21:30.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.998+0000 7f0f6ea00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0f540b2b90 0x7f0f540b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:30.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:30.999+0000 7f0f6ea00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0f540b2b90 0x7f0f540b50a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005950 tx=0x7f0f6000b410 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.001+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f5c013070 con 0x7f0f70108d40 2024-05-27T10:21:31.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:30 smithi104 ceph-mon[34812]: pgmap v86: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:21:31.200 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T10:21:31.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.300+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f701a9890 con 0x7f0f70108d40 2024-05-27T10:21:31.300 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:30 smithi042 ceph-mon[31661]: pgmap v86: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:21:31.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.301+0000 7f0f67400700 1 -- 172.21.15.42:0/2848562955 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f0f5c0ab060 con 0x7f0f70108d40 2024-05-27T10:21:31.301 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:21:31.302 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:18:42.569936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":17,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.306+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0f540b2b90 msgr2=0x7f0f540b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.306+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0f540b2b90 0x7f0f540b50a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005950 tx=0x7f0f6000b410 comp rx=0 tx=0).stop 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.306+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 msgr2=0x7f0f7019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.306+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f7019d350 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00b580 tx=0x7f0f5c0057c0 comp rx=0 tx=0).stop 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 shutdown_connections 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f0f540b2b90 0x7f0f540b50a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f70109690 0x7f0f701a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f70104410 0x7f0f7019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 --2- 172.21.15.42:0/2848562955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f70108d40 0x7f0f7019d350 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:31.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 >> 172.21.15.42:0/2848562955 conn(0x7f0f7007aca0 msgr2=0x7f0f70110cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:31.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 shutdown_connections 2024-05-27T10:21:31.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:31.307+0000 7f0f6fe00700 1 -- 172.21.15.42:0/2848562955 wait complete. 2024-05-27T10:21:31.308 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T10:21:31.308 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-05-27T10:21:31.309 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:30 smithi143 ceph-mon[35545]: pgmap v86: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T10:21:31.392 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T10:21:31.397 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T10:21:31.397 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T10:21:31.471 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T10:21:31.482 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:21:31.482 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-27T10:21:31.534 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-05-27T10:21:31.535 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-05-27T10:21:31.535 DEBUG:teuthology.orchestra.run.smithi042:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-05-27T10:21:31.692 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 'getting blogbench' 2024-05-27T10:21:31.692 INFO:tasks.workunit.client.0.smithi042.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-05-27T10:21:31.693 INFO:tasks.workunit.client.0.smithi042.stdout:getting blogbench 2024-05-27T10:21:31.726 INFO:tasks.workunit.client.0.smithi042.stderr:--2024-05-27 10:21:31-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-05-27T10:21:31.728 INFO:tasks.workunit.client.0.smithi042.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-27T10:21:31.765 INFO:tasks.workunit.client.0.smithi042.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-27T10:21:31.802 INFO:tasks.workunit.client.0.smithi042.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-27T10:21:31.803 INFO:tasks.workunit.client.0.smithi042.stderr:Length: 103355 (101K) [application/octet-stream] 2024-05-27T10:21:31.810 INFO:tasks.workunit.client.0.smithi042.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-05-27T10:21:31.821 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:31.877 INFO:tasks.workunit.client.0.smithi042.stderr: 0K .......... .......... .......... .......... .......... 49% 768K 0s 2024-05-27T10:21:31.913 INFO:tasks.workunit.client.0.smithi042.stderr: 50K .......... .......... .......... .......... .......... 99% 1.39M 0s 2024-05-27T10:21:31.913 INFO:tasks.workunit.client.0.smithi042.stderr: 100K 100% 1.74T=0.1s 2024-05-27T10:21:31.913 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:31.914 INFO:tasks.workunit.client.0.smithi042.stderr:2024-05-27 10:21:31 (1006 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-05-27T10:21:31.914 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-05-27T10:21:31.916 INFO:tasks.workunit.client.0.smithi042.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-05-27T10:21:32.025 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/ 2024-05-27T10:21:32.030 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/m4/ 2024-05-27T10:21:32.035 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-05-27T10:21:32.044 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/man/ 2024-05-27T10:21:32.052 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/man/blogbench.8 2024-05-27T10:21:32.060 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/man/Makefile.am 2024-05-27T10:21:32.069 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/man/Makefile.in 2024-05-27T10:21:32.078 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/ 2024-05-27T10:21:32.082 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/mysnprintf.c 2024-05-27T10:21:32.089 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/mysnprintf.h 2024-05-27T10:21:32.097 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/blogbench.c 2024-05-27T10:21:32.106 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/blogbench.h 2024-05-27T10:21:32.114 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/fakesnprintf.c 2024-05-27T10:21:32.125 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/fakesnprintf.h 2024-05-27T10:21:32.131 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/blogbench_p.h 2024-05-27T10:21:32.137 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-05-27T10:21:32.148 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-05-27T10:21:32.154 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/Makefile.am 2024-05-27T10:21:32.161 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/Makefile.in 2024-05-27T10:21:32.169 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/process.c 2024-05-27T10:21:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 -- 172.21.15.42:0/299177677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5010e000 msgr2=0x7fae501104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 --2- 172.21.15.42:0/299177677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5010e000 0x7fae501104f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fae48009ca0 tx=0x7fae4802a8c0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 -- 172.21.15.42:0/299177677 shutdown_connections 2024-05-27T10:21:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 --2- 172.21.15.42:0/299177677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5010e000 0x7fae501104f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 --2- 172.21.15.42:0/299177677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010b610 0x7fae5010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 --2- 172.21.15.42:0/299177677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae50104bb0 0x7fae50104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 -- 172.21.15.42:0/299177677 >> 172.21.15.42:0/299177677 conn(0x7fae500fe210 msgr2=0x7fae50100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.175+0000 7fae4fe00640 1 -- 172.21.15.42:0/299177677 shutdown_connections 2024-05-27T10:21:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.176+0000 7fae4fe00640 1 -- 172.21.15.42:0/299177677 wait complete. 2024-05-27T10:21:32.176 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/globals.h 2024-05-27T10:21:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.177+0000 7fae4fe00640 1 Processor -- start 2024-05-27T10:21:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.177+0000 7fae4fe00640 1 -- start start 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.178+0000 7fae4fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 0x7fae50102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.178+0000 7fae4fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5010b610 0x7fae501029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.178+0000 7fae4fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010e000 0x7fae50102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.178+0000 7fae4fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae50113bc0 con 0x7fae50104bb0 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.178+0000 7fae4fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae50113a40 con 0x7fae5010e000 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.178+0000 7fae4fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fae50113d40 con 0x7fae5010b610 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 0x7fae50102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 0x7fae50102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38680/0 (socket says 172.21.15.42:38680) 2024-05-27T10:21:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 -- 172.21.15.42:0/1051224712 learned_addr learned my addr 172.21.15.42:0/1051224712 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4e000640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5010b610 0x7fae501029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4f400640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010e000 0x7fae50102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 -- 172.21.15.42:0/1051224712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5010b610 msgr2=0x7fae501029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5010b610 0x7fae501029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 -- 172.21.15.42:0/1051224712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010e000 msgr2=0x7fae50102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010e000 0x7fae50102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4ea00640 1 -- 172.21.15.42:0/1051224712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae501a8920 con 0x7fae50104bb0 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.179+0000 7fae4f400640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010e000 0x7fae50102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.180+0000 7fae4ea00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 0x7fae50102470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fae4000d770 tx=0x7fae4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.180+0000 7fae47400640 1 -- 172.21.15.42:0/1051224712 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40017070 con 0x7fae50104bb0 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.180+0000 7fae47400640 1 -- 172.21.15.42:0/1051224712 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae400054b0 con 0x7fae50104bb0 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.180+0000 7fae47400640 1 -- 172.21.15.42:0/1051224712 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40004c60 con 0x7fae50104bb0 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.180+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae501a8c10 con 0x7fae50104bb0 2024-05-27T10:21:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.180+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae501a9160 con 0x7fae50104bb0 2024-05-27T10:21:32.181 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/commenter.c 2024-05-27T10:21:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.185+0000 7fae47400640 1 -- 172.21.15.42:0/1051224712 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fae40004e00 con 0x7fae50104bb0 2024-05-27T10:21:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.185+0000 7fae5c58e740 1 Processor -- start 2024-05-27T10:21:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.185+0000 7fae5c58e740 1 -- start start 2024-05-27T10:21:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.185+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfaca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.185+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae5c58e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fa3cf1db90 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae5c58e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa3cf0d730 con 0x55fa3cbe12f0 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae5c58e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fa3cf11770 con 0x55fa3cbe94d0 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfaca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfaca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38690/0 (socket says 172.21.15.42:38690) 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae4f400640 1 -- 172.21.15.42:0/414076587 learned_addr learned my addr 172.21.15.42:0/414076587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae4ea00640 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.186+0000 7fae4e000640 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4f400640 1 -- 172.21.15.42:0/414076587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe94d0 msgr2=0x55fa3cfad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4f400640 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4f400640 1 -- 172.21.15.42:0/414076587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 msgr2=0x55fa3cfacfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4f400640 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfacfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4f400640 1 -- 172.21.15.42:0/414076587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3cea68b0 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae47400640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fae340b29d0 0x7fae340b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae47400640 1 -- 172.21.15.42:0/1051224712 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fae400e2060 con 0x7fae50104bb0 2024-05-27T10:21:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4ea00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fae340b29d0 0x7fae340b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.187+0000 7fae4f400640 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfaca60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fae48006010 tx=0x7fae48031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1c098daefeca82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:32.188 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/rewriter.c 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.188+0000 7fae3b400640 1 -- 172.21.15.42:0/414076587 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48031980 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.188+0000 7fae4ea00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fae340b29d0 0x7fae340b4ee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae400df8b0 tx=0x7fae400e5bb0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.188+0000 7fae3b400640 1 -- 172.21.15.42:0/414076587 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae48036070 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.188+0000 7fae3b400640 1 -- 172.21.15.42:0/414076587 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48004480 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.189+0000 7fae5c58e740 1 -- 172.21.15.42:0/414076587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 msgr2=0x55fa3cfaca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.189+0000 7fae5c58e740 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfaca60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fae48006010 tx=0x7fae48031500 comp rx=0 tx=0).stop 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 -- 172.21.15.42:0/414076587 shutdown_connections 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfacfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 --2- 172.21.15.42:0/414076587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfaca60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 -- 172.21.15.42:0/414076587 >> 172.21.15.42:0/414076587 conn(0x55fa3cbfe300 msgr2=0x55fa3cfa9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 -- 172.21.15.42:0/414076587 shutdown_connections 2024-05-27T10:21:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 -- 172.21.15.42:0/414076587 wait complete. 2024-05-27T10:21:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.190+0000 7fae5c58e740 1 -- start start 2024-05-27T10:21:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 Processor -- start 2024-05-27T10:21:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfb7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fa3cbf3790 con 0x55fa3cbe94d0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa3ceb5bc0 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fa3cfb9b30 con 0x55fa3cbe12f0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae5c58e740 10 client.0 _ll_register_callbacks cb 0x55fa3ce93400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.191+0000 7fae4e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfb7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfb7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38696/0 (socket says 172.21.15.42:38696) 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 -- 172.21.15.42:0/333994046 learned_addr learned my addr 172.21.15.42:0/333994046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4f400640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 -- 172.21.15.42:0/333994046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 msgr2=0x55fa3cfb6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4f400640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 -- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cc0a1b0 msgr2=0x55fa3cfc2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4e000640 1 -- 172.21.15.42:0/333994046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3cc22210 con 0x55fa3cbe94d0 2024-05-27T10:21:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.192+0000 7fae4f400640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.193+0000 7fae4e000640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfb7130 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0104d0 tx=0x7fae3c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42a18fc9ef7347b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.193+0000 7fae2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:32.194 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/reader.c 2024-05-27T10:21:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.193+0000 7fae2fe00640 1 -- 172.21.15.42:0/333994046 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01a020 con 0x55fa3cbe94d0 2024-05-27T10:21:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.193+0000 7fae2fe00640 1 -- 172.21.15.42:0/333994046 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fae3c016690 con 0x55fa3cbe94d0 2024-05-27T10:21:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.193+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae48036070 con 0x55fa3cbe94d0 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.194+0000 7fae2fe00640 1 -- 172.21.15.42:0/333994046 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae3c016890 con 0x55fa3cbe94d0 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.194+0000 7fae5c58e740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.194+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa3ccee620 con 0x55fa3cbe94d0 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae2fe00640 1 -- 172.21.15.42:0/333994046 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae48036070 con 0x55fa3cbe94d0 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae2fe00640 1 -- 172.21.15.42:0/333994046 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fae3c027020 con 0x55fa3cbe94d0 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae5c58e740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae5c58e740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae5c58e740 20 client.15909 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae5c58e740 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cbc8d80 0x55fa3cfc8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae5c58e740 4 client.15909 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.195+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa3cf1fae0 con 0x55fa3cbc8d80 2024-05-27T10:21:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.196+0000 7fae4f400640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cbc8d80 0x55fa3cfc8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.197+0000 7fae4f400640 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cbc8d80 0x55fa3cfc8ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8945a58a528145 server_cookie=90e32e8022795129 in_seq=0 out_seq=0 2024-05-27T10:21:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.197+0000 7fae2fe00640 10 client.15909 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:32.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:31 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2848562955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:21:32.198 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/writer.c 2024-05-27T10:21:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae2fe00640 1 -- 172.21.15.42:0/333994046 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fa3cf1fae0 con 0x55fa3cbc8d80 2024-05-27T10:21:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae2fe00640 10 client.15909 handle_command_reply: tid=0 2024-05-27T10:21:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 client.15909 shutdown 2024-05-27T10:21:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 msgr2=0x55fa3cfb7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfb7130 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0104d0 tx=0x7fae3c010990 comp rx=0 tx=0).stop 2024-05-27T10:21:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 shutdown_connections 2024-05-27T10:21:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfb7130 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cbc8d80 0x55fa3cfc8ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 --2- 172.21.15.42:0/333994046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 >> 172.21.15.42:0/333994046 conn(0x55fa3cbfe300 msgr2=0x55fa3cfac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae2fe00640 0 client.15909 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 shutdown_connections 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.200+0000 7fae5c58e740 1 -- 172.21.15.42:0/333994046 wait complete. 2024-05-27T10:21:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.201+0000 7fae5c58e740 20 client.15909 trim_cache size 0 max 16384 2024-05-27T10:21:32.203 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/src/helpers.c 2024-05-27T10:21:32.208 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/NEWS 2024-05-27T10:21:32.214 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/compile 2024-05-27T10:21:32.218 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/depcomp 2024-05-27T10:21:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.220+0000 7fae5c58e740 1 Processor -- start 2024-05-27T10:21:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.220+0000 7fae5c58e740 1 -- start start 2024-05-27T10:21:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfc2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae5c58e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae501b4260 con 0x55fa3cbe94d0 2024-05-27T10:21:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae5c58e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae501b1b30 con 0x55fa3cbe12f0 2024-05-27T10:21:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae5c58e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fae501af980 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae4e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfc2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae4e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfc2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38706/0 (socket says 172.21.15.42:38706) 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae4e000640 1 -- 172.21.15.42:0/1014120769 learned_addr learned my addr 172.21.15.42:0/1014120769 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.221+0000 7fae4f400640 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4f400640 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4e000640 1 -- 172.21.15.42:0/1014120769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cc0a1b0 msgr2=0x55fa3cfc3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4e000640 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4e000640 1 -- 172.21.15.42:0/1014120769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 msgr2=0x55fa3cfb2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4e000640 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4e000640 1 -- 172.21.15.42:0/1014120769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3cfac3f0 con 0x55fa3cbe94d0 2024-05-27T10:21:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae4e000640 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfc2bd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b700 tx=0x7fae3c006ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a3d0c1b5301387 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae3aa00640 1 -- 172.21.15.42:0/1014120769 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01a040 con 0x55fa3cbe94d0 2024-05-27T10:21:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae3aa00640 1 -- 172.21.15.42:0/1014120769 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae3c01f3f0 con 0x55fa3cbe94d0 2024-05-27T10:21:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.222+0000 7fae3aa00640 1 -- 172.21.15.42:0/1014120769 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01f6e0 con 0x55fa3cbe94d0 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.223+0000 7fae5c58e740 1 -- 172.21.15.42:0/1014120769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 msgr2=0x55fa3cfc2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.223+0000 7fae5c58e740 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfc2bd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b700 tx=0x7fae3c006ab0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 -- 172.21.15.42:0/1014120769 shutdown_connections 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cc0a1b0 0x55fa3cfc3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfc2bd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 --2- 172.21.15.42:0/1014120769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfb2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 -- 172.21.15.42:0/1014120769 >> 172.21.15.42:0/1014120769 conn(0x55fa3cbfe300 msgr2=0x55fa3cfb6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 -- 172.21.15.42:0/1014120769 shutdown_connections 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 -- 172.21.15.42:0/1014120769 wait complete. 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.224+0000 7fae5c58e740 1 -- start start 2024-05-27T10:21:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 Processor -- start 2024-05-27T10:21:32.225 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/aclocal.m4 2024-05-27T10:21:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfbc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfbc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3d02f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae501afff0 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae501b1460 con 0x55fa3cbe94d0 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fae501b06c0 con 0x55fa3cbe12f0 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.225+0000 7fae5c58e740 10 client.0 _ll_register_callbacks cb 0x7fae3c00b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3d02f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3d02f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38714/0 (socket says 172.21.15.42:38714) 2024-05-27T10:21:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 -- 172.21.15.42:0/4066709597 learned_addr learned my addr 172.21.15.42:0/4066709597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfbc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4e000640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfbc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 -- 172.21.15.42:0/4066709597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 msgr2=0x55fa3cfbc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfbc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 -- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe94d0 msgr2=0x55fa3cfbc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfbc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.226+0000 7fae4f400640 1 -- 172.21.15.42:0/4066709597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3d02fa40 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae4f400640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3d02f500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fae4802ae80 tx=0x7fae48004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c5816f9455c963 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae2fe00640 1 -- 172.21.15.42:0/4066709597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48049040 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3c01f3f0 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae2fe00640 1 -- 172.21.15.42:0/4066709597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fae48003b70 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae2fe00640 1 -- 172.21.15.42:0/4066709597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae48003d70 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae5c58e740 10 client.15915 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.227+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa3d030c60 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae2fe00640 1 -- 172.21.15.42:0/4066709597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae3c01f3f0 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae2fe00640 1 -- 172.21.15.42:0/4066709597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fae48030830 con 0x55fa3cc0a1b0 2024-05-27T10:21:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae5c58e740 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae5c58e740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae5c58e740 20 client.15915 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae5c58e740 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cb9e1c0 0x55fa3d032f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae5c58e740 4 client.15915 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:21:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.228+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa3cf192c0 con 0x55fa3cb9e1c0 2024-05-27T10:21:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.229+0000 7fae4e000640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cb9e1c0 0x55fa3d032f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.230+0000 7fae4e000640 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cb9e1c0 0x55fa3d032f60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7a0fe12769bbfe server_cookie=314d92ccf28c0c21 in_seq=0 out_seq=0 2024-05-27T10:21:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.230+0000 7fae2fe00640 10 client.15915 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:32.232 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/README 2024-05-27T10:21:32.237 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/configure 2024-05-27T10:21:32.257 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/configure.ac 2024-05-27T10:21:32.263 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/config.guess 2024-05-27T10:21:32.273 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/install-sh 2024-05-27T10:21:32.281 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/config.sub 2024-05-27T10:21:32.291 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/missing 2024-05-27T10:21:32.299 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/mkinstalldirs 2024-05-27T10:21:32.306 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/Makefile.am 2024-05-27T10:21:32.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:31 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2848562955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:21:32.312 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/Makefile.in 2024-05-27T10:21:32.320 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/config.h.in 2024-05-27T10:21:32.329 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/AUTHORS 2024-05-27T10:21:32.336 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/INSTALL 2024-05-27T10:21:32.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:31 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2848562955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:21:32.346 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/ChangeLog 2024-05-27T10:21:32.355 INFO:tasks.workunit.client.0.smithi042.stdout:blogbench-1.0/COPYING 2024-05-27T10:21:32.366 INFO:tasks.workunit.client.0.smithi042.stderr:+ cd blogbench-1.0/ 2024-05-27T10:21:32.374 INFO:tasks.workunit.client.0.smithi042.stdout:making blogbench 2024-05-27T10:21:32.374 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 'making blogbench' 2024-05-27T10:21:32.374 INFO:tasks.workunit.client.0.smithi042.stderr:+ ./configure 2024-05-27T10:21:32.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T10:21:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae2fe00640 1 -- 172.21.15.42:0/4066709597 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fa3cf192c0 con 0x55fa3cb9e1c0 2024-05-27T10:21:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae2fe00640 10 client.15915 handle_command_reply: tid=0 2024-05-27T10:21:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 client.15915 shutdown 2024-05-27T10:21:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 msgr2=0x55fa3d02f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3d02f500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fae4802ae80 tx=0x7fae48004a60 comp rx=0 tx=0).stop 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 shutdown_connections 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa3cc0a1b0 0x55fa3d02f500 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa3cbe94d0 0x55fa3cfbc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fa3cb9e1c0 0x55fa3d032f60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.576+0000 7fae5c58e740 1 --2- 172.21.15.42:0/4066709597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fa3cbe12f0 0x55fa3cfbc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.577+0000 7fae2fe00640 0 client.15915 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.577+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 >> 172.21.15.42:0/4066709597 conn(0x55fa3cbfe300 msgr2=0x55fa3cfb13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.577+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 shutdown_connections 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.577+0000 7fae5c58e740 1 -- 172.21.15.42:0/4066709597 wait complete. 2024-05-27T10:21:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.577+0000 7fae5c58e740 20 client.15915 trim_cache size 0 max 16384 2024-05-27T10:21:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:21:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:21:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:21:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fae340b29d0 msgr2=0x7fae340b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fae340b29d0 0x7fae340b4ee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae400df8b0 tx=0x7fae400e5bb0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 msgr2=0x7fae50102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 0x7fae50102470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fae4000d770 tx=0x7fae4000dc30 comp rx=0 tx=0).stop 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 shutdown_connections 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fae340b29d0 0x7fae340b4ee0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae5010e000 0x7fae50102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5010b610 0x7fae501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 --2- 172.21.15.42:0/1051224712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae50104bb0 0x7fae50102470 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 >> 172.21.15.42:0/1051224712 conn(0x7fae500fe210 msgr2=0x7fae5010e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 shutdown_connections 2024-05-27T10:21:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:32.578+0000 7fae4fe00640 1 -- 172.21.15.42:0/1051224712 wait complete. 2024-05-27T10:21:32.749 INFO:tasks.workunit.client.0.smithi042.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-27T10:21:32.924 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:33.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:32 smithi104 ceph-mon[34812]: pgmap v87: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-05-27T10:21:33.265 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:32 smithi042 ceph-mon[31661]: pgmap v87: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-05-27T10:21:33.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:32 smithi143 ceph-mon[35545]: pgmap v87: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-05-27T10:21:33.786 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether build environment is sane... yes 2024-05-27T10:21:33.869 INFO:tasks.workunit.client.0.smithi042.stdout:checking for gawk... gawk 2024-05-27T10:21:33.934 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether make sets $(MAKE)... yes 2024-05-27T10:21:34.025 INFO:tasks.workunit.client.0.smithi042.stdout:checking for gcc... gcc 2024-05-27T10:21:34.271 INFO:tasks.workunit.client.0.smithi042.stdout:checking for C compiler default output file name... a.out 2024-05-27T10:21:34.287 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether the C compiler works... yes 2024-05-27T10:21:34.307 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether we are cross compiling... no 2024-05-27T10:21:34.492 INFO:tasks.workunit.client.0.smithi042.stdout:checking for suffix of executables... 2024-05-27T10:21:34.653 INFO:tasks.workunit.client.0.smithi042.stdout:checking for suffix of object files... o 2024-05-27T10:21:34.803 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether we are using the GNU C compiler... yes 2024-05-27T10:21:34.978 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether gcc accepts -g... yes 2024-05-27T10:21:35.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:34 smithi104 ceph-mon[34812]: pgmap v88: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-05-27T10:21:35.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:34 smithi042 ceph-mon[31661]: pgmap v88: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-05-27T10:21:35.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:34 smithi143 ceph-mon[35545]: pgmap v88: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s 2024-05-27T10:21:35.347 INFO:tasks.workunit.client.0.smithi042.stdout:checking for gcc option to accept ANSI C... none needed 2024-05-27T10:21:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.529+0000 7fa90b200700 1 -- 172.21.15.42:0/2127693361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 msgr2=0x7fa90c1110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.529+0000 7fa90b200700 1 --2- 172.21.15.42:0/2127693361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c1110c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa90400b0a0 tx=0x7fa90400b3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 -- 172.21.15.42:0/2127693361 shutdown_connections 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 --2- 172.21.15.42:0/2127693361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c1110c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 --2- 172.21.15.42:0/2127693361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa90c109250 0x7fa90c109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 --2- 172.21.15.42:0/2127693361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90c1025d0 0x7fa90c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 -- 172.21.15.42:0/2127693361 >> 172.21.15.42:0/2127693361 conn(0x7fa90c078360 msgr2=0x7fa90c0787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 -- 172.21.15.42:0/2127693361 shutdown_connections 2024-05-27T10:21:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.530+0000 7fa90b200700 1 -- 172.21.15.42:0/2127693361 wait complete. 2024-05-27T10:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.531+0000 7fa90b200700 1 Processor -- start 2024-05-27T10:21:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.531+0000 7fa90b200700 1 -- start start 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa90b200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90c1025d0 0x7fa90c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa90b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa90b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa90c109250 0x7fa90c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa90b200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1143f0 con 0x7fa90c102f20 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa90b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c114270 con 0x7fa90c109250 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa90b200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c114570 con 0x7fa90c1025d0 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38122/0 (socket says 172.21.15.42:38122) 2024-05-27T10:21:35.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.532+0000 7fa909400700 1 -- 172.21.15.42:0/4278138841 learned_addr learned my addr 172.21.15.42:0/4278138841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909e00700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90c1025d0 0x7fa90c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa90a800700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa90c109250 0x7fa90c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909400700 1 -- 172.21.15.42:0/4278138841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90c1025d0 msgr2=0x7fa90c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909400700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90c1025d0 0x7fa90c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909400700 1 -- 172.21.15.42:0/4278138841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa90c109250 msgr2=0x7fa90c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909400700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa90c109250 0x7fa90c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909400700 1 -- 172.21.15.42:0/4278138841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c19fbf0 con 0x7fa90c102f20 2024-05-27T10:21:35.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.533+0000 7fa909400700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c19b2c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800e970 tx=0x7fa8f800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:35.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.534+0000 7fa902a00700 1 -- 172.21.15.42:0/4278138841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800cbd0 con 0x7fa90c102f20 2024-05-27T10:21:35.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.534+0000 7fa902a00700 1 -- 172.21.15.42:0/4278138841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8f800cd70 con 0x7fa90c102f20 2024-05-27T10:21:35.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.534+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c19fee0 con 0x7fa90c102f20 2024-05-27T10:21:35.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.534+0000 7fa902a00700 1 -- 172.21.15.42:0/4278138841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8005610 con 0x7fa90c102f20 2024-05-27T10:21:35.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.534+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c1a7800 con 0x7fa90c102f20 2024-05-27T10:21:35.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.540+0000 7fa91f943b80 1 Processor -- start 2024-05-27T10:21:35.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa902a00700 1 -- 172.21.15.42:0/4278138841 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa8f8021440 con 0x7fa90c102f20 2024-05-27T10:21:35.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 -- start start 2024-05-27T10:21:35.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb62ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ddcfa91e40 con 0x55ddcfb63410 2024-05-27T10:21:35.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ddcfa819e0 con 0x55ddcf9e4e40 2024-05-27T10:21:35.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.541+0000 7fa91f943b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ddcfa85a20 con 0x55ddcfb63820 2024-05-27T10:21:35.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38136/0 (socket says 172.21.15.42:38136) 2024-05-27T10:21:35.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 -- 172.21.15.42:0/3548954870 learned_addr learned my addr 172.21.15.42:0/3548954870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:35.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa90a800700 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb62ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909400700 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 -- 172.21.15.42:0/3548954870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 msgr2=0x55ddcfb70020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb70020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 -- 172.21.15.42:0/3548954870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 msgr2=0x55ddcfb62ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.542+0000 7fa909e00700 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb62ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa909e00700 1 -- 172.21.15.42:0/3548954870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddcfa4a420 con 0x55ddcfb63410 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa902a00700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa8f00b2aa0 0x7fa8f00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa909e00700 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009c80 tx=0x7fa8fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b4857da88126fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa902a00700 1 -- 172.21.15.42:0/4278138841 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fa8f80e28c0 con 0x7fa90c102f20 2024-05-27T10:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa90a800700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa8f00b2aa0 0x7fa8f00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa8f7400700 1 -- 172.21.15.42:0/3548954870 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011bf0 con 0x55ddcfb63410 2024-05-27T10:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.543+0000 7fa8f7400700 1 -- 172.21.15.42:0/3548954870 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8fc0044c0 con 0x55ddcfb63410 2024-05-27T10:21:35.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.544+0000 7fa8f7400700 1 -- 172.21.15.42:0/3548954870 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc003e20 con 0x55ddcfb63410 2024-05-27T10:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 -- 172.21.15.42:0/3548954870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 msgr2=0x55ddcfb6dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009c80 tx=0x7fa8fc009f90 comp rx=0 tx=0).stop 2024-05-27T10:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa90a800700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa8f00b2aa0 0x7fa8f00b4fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55ddcfb5f690 tx=0x7fa904006020 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 -- 172.21.15.42:0/3548954870 shutdown_connections 2024-05-27T10:21:35.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb70020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb62ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3548954870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dad0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 -- 172.21.15.42:0/3548954870 >> 172.21.15.42:0/3548954870 conn(0x55ddcfa44fc0 msgr2=0x55ddcfb5ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 -- 172.21.15.42:0/3548954870 shutdown_connections 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.545+0000 7fa91f943b80 1 -- 172.21.15.42:0/3548954870 wait complete. 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 -- start start 2024-05-27T10:21:35.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 Processor -- start 2024-05-27T10:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb6d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ddcf987fa0 con 0x55ddcfb63410 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ddcfa22760 con 0x55ddcf9e4e40 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.546+0000 7fa91f943b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ddcfb74da0 con 0x55ddcfb63820 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa91f943b80 10 client.0 _ll_register_callbacks cb 0x55ddcfa2bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38142/0 (socket says 172.21.15.42:38142) 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 -- 172.21.15.42:0/3153258321 learned_addr learned my addr 172.21.15.42:0/3153258321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa90a800700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909e00700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb6d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 -- 172.21.15.42:0/3153258321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 msgr2=0x55ddcfb6e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 -- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 msgr2=0x55ddcfb6d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb6d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.547+0000 7fa909400700 1 -- 172.21.15.42:0/3153258321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddcf9d18d0 con 0x55ddcfb63410 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa909400700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dac0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80a6e60 tx=0x7fa8f80a9670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ae0ae6abe36dea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa8ebe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa8ebe00700 1 -- 172.21.15.42:0/3153258321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80aa2f0 con 0x55ddcfb63410 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa8ebe00700 1 -- 172.21.15.42:0/3153258321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fa8f80aa060 con 0x55ddcfb63410 2024-05-27T10:21:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc0044c0 con 0x55ddcfb63410 2024-05-27T10:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa8ebe00700 1 -- 172.21.15.42:0/3153258321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa8f80a9bf0 con 0x55ddcfb63410 2024-05-27T10:21:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa91f943b80 10 client.15927 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.548+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddcfa52060 con 0x55ddcfb63410 2024-05-27T10:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa8ebe00700 1 -- 172.21.15.42:0/3153258321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8fc0044c0 con 0x55ddcfb63410 2024-05-27T10:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa8ebe00700 1 -- 172.21.15.42:0/3153258321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa8f80b25c0 con 0x55ddcfb63410 2024-05-27T10:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa91f943b80 10 client.15927 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa91f943b80 10 client.15927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:21:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa91f943b80 20 client.15927 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfb81dc0 0x55ddcfb841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa91f943b80 4 client.15927 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.549+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddcfa93d90 con 0x55ddcfb81dc0 2024-05-27T10:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.550+0000 7fa909e00700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfb81dc0 0x55ddcfb841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.551+0000 7fa909e00700 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfb81dc0 0x55ddcfb841e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=482122c2ce5ee44e server_cookie=891abd185080491d in_seq=0 out_seq=0 2024-05-27T10:21:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.551+0000 7fa8ebe00700 10 client.15927 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.553+0000 7fa8ebe00700 1 -- 172.21.15.42:0/3153258321 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ddcfa93d90 con 0x55ddcfb81dc0 2024-05-27T10:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.553+0000 7fa8ebe00700 10 client.15927 handle_command_reply: tid=0 2024-05-27T10:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.553+0000 7fa91f943b80 1 client.15927 shutdown 2024-05-27T10:21:35.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 msgr2=0x55ddcfb6dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dac0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80a6e60 tx=0x7fa8f80a9670 comp rx=0 tx=0).stop 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 shutdown_connections 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfb81dc0 0x55ddcfb841e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb6d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 --2- 172.21.15.42:0/3153258321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6dac0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 >> 172.21.15.42:0/3153258321 conn(0x55ddcfa44fc0 msgr2=0x55ddcfb5dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:35.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa8ebe00700 0 client.15927 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 shutdown_connections 2024-05-27T10:21:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.554+0000 7fa91f943b80 1 -- 172.21.15.42:0/3153258321 wait complete. 2024-05-27T10:21:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.555+0000 7fa91f943b80 20 client.15927 trim_cache size 0 max 16384 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.574+0000 7fa91f943b80 1 Processor -- start 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.574+0000 7fa91f943b80 1 -- start start 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa91f943b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1af1b0 con 0x55ddcfb63410 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa91f943b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1aca40 con 0x55ddcf9e4e40 2024-05-27T10:21:35.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa91f943b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1aa8b0 con 0x55ddcfb63820 2024-05-27T10:21:35.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38146/0 (socket says 172.21.15.42:38146) 2024-05-27T10:21:35.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa909400700 1 -- 172.21.15.42:0/1992442609 learned_addr learned my addr 172.21.15.42:0/1992442609 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:35.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa909e00700 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.575+0000 7fa90a800700 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa909400700 1 -- 172.21.15.42:0/1992442609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 msgr2=0x55ddcfb6fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa909400700 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa909400700 1 -- 172.21.15.42:0/1992442609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 msgr2=0x55ddcfb5eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa909400700 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb5eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa909400700 1 -- 172.21.15.42:0/1992442609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddcfb6baf0 con 0x55ddcfb63410 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa909400700 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6c8a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80a6190 tx=0x7fa8f8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8a3f3672518702 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.576+0000 7fa8f6a00700 1 -- 172.21.15.42:0/1992442609 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80aa2f0 con 0x55ddcfb63410 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.577+0000 7fa8f6a00700 1 -- 172.21.15.42:0/1992442609 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8f80a5560 con 0x55ddcfb63410 2024-05-27T10:21:35.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.577+0000 7fa8f6a00700 1 -- 172.21.15.42:0/1992442609 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a57a0 con 0x55ddcfb63410 2024-05-27T10:21:35.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 -- 172.21.15.42:0/1992442609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 msgr2=0x55ddcfb6c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6c8a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80a6190 tx=0x7fa8f8009520 comp rx=0 tx=0).stop 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 -- 172.21.15.42:0/1992442609 shutdown_connections 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfb6fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb5eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 --2- 172.21.15.42:0/1992442609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb6c8a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 -- 172.21.15.42:0/1992442609 >> 172.21.15.42:0/1992442609 conn(0x55ddcfa44fc0 msgr2=0x55ddcfb6d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 -- 172.21.15.42:0/1992442609 shutdown_connections 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.578+0000 7fa91f943b80 1 -- 172.21.15.42:0/1992442609 wait complete. 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.579+0000 7fa91f943b80 1 -- start start 2024-05-27T10:21:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.579+0000 7fa91f943b80 1 Processor -- start 2024-05-27T10:21:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfbea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1aaf00 con 0x55ddcfb63410 2024-05-27T10:21:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1ac370 con 0x55ddcf9e4e40 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1ab5d0 con 0x55ddcfb63820 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa91f943b80 10 client.0 _ll_register_callbacks cb 0x55ddcfb7e620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa909400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb75630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38150/0 (socket says 172.21.15.42:38150) 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa909400700 1 -- 172.21.15.42:0/2709569549 learned_addr learned my addr 172.21.15.42:0/2709569549 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.580+0000 7fa909e00700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa90a800700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfbea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa909400700 1 -- 172.21.15.42:0/2709569549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 msgr2=0x55ddcfbea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa909400700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfbea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa909400700 1 -- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 msgr2=0x55ddcfb750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa909400700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb750f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa909400700 1 -- 172.21.15.42:0/2709569549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddcfb75f30 con 0x55ddcfb63410 2024-05-27T10:21:35.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa90a800700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfbea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa909400700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb75630 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80a6e60 tx=0x7fa8f80a9670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aa9418371bbb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.581+0000 7fa8ebe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa8ebe00700 1 -- 172.21.15.42:0/2709569549 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f808c060 con 0x55ddcfb63410 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa8ebe00700 1 -- 172.21.15.42:0/2709569549 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fa8f80a57a0 con 0x55ddcfb63410 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f80a5560 con 0x55ddcfb63410 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa8ebe00700 1 -- 172.21.15.42:0/2709569549 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa8f80aa2f0 con 0x55ddcfb63410 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa91f943b80 10 client.15933 fetch_fsmap learned FSMap version 26 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddcfbebde0 con 0x55ddcfb63410 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.582+0000 7fa8ebe00700 1 -- 172.21.15.42:0/2709569549 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8f80a5560 con 0x55ddcfb63410 2024-05-27T10:21:35.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa8ebe00700 1 -- 172.21.15.42:0/2709569549 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa8f80af130 con 0x55ddcfb63410 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa91f943b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa91f943b80 10 client.15933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa91f943b80 20 client.15933 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa91f943b80 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfbec150 0x55ddcfbee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa91f943b80 4 client.15933 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddcfa8d570 con 0x55ddcfbec150 2024-05-27T10:21:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.583+0000 7fa909e00700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfbec150 0x55ddcfbee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:35.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.585+0000 7fa909e00700 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfbec150 0x55ddcfbee570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=286a59699a7f81c0 server_cookie=14fa1183073f7009 in_seq=0 out_seq=0 2024-05-27T10:21:35.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.585+0000 7fa8ebe00700 10 client.15933 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:35.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.585+0000 7fa8ebe00700 1 -- 172.21.15.42:0/2709569549 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddcfa8d570 con 0x55ddcfbec150 2024-05-27T10:21:35.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.585+0000 7fa8ebe00700 10 client.15933 handle_command_reply: tid=0 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.585+0000 7fa91f943b80 1 client.15933 shutdown 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 msgr2=0x55ddcfb75630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb75630 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80a6e60 tx=0x7fa8f80a9670 comp rx=0 tx=0).stop 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 shutdown_connections 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x55ddcfbec150 0x55ddcfbee570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ddcfb63820 0x55ddcfbea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ddcf9e4e40 0x55ddcfb750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 --2- 172.21.15.42:0/2709569549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddcfb63410 0x55ddcfb75630 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 >> 172.21.15.42:0/2709569549 conn(0x55ddcfa44fc0 msgr2=0x55ddcfb6c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa8ebe00700 0 client.15933 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:21:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 shutdown_connections 2024-05-27T10:21:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 1 -- 172.21.15.42:0/2709569549 wait complete. 2024-05-27T10:21:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.586+0000 7fa91f943b80 20 client.15933 trim_cache size 0 max 16384 2024-05-27T10:21:35.588 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:21:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa8f00b2aa0 msgr2=0x7fa8f00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa8f00b2aa0 0x7fa8f00b4fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55ddcfb5f690 tx=0x7fa904006020 comp rx=0 tx=0).stop 2024-05-27T10:21:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 msgr2=0x7fa90c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c19b2c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800e970 tx=0x7fa8f800ed30 comp rx=0 tx=0).stop 2024-05-27T10:21:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 shutdown_connections 2024-05-27T10:21:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa8f00b2aa0 0x7fa8f00b4fb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa90c109250 0x7fa90c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90c1025d0 0x7fa90c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 --2- 172.21.15.42:0/4278138841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c102f20 0x7fa90c19b2c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:35.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 >> 172.21.15.42:0/4278138841 conn(0x7fa90c078360 msgr2=0x7fa90c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:35.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 shutdown_connections 2024-05-27T10:21:35.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:35.587+0000 7fa90b200700 1 -- 172.21.15.42:0/4278138841 wait complete. 2024-05-27T10:21:35.597 INFO:tasks.workunit.client.0.smithi042.stdout:checking for style of include used by make... GNU 2024-05-27T10:21:36.323 INFO:tasks.workunit.client.0.smithi042.stdout:checking dependency style of gcc... gcc3 2024-05-27T10:21:36.325 INFO:tasks.workunit.client.0.smithi042.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-27T10:21:36.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T10:21:36.795 INFO:tasks.workunit.client.0.smithi042.stdout:checking how to run the C preprocessor... gcc -E 2024-05-27T10:21:37.075 INFO:tasks.workunit.client.0.smithi042.stdout:checking for egrep... grep -E 2024-05-27T10:21:37.138 INFO:tasks.workunit.client.0.smithi042.stdout:checking for AIX... no 2024-05-27T10:21:37.165 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:37.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:36 smithi104 ceph-mon[34812]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 80 KiB/s wr, 22 op/s 2024-05-27T10:21:37.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:36 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:37.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:36 smithi143 ceph-mon[35545]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 80 KiB/s wr, 22 op/s 2024-05-27T10:21:37.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:36 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:37.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:36 smithi042 ceph-mon[31661]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 80 KiB/s wr, 22 op/s 2024-05-27T10:21:37.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:36 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:37.501 INFO:tasks.workunit.client.0.smithi042.stdout:checking for library containing strerror... none required 2024-05-27T10:21:38.397 INFO:tasks.workunit.client.0.smithi042.stdout:checking for ANSI C header files... yes 2024-05-27T10:21:38.652 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-27T10:21:38.946 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/types.h... yes 2024-05-27T10:21:39.166 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:38 smithi042 ceph-mon[31661]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 80 KiB/s wr, 18 op/s 2024-05-27T10:21:39.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:38 smithi104 ceph-mon[34812]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 80 KiB/s wr, 18 op/s 2024-05-27T10:21:39.235 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/stat.h... yes 2024-05-27T10:21:39.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:38 smithi143 ceph-mon[35545]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 80 KiB/s wr, 18 op/s 2024-05-27T10:21:39.521 INFO:tasks.workunit.client.0.smithi042.stdout:checking for stdlib.h... yes 2024-05-27T10:21:39.793 INFO:tasks.workunit.client.0.smithi042.stdout:checking for string.h... yes 2024-05-27T10:21:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 -- 172.21.15.42:0/3759757756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 msgr2=0x7f9af81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 --2- 172.21.15.42:0/3759757756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af81049f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0099c0 tx=0x7f9aec009cd0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 -- 172.21.15.42:0/3759757756 shutdown_connections 2024-05-27T10:21:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 --2- 172.21.15.42:0/3759757756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9af8109860 0x7f9af81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 --2- 172.21.15.42:0/3759757756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af81049f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 --2- 172.21.15.42:0/3759757756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af8108f10 0x7f9af8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 -- 172.21.15.42:0/3759757756 >> 172.21.15.42:0/3759757756 conn(0x7f9af8100030 msgr2=0x7f9af8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 -- 172.21.15.42:0/3759757756 shutdown_connections 2024-05-27T10:21:39.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.821+0000 7f9affa00700 1 -- 172.21.15.42:0/3759757756 wait complete. 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 Processor -- start 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 -- start start 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af8108f10 0x7f9af819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9af8109860 0x7f9af81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81143b0 con 0x7f9af81045e0 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8114230 con 0x7f9af8109860 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.823+0000 7f9affa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8114530 con 0x7f9af8108f10 2024-05-27T10:21:39.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38174/0 (socket says 172.21.15.42:38174) 2024-05-27T10:21:39.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 -- 172.21.15.42:0/3286490198 learned_addr learned my addr 172.21.15.42:0/3286490198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afda00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9af8109860 0x7f9af81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9af7e00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af8108f10 0x7f9af819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 -- 172.21.15.42:0/3286490198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af8108f10 msgr2=0x7f9af819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af8108f10 0x7f9af819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 -- 172.21.15.42:0/3286490198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9af8109860 msgr2=0x7f9af81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9af8109860 0x7f9af81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.824+0000 7f9afd000700 1 -- 172.21.15.42:0/3286490198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af81a1ef0 con 0x7f9af81045e0 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.825+0000 7f9afd000700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af819d020 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00f690 tx=0x7f9aec00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:39.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.825+0000 7f9af5600700 1 -- 172.21.15.42:0/3286490198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec027070 con 0x7f9af81045e0 2024-05-27T10:21:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.825+0000 7f9af5600700 1 -- 172.21.15.42:0/3286490198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9aec0056d0 con 0x7f9af81045e0 2024-05-27T10:21:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.825+0000 7f9af5600700 1 -- 172.21.15.42:0/3286490198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0173f0 con 0x7f9af81045e0 2024-05-27T10:21:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.825+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81a2180 con 0x7f9af81045e0 2024-05-27T10:21:39.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.825+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81a2660 con 0x7f9af81045e0 2024-05-27T10:21:39.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.829+0000 7f9af5600700 1 -- 172.21.15.42:0/3286490198 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9aec02a070 con 0x7f9af81045e0 2024-05-27T10:21:39.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.830+0000 7f9b0e4e1b80 1 Processor -- start 2024-05-27T10:21:39.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.830+0000 7f9b0e4e1b80 1 -- start start 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.830+0000 7f9af5600700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9ae00b29f0 0x7f9ae00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7bedb0 0x5570ed7bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5570ed6ece40 con 0x5570ed7c4d30 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5570ed6dc9e0 con 0x5570ed7bedb0 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9af5600700 1 -- 172.21.15.42:0/3286490198 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9aec0e3520 con 0x7f9af81045e0 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5570ed6e0a20 con 0x5570ed63fe40 2024-05-27T10:21:39.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9afda00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9ae00b29f0 0x7f9ae00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9af7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9af7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7be870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38176/0 (socket says 172.21.15.42:38176) 2024-05-27T10:21:39.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9af7e00700 1 -- 172.21.15.42:0/1849475971 learned_addr learned my addr 172.21.15.42:0/1849475971 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:39.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.831+0000 7f9afda00700 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7bedb0 0x5570ed7bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9afd000700 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9af7e00700 1 -- 172.21.15.42:0/1849475971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 msgr2=0x5570ed7c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9af7e00700 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9af7e00700 1 -- 172.21.15.42:0/1849475971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7bedb0 msgr2=0x5570ed7bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9af7e00700 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7bedb0 0x5570ed7bf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9af7e00700 1 -- 172.21.15.42:0/1849475971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ed6a5420 con 0x5570ed7c4d30 2024-05-27T10:21:39.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9afda00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9ae00b29f0 0x7f9ae00b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9af00060b0 tx=0x7f9af00089f0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.832+0000 7f9af7e00700 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7be870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40077c0 tx=0x7f9ae4007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28f7d90eabef7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.833+0000 7f9aea000700 1 -- 172.21.15.42:0/1849475971 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40125f0 con 0x5570ed7c4d30 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.833+0000 7f9aea000700 1 -- 172.21.15.42:0/1849475971 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ae4018070 con 0x5570ed7c4d30 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.833+0000 7f9aea000700 1 -- 172.21.15.42:0/1849475971 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4013b70 con 0x5570ed7c4d30 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1849475971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 msgr2=0x5570ed7be870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7be870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40077c0 tx=0x7f9ae4007ad0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1849475971 shutdown_connections 2024-05-27T10:21:39.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7bedb0 0x5570ed7bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1849475971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7be870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1849475971 >> 172.21.15.42:0/1849475971 conn(0x5570ed69ffc0 msgr2=0x5570ed7b98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1849475971 shutdown_connections 2024-05-27T10:21:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.834+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1849475971 wait complete. 2024-05-27T10:21:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.835+0000 7f9b0e4e1b80 1 -- start start 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.835+0000 7f9b0e4e1b80 1 Processor -- start 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5570ed5e2fa0 con 0x5570ed7c4d30 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5570ed67d760 con 0x5570ed63fe40 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5570ed7cb800 con 0x5570ed7bedb0 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9b0e4e1b80 10 client.0 _ll_register_callbacks cb 0x5570ed686c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:39.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9afda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9afda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38188/0 (socket says 172.21.15.42:38188) 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9afda00700 1 -- 172.21.15.42:0/818089099 learned_addr learned my addr 172.21.15.42:0/818089099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.836+0000 7f9af7e00700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afd000700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afda00700 1 -- 172.21.15.42:0/818089099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 msgr2=0x5570ed7c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afda00700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afda00700 1 -- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 msgr2=0x5570ed7c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afda00700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afda00700 1 -- 172.21.15.42:0/818089099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ed62c8d0 con 0x5570ed7c4d30 2024-05-27T10:21:39.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9af7e00700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9afda00700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7c8fc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9af0019ce0 tx=0x7f9af000c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff61764ab1cdb3a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.837+0000 7f9adea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9adea00700 1 -- 172.21.15.42:0/818089099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0026060 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9adea00700 1 -- 172.21.15.42:0/818089099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9af0021600 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae4018070 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9adea00700 1 -- 172.21.15.42:0/818089099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9af0013040 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9b0e4e1b80 10 client.15945 fetch_fsmap learned FSMap version 27 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570ed6ad060 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.838+0000 7f9adea00700 1 -- 172.21.15.42:0/818089099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ae4018070 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9adea00700 1 -- 172.21.15.42:0/818089099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7f9af0016030 con 0x5570ed7c4d30 2024-05-27T10:21:39.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9b0e4e1b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9b0e4e1b80 10 client.15945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9b0e4e1b80 20 client.15945 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed7d87f0 0x5570ed7dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9b0e4e1b80 4 client.15945 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.839+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570ed6eed90 con 0x5570ed7d87f0 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.840+0000 7f9afd000700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed7d87f0 0x5570ed7dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.841+0000 7f9afd000700 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed7d87f0 0x5570ed7dac10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d82daed1ddad6cf0 server_cookie=cc9895a3b8b14e1d in_seq=0 out_seq=0 2024-05-27T10:21:39.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.841+0000 7f9adea00700 10 client.15945 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:39.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.842+0000 7f9adea00700 1 -- 172.21.15.42:0/818089099 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5570ed6eed90 con 0x5570ed7d87f0 2024-05-27T10:21:39.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.842+0000 7f9adea00700 10 client.15945 handle_command_reply: tid=0 2024-05-27T10:21:39.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 client.15945 shutdown 2024-05-27T10:21:39.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 msgr2=0x5570ed7c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7c8fc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9af0019ce0 tx=0x7f9af000c050 comp rx=0 tx=0).stop 2024-05-27T10:21:39.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 shutdown_connections 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed7d87f0 0x5570ed7dac10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7c8fc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/818089099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 >> 172.21.15.42:0/818089099 conn(0x5570ed69ffc0 msgr2=0x5570ed7b9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9adea00700 0 client.15945 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 shutdown_connections 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.843+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/818089099 wait complete. 2024-05-27T10:21:39.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.844+0000 7f9b0e4e1b80 20 client.15945 trim_cache size 0 max 16384 2024-05-27T10:21:39.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.862+0000 7f9b0e4e1b80 1 Processor -- start 2024-05-27T10:21:39.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 -- start start 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 0x5570ed7ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7c4d30 0x5570ed7c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b3640 con 0x5570ed7bedb0 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b0ed0 con 0x5570ed7c4d30 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81aed20 con 0x5570ed63fe40 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9af7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 0x5570ed7ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9af7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 0x5570ed7ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38196/0 (socket says 172.21.15.42:38196) 2024-05-27T10:21:39.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.863+0000 7f9af7e00700 1 -- 172.21.15.42:0/1456179063 learned_addr learned my addr 172.21.15.42:0/1456179063 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9afda00700 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7c4d30 0x5570ed7c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9afd000700 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9af7e00700 1 -- 172.21.15.42:0/1456179063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 msgr2=0x5570ed7be870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9af7e00700 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7be870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9af7e00700 1 -- 172.21.15.42:0/1456179063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7c4d30 msgr2=0x5570ed7c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9af7e00700 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7c4d30 0x5570ed7c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9af7e00700 1 -- 172.21.15.42:0/1456179063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ed7c5140 con 0x5570ed7bedb0 2024-05-27T10:21:39.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.864+0000 7f9af7e00700 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 0x5570ed7ca9e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4000c00 tx=0x7f9ae40071e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac4af9b02b7c2f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.865+0000 7f9ae9600700 1 -- 172.21.15.42:0/1456179063 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4012820 con 0x5570ed7bedb0 2024-05-27T10:21:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.865+0000 7f9ae9600700 1 -- 172.21.15.42:0/1456179063 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ae4018040 con 0x5570ed7bedb0 2024-05-27T10:21:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.865+0000 7f9ae9600700 1 -- 172.21.15.42:0/1456179063 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4028d40 con 0x5570ed7bedb0 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1456179063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 msgr2=0x5570ed7ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 0x5570ed7ca9e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4000c00 tx=0x7f9ae40071e0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1456179063 shutdown_connections 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed7c4d30 0x5570ed7c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed63fe40 0x5570ed7be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/1456179063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7bedb0 0x5570ed7ca9e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1456179063 >> 172.21.15.42:0/1456179063 conn(0x5570ed69ffc0 msgr2=0x5570ed7c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1456179063 shutdown_connections 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.866+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/1456179063 wait complete. 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 -- start start 2024-05-27T10:21:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 Processor -- start 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7e0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81af390 con 0x5570ed7c4d30 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b0800 con 0x5570ed63fe40 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.867+0000 7f9b0e4e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81afa60 con 0x5570ed7bedb0 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9b0e4e1b80 10 client.0 _ll_register_callbacks cb 0x5570ed686c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7e0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7e0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38204/0 (socket says 172.21.15.42:38204) 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 -- 172.21.15.42:0/582673931 learned_addr learned my addr 172.21.15.42:0/582673931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afd000700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9af7e00700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 -- 172.21.15.42:0/582673931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 msgr2=0x5570ed7dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 -- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 msgr2=0x5570ed7d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9afda00700 1 -- 172.21.15.42:0/582673931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ed7e0910 con 0x5570ed7c4d30 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.868+0000 7f9af7e00700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9afda00700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7e0200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9af0007400 tx=0x7f9af0020ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9cf71d81d74ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9adea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9adea00700 1 -- 172.21.15.42:0/582673931 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0008df0 con 0x5570ed7c4d30 2024-05-27T10:21:39.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9adea00700 1 -- 172.21.15.42:0/582673931 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9af0003a00 con 0x5570ed7c4d30 2024-05-27T10:21:39.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae4018040 con 0x5570ed7c4d30 2024-05-27T10:21:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9adea00700 1 -- 172.21.15.42:0/582673931 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9af0017e20 con 0x5570ed7c4d30 2024-05-27T10:21:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9b0e4e1b80 10 client.15951 fetch_fsmap learned FSMap version 27 2024-05-27T10:21:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.869+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570ed8431a0 con 0x5570ed7c4d30 2024-05-27T10:21:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9adea00700 1 -- 172.21.15.42:0/582673931 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ae4018040 con 0x5570ed7c4d30 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9adea00700 1 -- 172.21.15.42:0/582673931 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7f9af0004540 con 0x5570ed7c4d30 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9b0e4e1b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9b0e4e1b80 10 client.15951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9b0e4e1b80 20 client.15951 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed843930 0x5570ed845d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9b0e4e1b80 4 client.15951 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.870+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570ed6e8570 con 0x5570ed843930 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.871+0000 7f9afd000700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed843930 0x5570ed845d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9afd000700 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed843930 0x5570ed845d50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f3b2099e8c7f4db server_cookie=f7ef987a464c935e in_seq=0 out_seq=0 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9adea00700 10 client.15951 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9adea00700 1 -- 172.21.15.42:0/582673931 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570ed6e8570 con 0x5570ed843930 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9adea00700 10 client.15951 handle_command_reply: tid=0 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9b0e4e1b80 1 client.15951 shutdown 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 msgr2=0x5570ed7e0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.872+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7e0200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9af0007400 tx=0x7f9af0020ac0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 shutdown_connections 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5570ed843930 0x5570ed845d50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5570ed7c4d30 0x5570ed7e0200 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570ed63fe40 0x5570ed7d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 --2- 172.21.15.42:0/582673931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5570ed7bedb0 0x5570ed7dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 >> 172.21.15.42:0/582673931 conn(0x5570ed69ffc0 msgr2=0x5570ed7c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9adea00700 0 client.15951 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:21:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 shutdown_connections 2024-05-27T10:21:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 1 -- 172.21.15.42:0/582673931 wait complete. 2024-05-27T10:21:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.873+0000 7f9b0e4e1b80 20 client.15951 trim_cache size 0 max 16384 2024-05-27T10:21:39.877 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9ae00b29f0 msgr2=0x7f9ae00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9ae00b29f0 0x7f9ae00b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9af00060b0 tx=0x7f9af00089f0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 msgr2=0x7f9af819d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af819d020 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00f690 tx=0x7f9aec00f770 comp rx=0 tx=0).stop 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 shutdown_connections 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9ae00b29f0 0x7f9ae00b4f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9af8109860 0x7f9af81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9af81045e0 0x7f9af819d020 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 --2- 172.21.15.42:0/3286490198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af8108f10 0x7f9af819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 >> 172.21.15.42:0/3286490198 conn(0x7f9af8100030 msgr2=0x7f9af810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 shutdown_connections 2024-05-27T10:21:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:39.874+0000 7f9affa00700 1 -- 172.21.15.42:0/3286490198 wait complete. 2024-05-27T10:21:40.057 INFO:tasks.workunit.client.0.smithi042.stdout:checking for memory.h... yes 2024-05-27T10:21:40.351 INFO:tasks.workunit.client.0.smithi042.stdout:checking for strings.h... yes 2024-05-27T10:21:40.646 INFO:tasks.workunit.client.0.smithi042.stdout:checking for inttypes.h... yes 2024-05-27T10:21:40.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T10:21:40.936 INFO:tasks.workunit.client.0.smithi042.stdout:checking for stdint.h... yes 2024-05-27T10:21:41.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:40 smithi104 ceph-mon[34812]: pgmap v91: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 163 KiB/s wr, 45 op/s 2024-05-27T10:21:41.219 INFO:tasks.workunit.client.0.smithi042.stdout:checking for unistd.h... yes 2024-05-27T10:21:41.290 INFO:tasks.workunit.client.0.smithi042.stdout:checking for memory.h... (cached) yes 2024-05-27T10:21:41.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:40 smithi143 ceph-mon[35545]: pgmap v91: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 163 KiB/s wr, 45 op/s 2024-05-27T10:21:41.325 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:41.326 INFO:tasks.workunit.client.0.smithi042.stdout:checking for strings.h... (cached) yes 2024-05-27T10:21:41.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:40 smithi042 ceph-mon[31661]: pgmap v91: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 163 KiB/s wr, 45 op/s 2024-05-27T10:21:41.359 INFO:tasks.workunit.client.0.smithi042.stdout:checking for unistd.h... (cached) yes 2024-05-27T10:21:41.690 INFO:tasks.workunit.client.0.smithi042.stdout:checking locale.h usability... yes 2024-05-27T10:21:41.844 INFO:tasks.workunit.client.0.smithi042.stdout:checking locale.h presence... yes 2024-05-27T10:21:41.851 INFO:tasks.workunit.client.0.smithi042.stdout:checking for locale.h... yes 2024-05-27T10:21:41.916 INFO:tasks.workunit.client.0.smithi042.stdout:checking for stdlib.h... (cached) yes 2024-05-27T10:21:41.954 INFO:tasks.workunit.client.0.smithi042.stdout:checking for string.h... (cached) yes 2024-05-27T10:21:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:41 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:42.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:41 smithi104 ceph-mon[34812]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 163 KiB/s wr, 43 op/s 2024-05-27T10:21:42.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:41 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:42.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:41 smithi143 ceph-mon[35545]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 163 KiB/s wr, 43 op/s 2024-05-27T10:21:42.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:41 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:42.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:41 smithi042 ceph-mon[31661]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 163 KiB/s wr, 43 op/s 2024-05-27T10:21:42.426 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/socket.h usability... yes 2024-05-27T10:21:42.588 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/socket.h presence... yes 2024-05-27T10:21:42.595 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/socket.h... yes 2024-05-27T10:21:42.896 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/time.h usability... yes 2024-05-27T10:21:43.056 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/time.h presence... yes 2024-05-27T10:21:43.060 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/time.h... yes 2024-05-27T10:21:43.342 INFO:tasks.workunit.client.0.smithi042.stdout:checking fcntl.h usability... yes 2024-05-27T10:21:43.474 INFO:tasks.workunit.client.0.smithi042.stdout:checking fcntl.h presence... yes 2024-05-27T10:21:43.479 INFO:tasks.workunit.client.0.smithi042.stdout:checking for fcntl.h... yes 2024-05-27T10:21:43.789 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/fcntl.h usability... yes 2024-05-27T10:21:43.942 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/fcntl.h presence... yes 2024-05-27T10:21:43.946 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/fcntl.h... yes 2024-05-27T10:21:44.024 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:44 smithi042 ceph-mon[31661]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 163 KiB/s wr, 43 op/s 2024-05-27T10:21:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.181+0000 7fe8cfa00700 1 -- 172.21.15.42:0/208308359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c81033a0 msgr2=0x7fe8c8113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.181+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/208308359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c81033a0 0x7fe8c8113020 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000b0a0 tx=0x7fe8c000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.181+0000 7fe8cfa00700 1 -- 172.21.15.42:0/208308359 shutdown_connections 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.181+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/208308359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c81033a0 0x7fe8c8113020 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.182+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/208308359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c8105230 0x7fe8c8105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.182+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/208308359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8102920 0x7fe8c8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.182+0000 7fe8cfa00700 1 -- 172.21.15.42:0/208308359 >> 172.21.15.42:0/208308359 conn(0x7fe8c80fe6e0 msgr2=0x7fe8c8100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.182+0000 7fe8cfa00700 1 -- 172.21.15.42:0/208308359 shutdown_connections 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.182+0000 7fe8cfa00700 1 -- 172.21.15.42:0/208308359 wait complete. 2024-05-27T10:21:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.183+0000 7fe8cfa00700 1 Processor -- start 2024-05-27T10:21:44.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.183+0000 7fe8cfa00700 1 -- start start 2024-05-27T10:21:44.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cfa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 0x7fe8c819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cfa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c81033a0 0x7fe8c819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cfa00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8105230 0x7fe8c81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cfa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8116560 con 0x7fe8c8102920 2024-05-27T10:21:44.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cfa00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81163e0 con 0x7fe8c8105230 2024-05-27T10:21:44.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cfa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81166e0 con 0x7fe8c81033a0 2024-05-27T10:21:44.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 0x7fe8c819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 0x7fe8c819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33212/0 (socket says 172.21.15.42:33212) 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.184+0000 7fe8cd000700 1 -- 172.21.15.42:0/4118155850 learned_addr learned my addr 172.21.15.42:0/4118155850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c81033a0 0x7fe8c819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cda00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8105230 0x7fe8c81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cd000700 1 -- 172.21.15.42:0/4118155850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c81033a0 msgr2=0x7fe8c819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cd000700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c81033a0 0x7fe8c819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cd000700 1 -- 172.21.15.42:0/4118155850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8105230 msgr2=0x7fe8c81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cd000700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8105230 0x7fe8c81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cd000700 1 -- 172.21.15.42:0/4118155850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c81a1dd0 con 0x7fe8c8102920 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c81033a0 0x7fe8c819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:21:44.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.185+0000 7fe8cd000700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 0x7fe8c819cf50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d950 tx=0x7fe8bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:44.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.186+0000 7fe8c5600700 1 -- 172.21.15.42:0/4118155850 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc017070 con 0x7fe8c8102920 2024-05-27T10:21:44.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.186+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c81a20c0 con 0x7fe8c8102920 2024-05-27T10:21:44.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.186+0000 7fe8c5600700 1 -- 172.21.15.42:0/4118155850 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8bc00de30 con 0x7fe8c8102920 2024-05-27T10:21:44.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.186+0000 7fe8c5600700 1 -- 172.21.15.42:0/4118155850 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc003aa0 con 0x7fe8c8102920 2024-05-27T10:21:44.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.186+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a99e0 con 0x7fe8c8102920 2024-05-27T10:21:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.190+0000 7fe8c5600700 1 -- 172.21.15.42:0/4118155850 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe8bc003c40 con 0x7fe8c8102920 2024-05-27T10:21:44.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.191+0000 7fe8de4f0b80 1 Processor -- start 2024-05-27T10:21:44.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.191+0000 7fe8de4f0b80 1 -- start start 2024-05-27T10:21:44.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e46330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e46870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e53110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ea8d74e40 con 0x557ea8cc7e40 2024-05-27T10:21:44.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ea8d649e0 con 0x557ea8e46e70 2024-05-27T10:21:44.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557ea8d68a20 con 0x557ea8db39f0 2024-05-27T10:21:44.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.192+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e46870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.193+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e46870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33222/0 (socket says 172.21.15.42:33222) 2024-05-27T10:21:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.193+0000 7fe8cd000700 1 -- 172.21.15.42:0/2322006646 learned_addr learned my addr 172.21.15.42:0/2322006646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.193+0000 7fe8cda00700 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e46330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.193+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e53110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.193+0000 7fe8c5600700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe8b00b29f0 0x7fe8b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8c5600700 1 -- 172.21.15.42:0/4118155850 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe8bc0e2160 con 0x7fe8c8102920 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cd000700 1 -- 172.21.15.42:0/2322006646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 msgr2=0x557ea8e46330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cd000700 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e46330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cd000700 1 -- 172.21.15.42:0/2322006646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 msgr2=0x557ea8e53110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cd000700 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e53110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cd000700 1 -- 172.21.15.42:0/2322006646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea8d2d420 con 0x557ea8cc7e40 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cda00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe8b00b29f0 0x7fe8b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.194+0000 7fe8cd000700 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e46870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0df870 tx=0x7fe8bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870b5fb25b441584 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.195+0000 7fe8ba000700 1 -- 172.21.15.42:0/2322006646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0a8de0 con 0x557ea8cc7e40 2024-05-27T10:21:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.195+0000 7fe8ba000700 1 -- 172.21.15.42:0/2322006646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8bc0a8fa0 con 0x557ea8cc7e40 2024-05-27T10:21:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.195+0000 7fe8ba000700 1 -- 172.21.15.42:0/2322006646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0a8c00 con 0x557ea8cc7e40 2024-05-27T10:21:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2322006646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 msgr2=0x557ea8e46870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8cda00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe8b00b29f0 0x7fe8b00b4f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00062a0 tx=0x7fe8c000b3b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e46870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0df870 tx=0x7fe8bc004750 comp rx=0 tx=0).stop 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2322006646 shutdown_connections 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e53110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e46330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2322006646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e46870 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2322006646 >> 172.21.15.42:0/2322006646 conn(0x557ea8d61390 msgr2=0x557ea8e424c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2322006646 shutdown_connections 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.196+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2322006646 wait complete. 2024-05-27T10:21:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.197+0000 7fe8de4f0b80 1 -- start start 2024-05-27T10:21:44.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.197+0000 7fe8de4f0b80 1 Processor -- start 2024-05-27T10:21:44.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8cc7e40 0x557ea8e50730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e50c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 0x557ea8e511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ea8c6afa0 con 0x557ea8e46e70 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ea8d05760 con 0x557ea8cc7e40 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557ea8e53b20 con 0x557ea8db39f0 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8de4f0b80 10 client.0 _ll_register_callbacks cb 0x557ea8d0ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8cda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 0x557ea8e511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8cda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 0x557ea8e511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33234/0 (socket says 172.21.15.42:33234) 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8cda00700 1 -- 172.21.15.42:0/764947698 learned_addr learned my addr 172.21.15.42:0/764947698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e50c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.198+0000 7fe8cd000700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8cc7e40 0x557ea8e50730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8cda00700 1 -- 172.21.15.42:0/764947698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 msgr2=0x557ea8e50c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8cda00700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e50c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8cda00700 1 -- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8cc7e40 msgr2=0x557ea8e50730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8cda00700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8cc7e40 0x557ea8e50730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8cda00700 1 -- 172.21.15.42:0/764947698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea8cb48d0 con 0x557ea8e46e70 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8cda00700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 0x557ea8e511b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007840 tx=0x7fe8c00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593f534e33f0a256 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8aea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8aea00700 1 -- 172.21.15.42:0/764947698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c002f060 con 0x557ea8e46e70 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8aea00700 1 -- 172.21.15.42:0/764947698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe8c0020690 con 0x557ea8e46e70 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.199+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8bc0a8fa0 con 0x557ea8e46e70 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8aea00700 1 -- 172.21.15.42:0/764947698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8c0020890 con 0x557ea8e46e70 2024-05-27T10:21:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 10 client.15963 fetch_fsmap learned FSMap version 28 2024-05-27T10:21:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ea8d35060 con 0x557ea8e46e70 2024-05-27T10:21:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8aea00700 1 -- 172.21.15.42:0/764947698 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8bc0a8fa0 con 0x557ea8e46e70 2024-05-27T10:21:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8aea00700 1 -- 172.21.15.42:0/764947698 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fe8c0011070 con 0x557ea8e46e70 2024-05-27T10:21:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T10:21:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 20 client.15963 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8e60ab0 0x557ea8e62f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 4 client.15963 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.200+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea8d76d90 con 0x557ea8e60ab0 2024-05-27T10:21:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.201+0000 7fe8cd000700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8e60ab0 0x557ea8e62f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.202+0000 7fe8cd000700 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8e60ab0 0x557ea8e62f50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbae33bfa503326 server_cookie=bfe6c62e9022289f in_seq=0 out_seq=0 2024-05-27T10:21:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.202+0000 7fe8aea00700 10 client.15963 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8aea00700 1 -- 172.21.15.42:0/764947698 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ea8d76d90 con 0x557ea8e60ab0 2024-05-27T10:21:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8aea00700 10 client.15963 handle_command_reply: tid=0 2024-05-27T10:21:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 client.15963 shutdown 2024-05-27T10:21:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 msgr2=0x557ea8e511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 0x557ea8e511b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007840 tx=0x7fe8c00099e0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 shutdown_connections 2024-05-27T10:21:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8e60ab0 0x557ea8e62f50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8e46e70 0x557ea8e511b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8cc7e40 0x557ea8e50730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/764947698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e50c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 >> 172.21.15.42:0/764947698 conn(0x557ea8d61390 msgr2=0x557ea8e46480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.204+0000 7fe8aea00700 0 client.15963 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.205+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 shutdown_connections 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.205+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/764947698 wait complete. 2024-05-27T10:21:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.205+0000 7fe8de4f0b80 20 client.15963 trim_cache size 0 max 16384 2024-05-27T10:21:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.223+0000 7fe8de4f0b80 1 Processor -- start 2024-05-27T10:21:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.223+0000 7fe8de4f0b80 1 -- start start 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8cc7e40 0x557ea8e46890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 0x557ea8e42590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e5cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b3830 con 0x557ea8db39f0 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b10c0 con 0x557ea8e46e70 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81aef30 con 0x557ea8cc7e40 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 0x557ea8e42590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 0x557ea8e42590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33244/0 (socket says 172.21.15.42:33244) 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.224+0000 7fe8c7e00700 1 -- 172.21.15.42:0/493828061 learned_addr learned my addr 172.21.15.42:0/493828061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8cd000700 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8cc7e40 0x557ea8e46890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8cda00700 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e5cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8c7e00700 1 -- 172.21.15.42:0/493828061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8cc7e40 msgr2=0x557ea8e46890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8cc7e40 0x557ea8e46890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8c7e00700 1 -- 172.21.15.42:0/493828061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 msgr2=0x557ea8e5cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e5cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8c7e00700 1 -- 172.21.15.42:0/493828061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea8e4eca0 con 0x557ea8db39f0 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8cda00700 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e5cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.225+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 0x557ea8e42590 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009e40 tx=0x7fe8b4013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ebe2f6537beaf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.226+0000 7fe8b9600700 1 -- 172.21.15.42:0/493828061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4004220 con 0x557ea8db39f0 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.226+0000 7fe8b9600700 1 -- 172.21.15.42:0/493828061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b40043c0 con 0x557ea8db39f0 2024-05-27T10:21:44.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.226+0000 7fe8b9600700 1 -- 172.21.15.42:0/493828061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4005630 con 0x557ea8db39f0 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/493828061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 msgr2=0x557ea8e42590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 0x557ea8e42590 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009e40 tx=0x7fe8b4013740 comp rx=0 tx=0).stop 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/493828061 shutdown_connections 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8e5cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8cc7e40 0x557ea8e46890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/493828061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8db39f0 0x557ea8e42590 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/493828061 >> 172.21.15.42:0/493828061 conn(0x557ea8d61390 msgr2=0x557ea8e41e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:44.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/493828061 shutdown_connections 2024-05-27T10:21:44.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.227+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/493828061 wait complete. 2024-05-27T10:21:44.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 -- start start 2024-05-27T10:21:44.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 Processor -- start 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8ec95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81af580 con 0x557ea8cc7e40 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b09f0 con 0x557ea8e46e70 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81afc50 con 0x557ea8db39f0 2024-05-27T10:21:44.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.228+0000 7fe8de4f0b80 10 client.0 _ll_register_callbacks cb 0x557ea8d0ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e54170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33246/0 (socket says 172.21.15.42:33246) 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 -- 172.21.15.42:0/2974797864 learned_addr learned my addr 172.21.15.42:0/2974797864 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cda00700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8ec95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 -- 172.21.15.42:0/2974797864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 msgr2=0x557ea8e546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 -- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 msgr2=0x557ea8ec95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8ec95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.229+0000 7fe8cd000700 1 -- 172.21.15.42:0/2974797864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea8ec9b10 con 0x557ea8cc7e40 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8cd000700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e54170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0a6700 tx=0x7fe8bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9114c0ff902856 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8aea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:44.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8aea00700 1 -- 172.21.15.42:0/2974797864 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0a98f0 con 0x557ea8cc7e40 2024-05-27T10:21:44.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8aea00700 1 -- 172.21.15.42:0/2974797864 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe8bc0b1e60 con 0x557ea8cc7e40 2024-05-27T10:21:44.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b40043c0 con 0x557ea8cc7e40 2024-05-27T10:21:44.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8aea00700 1 -- 172.21.15.42:0/2974797864 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8bc0a8fa0 con 0x557ea8cc7e40 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8de4f0b80 10 client.15969 fetch_fsmap learned FSMap version 28 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.230+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ea8ecaca0 con 0x557ea8cc7e40 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8aea00700 1 -- 172.21.15.42:0/2974797864 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b40043c0 con 0x557ea8cc7e40 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8aea00700 1 -- 172.21.15.42:0/2974797864 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fe8bc0b0bc0 con 0x557ea8cc7e40 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8de4f0b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8de4f0b80 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8de4f0b80 20 client.15969 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8ecae70 0x557ea8ecd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8de4f0b80 4 client.15969 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.231+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ea8d70570 con 0x557ea8ecae70 2024-05-27T10:21:44.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.232+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8ecae70 0x557ea8ecd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:44.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8c7e00700 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8ecae70 0x557ea8ecd2b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b4a3d2540dd668 server_cookie=d73db8ee4d7cedb7 in_seq=0 out_seq=0 2024-05-27T10:21:44.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8aea00700 10 client.15969 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:44.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8aea00700 1 -- 172.21.15.42:0/2974797864 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ea8d70570 con 0x557ea8ecae70 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8aea00700 10 client.15969 handle_command_reply: tid=0 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8de4f0b80 1 client.15969 shutdown 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 msgr2=0x557ea8e54170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.233+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e54170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0a6700 tx=0x7fe8bc000bc0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 shutdown_connections 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557ea8ecae70 0x557ea8ecd2b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea8e46e70 0x557ea8ec95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea8cc7e40 0x557ea8e54170 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 --2- 172.21.15.42:0/2974797864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557ea8db39f0 0x557ea8e546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8aea00700 0 client.15969 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 >> 172.21.15.42:0/2974797864 conn(0x557ea8d61390 msgr2=0x557ea8e50510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 shutdown_connections 2024-05-27T10:21:44.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 1 -- 172.21.15.42:0/2974797864 wait complete. 2024-05-27T10:21:44.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.234+0000 7fe8de4f0b80 20 client.15969 trim_cache size 0 max 16384 2024-05-27T10:21:44.235 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:21:44.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe8b00b29f0 msgr2=0x7fe8b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe8b00b29f0 0x7fe8b00b4f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00062a0 tx=0x7fe8c000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 msgr2=0x7fe8c819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 0x7fe8c819cf50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d950 tx=0x7fe8bc00dd10 comp rx=0 tx=0).stop 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 shutdown_connections 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe8b00b29f0 0x7fe8b00b4f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8c8105230 0x7fe8c81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c8102920 0x7fe8c819cf50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 --2- 172.21.15.42:0/4118155850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8c81033a0 0x7fe8c819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 >> 172.21.15.42:0/4118155850 conn(0x7fe8c80fe6e0 msgr2=0x7fe8c80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 shutdown_connections 2024-05-27T10:21:44.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:44.235+0000 7fe8cfa00700 1 -- 172.21.15.42:0/4118155850 wait complete. 2024-05-27T10:21:44.240 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/param.h usability... yes 2024-05-27T10:21:44.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:44 smithi143 ceph-mon[35545]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 163 KiB/s wr, 43 op/s 2024-05-27T10:21:44.375 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/param.h presence... yes 2024-05-27T10:21:44.382 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/param.h... yes 2024-05-27T10:21:44.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:44 smithi104 ceph-mon[34812]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 163 KiB/s wr, 43 op/s 2024-05-27T10:21:44.664 INFO:tasks.workunit.client.0.smithi042.stdout:checking ioctl.h usability... no 2024-05-27T10:21:44.828 INFO:tasks.workunit.client.0.smithi042.stdout:checking ioctl.h presence... no 2024-05-27T10:21:44.832 INFO:tasks.workunit.client.0.smithi042.stdout:checking for ioctl.h... no 2024-05-27T10:21:45.111 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/ioctl.h usability... yes 2024-05-27T10:21:45.282 INFO:tasks.workunit.client.0.smithi042.stdout:checking sys/ioctl.h presence... yes 2024-05-27T10:21:45.290 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sys/ioctl.h... yes 2024-05-27T10:21:45.582 INFO:tasks.workunit.client.0.smithi042.stdout:checking return type of signal handlers... void 2024-05-27T10:21:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 -- 172.21.15.42:0/3126792745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e18076530 msgr2=0x7f7e18076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/3126792745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e18076530 0x7f7e18076940 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009900 tx=0x7f7e0802a480 comp rx=0 tx=0).stop 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 -- 172.21.15.42:0/3126792745 shutdown_connections 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/3126792745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e181104e0 0x7f7e18112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/3126792745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e1810db80 0x7f7e1810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/3126792745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e18076530 0x7f7e18076940 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 -- 172.21.15.42:0/3126792745 >> 172.21.15.42:0/3126792745 conn(0x7f7e180ff330 msgr2=0x7f7e18101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 -- 172.21.15.42:0/3126792745 shutdown_connections 2024-05-27T10:21:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.814+0000 7f7e1cc00640 1 -- 172.21.15.42:0/3126792745 wait complete. 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.815+0000 7f7e1cc00640 1 Processor -- start 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.815+0000 7f7e1cc00640 1 -- start start 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e1cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e18076530 0x7f7e181a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e1cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e1810db80 0x7f7e181a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e1cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 0x7f7e1819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e1cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18116030 con 0x7f7e181104e0 2024-05-27T10:21:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e1cc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18115eb0 con 0x7f7e1810db80 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e1cc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181161b0 con 0x7f7e18076530 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e16800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 0x7f7e1819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e16800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 0x7f7e1819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33268/0 (socket says 172.21.15.42:33268) 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.816+0000 7f7e16800640 1 -- 172.21.15.42:0/2649093130 learned_addr learned my addr 172.21.15.42:0/2649093130 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e15e00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e18076530 0x7f7e181a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e15400640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e1810db80 0x7f7e181a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e16800640 1 -- 172.21.15.42:0/2649093130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e18076530 msgr2=0x7f7e181a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e16800640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e18076530 0x7f7e181a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e16800640 1 -- 172.21.15.42:0/2649093130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e1810db80 msgr2=0x7f7e181a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e16800640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e1810db80 0x7f7e181a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e16800640 1 -- 172.21.15.42:0/2649093130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1819d090 con 0x7f7e181104e0 2024-05-27T10:21:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e15400640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e1810db80 0x7f7e181a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.817+0000 7f7e16800640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 0x7f7e1819c9f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000b970 tx=0x7f7e1000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.818+0000 7f7e0ea00640 1 -- 172.21.15.42:0/2649093130 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10019070 con 0x7f7e181104e0 2024-05-27T10:21:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.818+0000 7f7e0ea00640 1 -- 172.21.15.42:0/2649093130 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e10014440 con 0x7f7e181104e0 2024-05-27T10:21:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.818+0000 7f7e0ea00640 1 -- 172.21.15.42:0/2649093130 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100075d0 con 0x7f7e181104e0 2024-05-27T10:21:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.818+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1819d380 con 0x7f7e181104e0 2024-05-27T10:21:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.818+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a9660 con 0x7f7e181104e0 2024-05-27T10:21:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.821+0000 7f7e23bed740 1 Processor -- start 2024-05-27T10:21:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e0ea00640 1 -- 172.21.15.42:0/2649093130 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7e1001c070 con 0x7f7e181104e0 2024-05-27T10:21:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 -- start start 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 0x55ceaf495860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf495da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ceaf406b90 con 0x55ceaf0f31b0 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ceaf3f6730 con 0x55ceaf0d24d0 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.822+0000 7f7e23bed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ceaf3fa770 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 0x55ceaf495860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 0x55ceaf495860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33282/0 (socket says 172.21.15.42:33282) 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 -- 172.21.15.42:0/3857770414 learned_addr learned my addr 172.21.15.42:0/3857770414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15e00640 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf495da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 -- 172.21.15.42:0/3857770414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0ca2f0 msgr2=0x55ceaf495da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf495da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 -- 172.21.15.42:0/3857770414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 msgr2=0x55ceaf4962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.823+0000 7f7e15400640 1 -- 172.21.15.42:0/3857770414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceaf38f8b0 con 0x55ceaf0f31b0 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e15400640 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 0x55ceaf495860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400ae20 tx=0x7f7e0400a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83751ca0edbd322 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e0ea00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7dfc0b29d0 0x7f7dfc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e0ea00640 1 -- 172.21.15.42:0/2649093130 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7e100e1ac0 con 0x7f7e181104e0 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e03000640 1 -- 172.21.15.42:0/3857770414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401b070 con 0x55ceaf0f31b0 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e15e00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7dfc0b29d0 0x7f7dfc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e03000640 1 -- 172.21.15.42:0/3857770414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e0400fd70 con 0x55ceaf0f31b0 2024-05-27T10:21:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.824+0000 7f7e03000640 1 -- 172.21.15.42:0/3857770414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04014650 con 0x55ceaf0f31b0 2024-05-27T10:21:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 -- 172.21.15.42:0/3857770414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 msgr2=0x55ceaf495860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 0x55ceaf495860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400ae20 tx=0x7f7e0400a4d0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e15e00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7dfc0b29d0 0x7f7dfc0b4ee0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e1819db30 tx=0x7f7e08009640 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 -- 172.21.15.42:0/3857770414 shutdown_connections 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf495da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3857770414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0f31b0 0x55ceaf495860 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 -- 172.21.15.42:0/3857770414 >> 172.21.15.42:0/3857770414 conn(0x55ceaf0e7300 msgr2=0x55ceaf492010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 -- 172.21.15.42:0/3857770414 shutdown_connections 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.825+0000 7f7e23bed740 1 -- 172.21.15.42:0/3857770414 wait complete. 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.826+0000 7f7e23bed740 1 -- start start 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.826+0000 7f7e23bed740 1 Processor -- start 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 0x55ceaf49ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ceaf0dc790 con 0x55ceaf0d24d0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ceaf39ebc0 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ceaf4a2990 con 0x55ceaf0f31b0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e23bed740 10 client.0 _ll_register_callbacks cb 0x55ceaf37c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e15400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 0x55ceaf49ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e15400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 0x55ceaf49ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33296/0 (socket says 172.21.15.42:33296) 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.827+0000 7f7e15400640 1 -- 172.21.15.42:0/2247260491 learned_addr learned my addr 172.21.15.42:0/2247260491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15e00640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e16800640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15400640 1 -- 172.21.15.42:0/2247260491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 msgr2=0x55ceaf4abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15400640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15400640 1 -- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0ca2f0 msgr2=0x55ceaf49fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15400640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15400640 1 -- 172.21.15.42:0/2247260491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceaf10b210 con 0x55ceaf0d24d0 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e15400640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 0x55ceaf49ff70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7e04004810 tx=0x7f7e04016980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1901d6d72df6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7df7400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7df7400640 1 -- 172.21.15.42:0/2247260491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04025060 con 0x55ceaf0d24d0 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0400fd70 con 0x55ceaf0d24d0 2024-05-27T10:21:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.828+0000 7f7df7400640 1 -- 172.21.15.42:0/2247260491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7e040059c0 con 0x55ceaf0d24d0 2024-05-27T10:21:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7df7400640 1 -- 172.21.15.42:0/2247260491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e0401b040 con 0x55ceaf0d24d0 2024-05-27T10:21:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-05-27T10:21:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ceaf1d7620 con 0x55ceaf0d24d0 2024-05-27T10:21:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7df7400640 1 -- 172.21.15.42:0/2247260491 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e0400fd70 con 0x55ceaf0d24d0 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7df7400640 1 -- 172.21.15.42:0/2247260491 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7e04015040 con 0x55ceaf0d24d0 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 20 client.15981 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0b1d80 0x55ceaf4b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 4 client.15981 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.829+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ceaf408ae0 con 0x55ceaf0b1d80 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.830+0000 7f7e15e00640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0b1d80 0x55ceaf4b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.831+0000 7f7e15e00640 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0b1d80 0x55ceaf4b1bf0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69e695d79b8a9c0 server_cookie=54490802bc3736ec in_seq=0 out_seq=0 2024-05-27T10:21:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.831+0000 7f7df7400640 10 client.15981 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.833+0000 7f7df7400640 1 -- 172.21.15.42:0/2247260491 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ceaf408ae0 con 0x55ceaf0b1d80 2024-05-27T10:21:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.833+0000 7f7df7400640 10 client.15981 handle_command_reply: tid=0 2024-05-27T10:21:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.833+0000 7f7e23bed740 1 client.15981 shutdown 2024-05-27T10:21:45.833 INFO:tasks.workunit.client.0.smithi042.stdout:checking for size_t... yes 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 msgr2=0x55ceaf49ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 0x55ceaf49ff70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7e04004810 tx=0x7f7e04016980 comp rx=0 tx=0).stop 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 shutdown_connections 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0b1d80 0x55ceaf4b1bf0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0d24d0 0x55ceaf49ff70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 --2- 172.21.15.42:0/2247260491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 >> 172.21.15.42:0/2247260491 conn(0x55ceaf0e7300 msgr2=0x55ceaf495630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7df7400640 0 client.15981 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 shutdown_connections 2024-05-27T10:21:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 1 -- 172.21.15.42:0/2247260491 wait complete. 2024-05-27T10:21:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.834+0000 7f7e23bed740 20 client.15981 trim_cache size 0 max 16384 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.853+0000 7f7e23bed740 1 Processor -- start 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.853+0000 7f7e23bed740 1 -- start start 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf495c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e23bed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b4530 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e23bed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b1e00 con 0x55ceaf0d24d0 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e23bed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181afc50 con 0x55ceaf0f31b0 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33298/0 (socket says 172.21.15.42:33298) 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e15e00640 1 -- 172.21.15.42:0/3918280702 learned_addr learned my addr 172.21.15.42:0/3918280702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e15400640 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf495c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.854+0000 7f7e16800640 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e15e00640 1 -- 172.21.15.42:0/3918280702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 msgr2=0x55ceaf4b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e15e00640 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e15e00640 1 -- 172.21.15.42:0/3918280702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 msgr2=0x55ceaf495c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e15e00640 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf495c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e15e00640 1 -- 172.21.15.42:0/3918280702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceaf492be0 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e15e00640 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49ff80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7e08004650 tx=0x7f7e08041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d35befd726768b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e02600640 1 -- 172.21.15.42:0/3918280702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08035050 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e02600640 1 -- 172.21.15.42:0/3918280702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e08056070 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.855+0000 7f7e02600640 1 -- 172.21.15.42:0/3918280702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0803c930 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 -- 172.21.15.42:0/3918280702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 msgr2=0x55ceaf49ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49ff80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7e08004650 tx=0x7f7e08041000 comp rx=0 tx=0).stop 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 -- 172.21.15.42:0/3918280702 shutdown_connections 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf4b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf495c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 --2- 172.21.15.42:0/3918280702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf49ff80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 -- 172.21.15.42:0/3918280702 >> 172.21.15.42:0/3918280702 conn(0x55ceaf0e7300 msgr2=0x55ceaf49a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.856+0000 7f7e23bed740 1 -- 172.21.15.42:0/3918280702 shutdown_connections 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.857+0000 7f7e23bed740 1 -- 172.21.15.42:0/3918280702 wait complete. 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.857+0000 7f7e23bed740 1 -- start start 2024-05-27T10:21:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.857+0000 7f7e23bed740 1 Processor -- start 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf4b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf520850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b02c0 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b1730 con 0x55ceaf0d24d0 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b0990 con 0x55ceaf0f31b0 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e23bed740 10 client.0 _ll_register_callbacks cb 0x7f7e04019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf4b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf4b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33310/0 (socket says 172.21.15.42:33310) 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 -- 172.21.15.42:0/564508366 learned_addr learned my addr 172.21.15.42:0/564508366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 -- 172.21.15.42:0/564508366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 msgr2=0x55ceaf520850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf520850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 -- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 msgr2=0x55ceaf4b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.858+0000 7f7e15e00640 1 -- 172.21.15.42:0/564508366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceaf4b4ac0 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.859+0000 7f7e15e00640 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf4b3d30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7e08003950 tx=0x7f7e08004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3293e6f64e2e64a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.859+0000 7f7df7400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.859+0000 7f7df7400640 1 -- 172.21.15.42:0/564508366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0805e070 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.859+0000 7f7df7400640 1 -- 172.21.15.42:0/564508366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7e08051950 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.859+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e08056070 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7df7400640 1 -- 172.21.15.42:0/564508366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e08035040 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ceaf521db0 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7df7400640 1 -- 172.21.15.42:0/564508366 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e08056070 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7df7400640 1 -- 172.21.15.42:0/564508366 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7e08049420 con 0x55ceaf0ca2f0 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 20 client.15987 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0871c0 0x55ceaf5245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 4 client.15987 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.860+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ceaf4022c0 con 0x55ceaf0871c0 2024-05-27T10:21:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.861+0000 7f7e15400640 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0871c0 0x55ceaf5245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.862+0000 7f7df7400640 10 client.15987 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.862+0000 7f7e15400640 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0871c0 0x55ceaf5245e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2412c1718669a79c server_cookie=6a219f7e62578b2d in_seq=0 out_seq=0 2024-05-27T10:21:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7df7400640 1 -- 172.21.15.42:0/564508366 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ceaf4022c0 con 0x55ceaf0871c0 2024-05-27T10:21:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7df7400640 10 client.15987 handle_command_reply: tid=0 2024-05-27T10:21:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 client.15987 shutdown 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 msgr2=0x55ceaf4b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf4b3d30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7e08003950 tx=0x7f7e08004230 comp rx=0 tx=0).stop 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 shutdown_connections 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ceaf0871c0 0x55ceaf5245e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ceaf0f31b0 0x55ceaf520850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ceaf0d24d0 0x55ceaf4b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 --2- 172.21.15.42:0/564508366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ceaf0ca2f0 0x55ceaf4b3d30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7df7400640 0 client.15987 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.902+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 >> 172.21.15.42:0/564508366 conn(0x55ceaf0e7300 msgr2=0x55ceaf4a1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.903+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 shutdown_connections 2024-05-27T10:21:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.903+0000 7f7e23bed740 1 -- 172.21.15.42:0/564508366 wait complete. 2024-05-27T10:21:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.903+0000 7f7e23bed740 20 client.15987 trim_cache size 0 max 16384 2024-05-27T10:21:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:21:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:21:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:21:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7dfc0b29d0 msgr2=0x7f7dfc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7dfc0b29d0 0x7f7dfc0b4ee0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e1819db30 tx=0x7f7e08009640 comp rx=0 tx=0).stop 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 msgr2=0x7f7e1819c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 0x7f7e1819c9f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000b970 tx=0x7f7e1000be30 comp rx=0 tx=0).stop 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 shutdown_connections 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7dfc0b29d0 0x7f7dfc0b4ee0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e181104e0 0x7f7e1819c9f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e1810db80 0x7f7e181a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/2649093130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e18076530 0x7f7e181a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 >> 172.21.15.42:0/2649093130 conn(0x7f7e180ff330 msgr2=0x7f7e18110930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 shutdown_connections 2024-05-27T10:21:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:45.904+0000 7f7e1cc00640 1 -- 172.21.15.42:0/2649093130 wait complete. 2024-05-27T10:21:46.070 INFO:tasks.workunit.client.0.smithi042.stdout:checking for dev_t... yes 2024-05-27T10:21:46.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:46 smithi143 ceph-mon[35545]: pgmap v94: 97 pgs: 97 active+clean; 3.5 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 228 KiB/s wr, 68 op/s 2024-05-27T10:21:46.316 INFO:tasks.workunit.client.0.smithi042.stdout:checking for short... yes 2024-05-27T10:21:46.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:46 smithi042 ceph-mon[31661]: pgmap v94: 97 pgs: 97 active+clean; 3.5 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 228 KiB/s wr, 68 op/s 2024-05-27T10:21:46.355 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T10:21:46.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T10:21:46.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:46 smithi104 ceph-mon[34812]: pgmap v94: 97 pgs: 97 active+clean; 3.5 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 228 KiB/s wr, 68 op/s 2024-05-27T10:21:46.659 INFO:tasks.workunit.client.0.smithi042.stdout:checking size of short... 2 2024-05-27T10:21:46.836 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:46.898 INFO:tasks.workunit.client.0.smithi042.stdout:checking for int... yes 2024-05-27T10:21:47.263 INFO:tasks.workunit.client.0.smithi042.stdout:checking size of int... 4 2024-05-27T10:21:47.502 INFO:tasks.workunit.client.0.smithi042.stdout:checking for long... yes 2024-05-27T10:21:47.861 INFO:tasks.workunit.client.0.smithi042.stdout:checking size of long... 8 2024-05-27T10:21:48.113 INFO:tasks.workunit.client.0.smithi042.stdout:checking for long long... yes 2024-05-27T10:21:48.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:48 smithi143 ceph-mon[35545]: pgmap v95: 97 pgs: 97 active+clean; 3.5 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 148 KiB/s wr, 52 op/s 2024-05-27T10:21:48.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:48 smithi042 ceph-mon[31661]: pgmap v95: 97 pgs: 97 active+clean; 3.5 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 148 KiB/s wr, 52 op/s 2024-05-27T10:21:48.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:48 smithi104 ceph-mon[34812]: pgmap v95: 97 pgs: 97 active+clean; 3.5 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 148 KiB/s wr, 52 op/s 2024-05-27T10:21:48.493 INFO:tasks.workunit.client.0.smithi042.stdout:checking size of long long... 8 2024-05-27T10:21:48.721 INFO:tasks.workunit.client.0.smithi042.stdout:checking for ssize_t... yes 2024-05-27T10:21:48.956 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sig_atomic_t... yes 2024-05-27T10:21:49.269 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether gcc needs -traditional... no 2024-05-27T10:21:49.481 INFO:tasks.workunit.client.0.smithi042.stdout:checking for an ANSI C-conforming const... yes 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.508+0000 7f9f34600700 1 -- 172.21.15.42:0/2098120037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c106560 msgr2=0x7f9f2c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.508+0000 7f9f34600700 1 --2- 172.21.15.42:0/2098120037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c106560 0x7f9f2c106970 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18009ca0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.508+0000 7f9f34600700 1 -- 172.21.15.42:0/2098120037 shutdown_connections 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.509+0000 7f9f34600700 1 --2- 172.21.15.42:0/2098120037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c106eb0 0x7f9f2c113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.509+0000 7f9f34600700 1 --2- 172.21.15.42:0/2098120037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.509+0000 7f9f34600700 1 --2- 172.21.15.42:0/2098120037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c106560 0x7f9f2c106970 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.509+0000 7f9f34600700 1 -- 172.21.15.42:0/2098120037 >> 172.21.15.42:0/2098120037 conn(0x7f9f2c102040 msgr2=0x7f9f2c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.509+0000 7f9f34600700 1 -- 172.21.15.42:0/2098120037 shutdown_connections 2024-05-27T10:21:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.509+0000 7f9f34600700 1 -- 172.21.15.42:0/2098120037 wait complete. 2024-05-27T10:21:49.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.510+0000 7f9f34600700 1 Processor -- start 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.510+0000 7f9f34600700 1 -- start start 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f34600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c106560 0x7f9f2c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f34600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f2c106eb0 0x7f9f2c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f34600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f34600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c116580 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f34600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c116400 con 0x7f9f2c106eb0 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f34600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c116700 con 0x7f9f2c106560 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f32600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f32600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33338/0 (socket says 172.21.15.42:33338) 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f32600700 1 -- 172.21.15.42:0/3096976953 learned_addr learned my addr 172.21.15.42:0/3096976953 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f31200700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f2c106eb0 0x7f9f2c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.511+0000 7f9f31c00700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c106560 0x7f9f2c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f32600700 1 -- 172.21.15.42:0/3096976953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c106560 msgr2=0x7f9f2c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f32600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c106560 0x7f9f2c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f32600700 1 -- 172.21.15.42:0/3096976953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f2c106eb0 msgr2=0x7f9f2c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f32600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f2c106eb0 0x7f9f2c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f32600700 1 -- 172.21.15.42:0/3096976953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c1a1d70 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f32600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c1a1780 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007d70 tx=0x7f9f2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f2a000700 1 -- 172.21.15.42:0/3096976953 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24016020 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c1a2060 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f2a000700 1 -- 172.21.15.42:0/3096976953 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2400ad40 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f2a000700 1 -- 172.21.15.42:0/3096976953 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240115e0 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.512+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1a2550 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.517+0000 7f9f2a000700 1 -- 172.21.15.42:0/3096976953 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9f24019020 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.517+0000 7f9f4309cb80 1 Processor -- start 2024-05-27T10:21:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 -- start start 2024-05-27T10:21:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb8f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 0x55d28bb898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb8a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d28bab7ec0 con 0x55d28bb8fdb0 2024-05-27T10:21:49.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d28baa7a60 con 0x55d28bb89e30 2024-05-27T10:21:49.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.518+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d28baabaa0 con 0x55d28ba0aec0 2024-05-27T10:21:49.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 0x55d28bb898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f2a000700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9f140b29f0 0x7f9f140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 0x55d28bb898f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33354/0 (socket says 172.21.15.42:33354) 2024-05-27T10:21:49.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 -- 172.21.15.42:0/2620032392 learned_addr learned my addr 172.21.15.42:0/2620032392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31200700 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb8a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31200700 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb8f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f2a000700 1 -- 172.21.15.42:0/3096976953 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9f240e2550 con 0x7f9f2c10c5a0 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f32600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9f140b29f0 0x7f9f140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 -- 172.21.15.42:0/2620032392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 msgr2=0x55d28bb8f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb8f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 -- 172.21.15.42:0/2620032392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 msgr2=0x55d28bb8a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb8a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.519+0000 7f9f31c00700 1 -- 172.21.15.42:0/2620032392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d28ba704a0 con 0x55d28bb8fdb0 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.520+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 0x55d28bb898f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000bd10 tx=0x7f9f2000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c354188b7ee1d3ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.520+0000 7f9f1ea00700 1 -- 172.21.15.42:0/2620032392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20015070 con 0x55d28bb8fdb0 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.520+0000 7f9f1ea00700 1 -- 172.21.15.42:0/2620032392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2000fe50 con 0x55d28bb8fdb0 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.520+0000 7f9f1ea00700 1 -- 172.21.15.42:0/2620032392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20011640 con 0x55d28bb8fdb0 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2620032392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 msgr2=0x55d28bb898f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f32600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9f140b29f0 0x7f9f140b4f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f240dfe60 tx=0x7f9f2400aef0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 0x55d28bb898f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000bd10 tx=0x7f9f2000f640 comp rx=0 tx=0).stop 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2620032392 shutdown_connections 2024-05-27T10:21:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb8a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb8f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2620032392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb8fdb0 0x55d28bb898f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2620032392 >> 172.21.15.42:0/2620032392 conn(0x55d28ba6b040 msgr2=0x55d28bb84950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.521+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2620032392 shutdown_connections 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.522+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2620032392 wait complete. 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.522+0000 7f9f4309cb80 1 -- start start 2024-05-27T10:21:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.522+0000 7f9f4309cb80 1 Processor -- start 2024-05-27T10:21:49.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb93a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb8fdb0 0x55d28bb93f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d28b9ae020 con 0x55d28ba0aec0 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d28ba487e0 con 0x55d28bb89e30 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d28bb96880 con 0x55d28bb8fdb0 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f4309cb80 10 client.0 _ll_register_callbacks cb 0x55d28ba51cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f31200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f31200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb934e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33360/0 (socket says 172.21.15.42:33360) 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.523+0000 7f9f31200700 1 -- 172.21.15.42:0/2337676055 learned_addr learned my addr 172.21.15.42:0/2337676055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb93a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31200700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb8fdb0 0x55d28bb93f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31200700 1 -- 172.21.15.42:0/2337676055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb8fdb0 msgr2=0x55d28bb93f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31200700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb8fdb0 0x55d28bb93f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31200700 1 -- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 msgr2=0x55d28bb93a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31200700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb93a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.524+0000 7f9f31200700 1 -- 172.21.15.42:0/2337676055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28b9f7950 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f31200700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb934e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f1800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feba48a8f527ca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f13400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f13400700 1 -- 172.21.15.42:0/2337676055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f180039d0 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2000fe50 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f13400700 1 -- 172.21.15.42:0/2337676055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9f18003b70 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f13400700 1 -- 172.21.15.42:0/2337676055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f18003d70 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f4309cb80 10 client.15999 fetch_fsmap learned FSMap version 28 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.525+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d28ba780e0 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f13400700 1 -- 172.21.15.42:0/2337676055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2000fe50 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f13400700 1 -- 172.21.15.42:0/2337676055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f9f18024070 con 0x55d28ba0aec0 2024-05-27T10:21:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f4309cb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f4309cb80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f4309cb80 20 client.15999 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bba3680 0x55d28bba5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f4309cb80 4 client.15999 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.526+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d28bab9e10 con 0x55d28bba3680 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.527+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bba3680 0x55d28bba5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.528+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bba3680 0x55d28bba5b10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c619112c173669 server_cookie=f2544c3eaf65d045 in_seq=0 out_seq=0 2024-05-27T10:21:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.528+0000 7f9f13400700 10 client.15999 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f13400700 1 -- 172.21.15.42:0/2337676055 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d28bab9e10 con 0x55d28bba3680 2024-05-27T10:21:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f13400700 10 client.15999 handle_command_reply: tid=0 2024-05-27T10:21:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 client.15999 shutdown 2024-05-27T10:21:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 msgr2=0x55d28bb934e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb934e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f1800b540 comp rx=0 tx=0).stop 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 shutdown_connections 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bba3680 0x55d28bba5b10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb8fdb0 0x55d28bb93f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb934e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2337676055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb89e30 0x55d28bb93a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 >> 172.21.15.42:0/2337676055 conn(0x55d28ba6b040 msgr2=0x55d28bb84600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f13400700 0 client.15999 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 shutdown_connections 2024-05-27T10:21:49.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.532+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2337676055 wait complete. 2024-05-27T10:21:49.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.533+0000 7f9f4309cb80 20 client.15999 trim_cache size 0 max 16384 2024-05-27T10:21:49.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.556+0000 7f9f4309cb80 1 Processor -- start 2024-05-27T10:21:49.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.556+0000 7f9f4309cb80 1 -- start start 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb89e30 0x55d28bb95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb9f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1b3660 con 0x55d28ba0aec0 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1b0ef0 con 0x55d28bb8fdb0 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1aed40 con 0x55d28bb89e30 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f31200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f31200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb898f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33374/0 (socket says 172.21.15.42:33374) 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f31200700 1 -- 172.21.15.42:0/2560353018 learned_addr learned my addr 172.21.15.42:0/2560353018 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.557+0000 7f9f31c00700 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb89e30 0x55d28bb95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f31200700 1 -- 172.21.15.42:0/2560353018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb89e30 msgr2=0x55d28bb95840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f31200700 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb89e30 0x55d28bb95840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f31200700 1 -- 172.21.15.42:0/2560353018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 msgr2=0x55d28bb9f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f31200700 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb9f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f31200700 1 -- 172.21.15.42:0/2560353018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d28bb95d80 con 0x55d28ba0aec0 2024-05-27T10:21:49.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f31200700 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb898f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9f18032bc0 tx=0x7f9f18003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c733bb5910f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:49.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f1e000700 1 -- 172.21.15.42:0/2560353018 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f180056d0 con 0x55d28ba0aec0 2024-05-27T10:21:49.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.558+0000 7f9f1e000700 1 -- 172.21.15.42:0/2560353018 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f18037070 con 0x55d28ba0aec0 2024-05-27T10:21:49.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.559+0000 7f9f1e000700 1 -- 172.21.15.42:0/2560353018 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800be60 con 0x55d28ba0aec0 2024-05-27T10:21:49.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.559+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2560353018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 msgr2=0x55d28bb898f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.559+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb898f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9f18032bc0 tx=0x7f9f18003f50 comp rx=0 tx=0).stop 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2560353018 shutdown_connections 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb9f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28ba0aec0 0x55d28bb898f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/2560353018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28bb89e30 0x55d28bb95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2560353018 >> 172.21.15.42:0/2560353018 conn(0x55d28ba6b040 msgr2=0x55d28bb93cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2560353018 shutdown_connections 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.560+0000 7f9f4309cb80 1 -- 172.21.15.42:0/2560353018 wait complete. 2024-05-27T10:21:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.561+0000 7f9f4309cb80 1 -- start start 2024-05-27T10:21:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.561+0000 7f9f4309cb80 1 Processor -- start 2024-05-27T10:21:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.561+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb9bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.561+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 0x55d28bb9c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f4309cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb98aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1af3b0 con 0x55d28bb89e30 2024-05-27T10:21:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1b0820 con 0x55d28bb8fdb0 2024-05-27T10:21:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f4309cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1afa80 con 0x55d28ba0aec0 2024-05-27T10:21:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f4309cb80 10 client.0 _ll_register_callbacks cb 0x55d28ba51cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 0x55d28bb9c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 0x55d28bb9c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33384/0 (socket says 172.21.15.42:33384) 2024-05-27T10:21:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 -- 172.21.15.42:0/3409570671 learned_addr learned my addr 172.21.15.42:0/3409570671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31200700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb9bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31200700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb98aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 -- 172.21.15.42:0/3409570671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 msgr2=0x55d28bb9bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb9bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 -- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 msgr2=0x55d28bb98aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb98aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.562+0000 7f9f31c00700 1 -- 172.21.15.42:0/3409570671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28bb99220 con 0x55d28bb89e30 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.563+0000 7f9f31c00700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 0x55d28bb9c210 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000fa40 tx=0x7f9f2000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f80c50720316c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.563+0000 7f9f13400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.563+0000 7f9f13400700 1 -- 172.21.15.42:0/3409570671 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20015070 con 0x55d28bb89e30 2024-05-27T10:21:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.563+0000 7f9f13400700 1 -- 172.21.15.42:0/3409570671 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9f20004440 con 0x55d28bb89e30 2024-05-27T10:21:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.563+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f18037070 con 0x55d28bb89e30 2024-05-27T10:21:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f13400700 1 -- 172.21.15.42:0/3409570671 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f20036050 con 0x55d28bb89e30 2024-05-27T10:21:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 10 client.16005 fetch_fsmap learned FSMap version 28 2024-05-27T10:21:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d28bc0ebb0 con 0x55d28bb89e30 2024-05-27T10:21:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f13400700 1 -- 172.21.15.42:0/3409570671 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f18037070 con 0x55d28bb89e30 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f13400700 1 -- 172.21.15.42:0/3409570671 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f9f20004ab0 con 0x55d28bb89e30 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 10 client.16005 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 20 client.16005 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bc0f090 0x55d28bc114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 4 client.16005 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.564+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d28bab35f0 con 0x55d28bc0f090 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.565+0000 7f9f31200700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bc0f090 0x55d28bc114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.566+0000 7f9f31200700 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bc0f090 0x55d28bc114e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1017e6ad61f3c8d0 server_cookie=88694a954f9c2589 in_seq=0 out_seq=0 2024-05-27T10:21:49.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.566+0000 7f9f13400700 10 client.16005 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:49.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f13400700 1 -- 172.21.15.42:0/3409570671 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d28bab35f0 con 0x55d28bc0f090 2024-05-27T10:21:49.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f13400700 10 client.16005 handle_command_reply: tid=0 2024-05-27T10:21:49.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 client.16005 shutdown 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 msgr2=0x55d28bb9c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 0x55d28bb9c210 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000fa40 tx=0x7f9f2000be60 comp rx=0 tx=0).stop 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 shutdown_connections 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d28bc0f090 0x55d28bc114e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d28bb8fdb0 0x55d28bb98aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d28ba0aec0 0x55d28bb9bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 --2- 172.21.15.42:0/3409570671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d28bb89e30 0x55d28bb9c210 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 >> 172.21.15.42:0/3409570671 conn(0x55d28ba6b040 msgr2=0x55d28bb93cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f13400700 0 client.16005 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:49.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.568+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 shutdown_connections 2024-05-27T10:21:49.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.569+0000 7f9f4309cb80 1 -- 172.21.15.42:0/3409570671 wait complete. 2024-05-27T10:21:49.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.569+0000 7f9f4309cb80 20 client.16005 trim_cache size 0 max 16384 2024-05-27T10:21:49.569 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:21:49.569 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-05-27T10:21:49.569 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "28fdbfa8-9597-4606-b729-fd1e3d1cc3f8", 2024-05-27T10:21:49.570 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-05-27T10:21:49.570 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:21:49.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9f140b29f0 msgr2=0x7f9f140b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9f140b29f0 0x7f9f140b4f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f240dfe60 tx=0x7f9f2400aef0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 msgr2=0x7f9f2c1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:49.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c1a1780 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007d70 tx=0x7f9f2400a340 comp rx=0 tx=0).stop 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 shutdown_connections 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9f140b29f0 0x7f9f140b4f00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f2c10c5a0 0x7f9f2c1a1780 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c106560 0x7f9f2c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 --2- 172.21.15.42:0/3096976953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f2c106eb0 0x7f9f2c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 >> 172.21.15.42:0/3096976953 conn(0x7f9f2c102040 msgr2=0x7f9f2c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 shutdown_connections 2024-05-27T10:21:49.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:49.570+0000 7f9f34600700 1 -- 172.21.15.42:0/3096976953 wait complete. 2024-05-27T10:21:49.653 INFO:tasks.workunit.client.0.smithi042.stdout:checking for inline... inline 2024-05-27T10:21:49.948 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.586+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [...4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/ auth v477 ap=1 f(v0 m2024-05-27T10:21:32.072420+0000 3=3+0) n(v0 rc2024-05-27T10:21:32.079545+0000 b10836 4=3+1) (inest lock) (ifile excl) caps={24925=pAsLsXsFsx/AsLsXsFsx@54},l=24925 | ptrwaiter=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3bc6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000204:[<0x10000000201/man v12>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000204:[<0x10000000201/man v477>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-27T10:21:32.072420+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-27T10:21:32.078111+0000 b10836 4=3+1)","memory_value.dirstat":"f(v0 m2024-05-27T10:21:32.072420+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-05-27T10:21:32.079545+0000 b10836 4=3+1)","error_str":""},"return_code":0} 2024-05-27T10:21:49.949 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.590+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/ auth v515 ap=1 f(v0 m2024-05-27T10:21:32.206154+0000 18=18+0) n(v0 rc2024-05-27T10:21:32.210923+0000 b73222 19=18+1) (inest lock) (ifile excl) caps={24925=pAsLsXsFsx/AsLsXsFsx@54},l=24925 | ptrwaiter=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3bc6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-27T10:21:32.206154+0000 18=18+0)","ondisk_value.rstat":"n(v0 rc2024-05-27T10:21:32.210143+0000 b73222 19=18+1)","memory_value.dirstat":"f(v0 m2024-05-27T10:21:32.206154+0000 18=18+0)","memory_value.rstat":"n(v0 rc2024-05-27T10:21:32.210923+0000 b73222 19=18+1)","error_str":""},"return_code":0} 2024-05-27T10:21:49.949 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.591+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/depcomp auth{0=1} v323 ap=1 s=15205 n(v0 rc2024-05-27T10:21:32.225999+0000 b15205 1=1+0) caps={24925=pAsLsXsFscr/-@30},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x558dd3bc6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021d:[<0x10000000201/depcomp v56>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000021d:[<0x10000000201/depcomp v323>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.949 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.591+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/compile auth v519 ap=1 s=3695 n(v0 rc2024-05-27T10:21:32.220054+0000 b3695 1=1+0) caps={24925=pAsLsXsFscr/-@29},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3bc7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021c:[<0x10000000201/compile v54>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000021c:[<0x10000000201/compile v519>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.949 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.593+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/aclocal.m4 auth v521 ap=1 s=38531 n(v0 rc2024-05-27T10:21:32.233564+0000 b38531 1=1+0) caps={24925=pAsLsXsFscr/-@29},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c25080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021e:[<0x10000000201/aclocal.m4 v58>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000021e:[<0x10000000201/aclocal.m4 v521>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.949 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.594+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/NEWS auth v517 ap=1 s=33 n(v0 rc2024-05-27T10:21:32.215147+0000 b33 1=1+0) caps={24925=pAsLsXsFscr/-@29},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c25b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021b:[<0x10000000201/NEWS v52>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000021b:[<0x10000000201/NEWS v517>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.950 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.596+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [...4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/ auth v469 ap=1 f(v0 m2024-05-27T10:21:32.038989+0000 1=1+0) n(v0 rc2024-05-27T10:21:32.048428+0000 b9243 2=1+1) (inest lock) (ifile excl) caps={24925=pAsLsXsFsx/AsLsXsFsx@54},l=24925 | ptrwaiter=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c26c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000202:[<0x10000000201/m4 v4>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x10000000202:[<0x10000000201/m4 v469>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-27T10:21:32.038989+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-05-27T10:21:32.045106+0000 b9243 2=1+1)","memory_value.dirstat":"f(v0 m2024-05-27T10:21:32.038989+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-27T10:21:32.048428+0000 b9243 2=1+1)","error_str":""},"return_code":0} 2024-05-27T10:21:49.950 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.604+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/blogbench.8 auth v8 ap=1 s=964 n(v0 rc2024-05-27T10:21:32.061792+0000 b964 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c30c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000205:[<0x10000000204/blogbench.8 v2>,<0x10000000201/man v12>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000205:[<0x10000000204/blogbench.8 v8>,<0x10000000201/man v477>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.950 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.604+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.in auth v12 ap=1 s=9817 n(v0 rc2024-05-27T10:21:32.078111+0000 b9817 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000207:[<0x10000000204/Makefile.in v6>,<0x10000000201/man v12>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000207:[<0x10000000204/Makefile.in v12>,<0x10000000201/man v477>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.950 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.605+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.am auth v10 ap=1 s=55 n(v0 rc2024-05-27T10:21:32.069722+0000 b55 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c31700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000206:[<0x10000000204/Makefile.am v4>,<0x10000000201/man v12>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000206:[<0x10000000204/Makefile.am v10>,<0x10000000201/man v477>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.951 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.605+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.c auth v42 ap=1 s=2117 n(v0 rc2024-05-27T10:21:32.107302+0000 b2117 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c38000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020b:[<0x10000000208/blogbench.c v6>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000020b:[<0x10000000208/blogbench.c v42>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.951 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.606+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/globals.h auth v62 ap=1 s=1419 n(v0 rc2024-05-27T10:21:32.182559+0000 b1419 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000215:[<0x10000000208/globals.h v26>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000215:[<0x10000000208/globals.h v62>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.951 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.606+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/writer.c auth v70 ap=1 s=1910 n(v0 rc2024-05-27T10:21:32.205084+0000 b1910 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000208/writer.c v34>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000208/writer.c v70>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.951 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.607+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.c auth v38 ap=1 s=729 n(v0 rc2024-05-27T10:21:32.090838+0000 b729 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c39080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000209:[<0x10000000208/mysnprintf.c v2>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000209:[<0x10000000208/mysnprintf.c v38>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.951 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.607+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/commenter.c auth v64 ap=1 s=939 n(v0 rc2024-05-27T10:21:32.188268+0000 b939 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c39b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000208/commenter.c v28>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000208/commenter.c v64>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.952 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.607+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/helpers.c auth v72 ap=1 s=4200 n(v0 rc2024-05-27T10:21:32.210143+0000 b4200 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c3a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000208/helpers.c v36>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000208/helpers.c v72>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.952 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.608+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench_p.h auth v50 ap=1 s=460 n(v0 rc2024-05-27T10:21:32.138941+0000 b460 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c3ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020f:[<0x10000000208/blogbench_p.h v14>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000020f:[<0x10000000208/blogbench_p.h v50>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.952 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.609+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h auth v54 ap=1 s=4944 n(v0 rc2024-05-27T10:21:32.155539+0000 b4944 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c39600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000211:[<0x10000000208/bsd-getopt_long.h v18>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000211:[<0x10000000208/bsd-getopt_long.h v54>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.952 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.609+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.h auth v44 ap=1 s=3764 n(v0 rc2024-05-27T10:21:32.115592+0000 b3764 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c3b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020c:[<0x10000000208/blogbench.h v8>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000020c:[<0x10000000208/blogbench.h v44>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.952 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.609+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/rewriter.c auth v66 ap=1 s=1581 n(v0 rc2024-05-27T10:21:32.193860+0000 b1581 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c3b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000208/rewriter.c v30>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000208/rewriter.c v66>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.953 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.610+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.h auth v40 ap=1 s=696 n(v0 rc2024-05-27T10:21:32.098816+0000 b696 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c3a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020a:[<0x10000000208/mysnprintf.h v4>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000020a:[<0x10000000208/mysnprintf.h v40>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.953 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.612+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.c auth v46 ap=1 s=10682 n(v0 rc2024-05-27T10:21:32.126587+0000 b10682 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c40580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x10000000208/fakesnprintf.c v10>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000020d:[<0x10000000208/fakesnprintf.c v46>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v436>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.953 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.613+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.in auth v58 ap=1 s=13775 n(v0 rc2024-05-27T10:21:32.170376+0000 b13775 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c40000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000213:[<0x10000000208/Makefile.in v22>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000213:[<0x10000000208/Makefile.in v58>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.953 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.613+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.am auth v56 ap=1 s=299 n(v0 rc2024-05-27T10:21:32.162401+0000 b299 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c40b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000212:[<0x10000000208/Makefile.am v20>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000212:[<0x10000000208/Makefile.am v56>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.953 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.614+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c auth v52 ap=1 s=18847 n(v0 rc2024-05-27T10:21:32.149125+0000 b18847 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c41080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000210:[<0x10000000208/bsd-getopt_long.c v16>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000210:[<0x10000000208/bsd-getopt_long.c v52>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.953 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.615+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/reader.c auth v68 ap=1 s=1605 n(v0 rc2024-05-27T10:21:32.199764+0000 b1605 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000208/reader.c v32>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000208/reader.c v68>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.954 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.615+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/process.c auth v60 ap=1 s=4649 n(v0 rc2024-05-27T10:21:32.177561+0000 b4649 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c41b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000214:[<0x10000000208/process.c v24>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000214:[<0x10000000208/process.c v60>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.954 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.615+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.h auth v48 ap=1 s=606 n(v0 rc2024-05-27T10:21:32.132910+0000 b606 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c42100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x10000000208/fakesnprintf.h v12>,<0x10000000201/src v50>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x1000000020e:[<0x10000000208/fakesnprintf.h v48>,<0x10000000201/src v515>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.954 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:21:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:21:49.615+0000 7fe187600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [4,head] /volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/acx_pthread.m4 auth v4 ap=1 s=9243 n(v0 rc2024-05-27T10:21:32.045106+0000 b9243 1=1+0) caps={24925=pAsLsXsFsc/-@1},l=24925(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558dd3c43700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000203:[<0x10000000202/acx_pthread.m4 v2>,<0x10000000201/m4 v4>,<0x10000000006/blogbench-1.0 v66>,<0x10000000005/tmp v14>,<0x10000000003/client.0 v6>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v47>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000203:[<0x10000000202/acx_pthread.m4 v4>,<0x10000000201/m4 v469>,<0x10000000006/blogbench-1.0 v438>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v34>,<0x10000000002/8b372384-d91f-4b21-9dce-13ee6ea8be63 v22>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v35>,<0x1/volumes v32>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-27T10:21:49.968 INFO:tasks.workunit.client.0.smithi042.stdout:checking for working alloca.h... yes 2024-05-27T10:21:50.297 INFO:tasks.workunit.client.0.smithi042.stdout:checking for alloca... yes 2024-05-27T10:21:50.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:50 smithi143 ceph-mon[35545]: pgmap v96: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 220 KiB/s wr, 83 op/s 2024-05-27T10:21:50.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:50 smithi143 ceph-mon[35545]: scrub queued for path: / 2024-05-27T10:21:50.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:50 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:50.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:50 smithi143 ceph-mon[35545]: scrub summary: active paths [/] 2024-05-27T10:21:50.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:50 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:50.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:50 smithi042 ceph-mon[31661]: pgmap v96: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 220 KiB/s wr, 83 op/s 2024-05-27T10:21:50.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:50 smithi042 ceph-mon[31661]: scrub queued for path: / 2024-05-27T10:21:50.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:50 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:50.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:50 smithi042 ceph-mon[31661]: scrub summary: active paths [/] 2024-05-27T10:21:50.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:50 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:50.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:50 smithi104 ceph-mon[34812]: pgmap v96: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 220 KiB/s wr, 83 op/s 2024-05-27T10:21:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:50 smithi104 ceph-mon[34812]: scrub queued for path: / 2024-05-27T10:21:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:50 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:50.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:50 smithi104 ceph-mon[34812]: scrub summary: active paths [/] 2024-05-27T10:21:50.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:50 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:50.640 INFO:tasks.workunit.client.0.smithi042.stdout:checking for working memcmp... yes 2024-05-27T10:21:50.734 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:21:50.973 INFO:tasks.workunit.client.0.smithi042.stdout:checking for strftime... yes 2024-05-27T10:21:51.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: scrub summary: active paths [/] 2024-05-27T10:21:51.067 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:51.294 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/depcomp) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/compile) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/aclocal.m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/NEWS) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000202 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/blogbench.8) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.in) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.am) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/globals.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/writer.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/reader.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/process.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/acx_pthread.m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:51 smithi042 ceph-mon[31661]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: scrub summary: active paths [/] 2024-05-27T10:21:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/depcomp) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/compile) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/aclocal.m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/NEWS) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000202 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/blogbench.8) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.in) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.am) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/globals.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/writer.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/reader.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/process.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/acx_pthread.m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:51 smithi104 ceph-mon[34812]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:51.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: scrub summary: active paths [/] 2024-05-27T10:21:51.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/depcomp) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/compile) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/aclocal.m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/NEWS) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000202 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/blogbench.8) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.in) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.am) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.559 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/globals.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/writer.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/reader.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/process.c) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/acx_pthread.m4) see mds.h log and `damage ls` output for details 2024-05-27T10:21:51.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:51 smithi143 ceph-mon[35545]: scrub summary: idle+waiting paths [/] 2024-05-27T10:21:51.659 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether stat accepts an empty string... no 2024-05-27T10:21:51.970 INFO:tasks.workunit.client.0.smithi042.stdout:checking for vprintf... yes 2024-05-27T10:21:52.268 INFO:tasks.workunit.client.0.smithi042.stdout:checking for _doprnt... no 2024-05-27T10:21:52.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:52 smithi042 ceph-mon[31661]: pgmap v97: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 137 KiB/s wr, 57 op/s 2024-05-27T10:21:52.400 INFO:tasks.workunit.client.0.smithi042.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-27T10:21:52.443 INFO:tasks.workunit.client.0.smithi042.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-27T10:21:52.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:52 smithi104 ceph-mon[34812]: pgmap v97: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 137 KiB/s wr, 57 op/s 2024-05-27T10:21:52.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:52 smithi143 ceph-mon[35545]: pgmap v97: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 137 KiB/s wr, 57 op/s 2024-05-27T10:21:52.752 INFO:tasks.workunit.client.0.smithi042.stdout:checking for the pthreads library -lpthreads... no 2024-05-27T10:21:53.047 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether pthreads work without any flags... yes 2024-05-27T10:21:53.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31657]: 2024-05-27T10:21:53.070+0000 7fd90f000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-27T10:21:53.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:53 smithi042 ceph-mon[31661]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-27T10:21:53.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:53 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:active 2024-05-27T10:21:53.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:53 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:53.259 INFO:tasks.workunit.client.0.smithi042.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-05-27T10:21:53.260 INFO:tasks.workunit.client.0.smithi042.stdout:checking if more special flags are required for pthreads... no 2024-05-27T10:21:53.260 INFO:tasks.workunit.client.0.smithi042.stdout:checking for cc_r... gcc 2024-05-27T10:21:53.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:53 smithi104 ceph-mon[34812]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-27T10:21:53.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:53 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:active 2024-05-27T10:21:53.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:53 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:53.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:53 smithi143 ceph-mon[35545]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-27T10:21:53.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:53 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] up:active 2024-05-27T10:21:53.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:53 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:21:53.609 INFO:tasks.workunit.client.0.smithi042.stdout:checking for setrlimit... yes 2024-05-27T10:21:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.923+0000 7f6eb2a00700 1 -- 172.21.15.42:0/3999788911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac1045e0 msgr2=0x7f6eac1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.923+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/3999788911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac1045e0 0x7f6eac1049f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0099c0 tx=0x7f6e9c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 -- 172.21.15.42:0/3999788911 shutdown_connections 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/3999788911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac109860 0x7f6eac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/3999788911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac1045e0 0x7f6eac1049f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/3999788911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eac108f10 0x7f6eac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 -- 172.21.15.42:0/3999788911 >> 172.21.15.42:0/3999788911 conn(0x7f6eac100030 msgr2=0x7f6eac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 -- 172.21.15.42:0/3999788911 shutdown_connections 2024-05-27T10:21:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.924+0000 7f6eb2a00700 1 -- 172.21.15.42:0/3999788911 wait complete. 2024-05-27T10:21:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.925+0000 7f6eb2a00700 1 Processor -- start 2024-05-27T10:21:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.925+0000 7f6eb2a00700 1 -- start start 2024-05-27T10:21:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb2a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1045e0 0x7f6eac075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb2a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eac108f10 0x7f6eac075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 0x7f6eac076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb2a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac114340 con 0x7f6eac109860 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb2a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1141c0 con 0x7f6eac108f10 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb2a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1144c0 con 0x7f6eac1045e0 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb0a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 0x7f6eac076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb0a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 0x7f6eac076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44862/0 (socket says 172.21.15.42:44862) 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eb0a00700 1 -- 172.21.15.42:0/493495128 learned_addr learned my addr 172.21.15.42:0/493495128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.926+0000 7f6eabe00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1045e0 0x7f6eac075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6ea3400700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eac108f10 0x7f6eac075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eb0a00700 1 -- 172.21.15.42:0/493495128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1045e0 msgr2=0x7f6eac075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eb0a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1045e0 0x7f6eac075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eb0a00700 1 -- 172.21.15.42:0/493495128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eac108f10 msgr2=0x7f6eac075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eb0a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eac108f10 0x7f6eac075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eb0a00700 1 -- 172.21.15.42:0/493495128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1a91b0 con 0x7f6eac109860 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eabe00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1045e0 0x7f6eac075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6eb0a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 0x7f6eac076530 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4007ee0 tx=0x7f6ea400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6ea9600700 1 -- 172.21.15.42:0/493495128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400ddd0 con 0x7f6eac109860 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6ea9600700 1 -- 172.21.15.42:0/493495128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ea4004020 con 0x7f6eac109860 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.927+0000 7f6ea9600700 1 -- 172.21.15.42:0/493495128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4015660 con 0x7f6eac109860 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.928+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac1a94a0 con 0x7f6eac109860 2024-05-27T10:21:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.928+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1a99e0 con 0x7f6eac109860 2024-05-27T10:21:53.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.933+0000 7f6ea9600700 1 -- 172.21.15.42:0/493495128 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6ea40041d0 con 0x7f6eac109860 2024-05-27T10:21:53.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.934+0000 7f6ec1620b80 1 Processor -- start 2024-05-27T10:21:53.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.934+0000 7f6ec1620b80 1 -- start start 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.934+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e74ffe40 0x5636e76847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e767e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 0x5636e767f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5636e75ace40 con 0x5636e767edb0 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636e759c9e0 con 0x5636e74ffe40 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5636e75a0a20 con 0x5636e7684d30 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 0x5636e767f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ea9600700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6e900b2a70 0x7f6e900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 0x5636e767f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44870/0 (socket says 172.21.15.42:44870) 2024-05-27T10:21:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ea3400700 1 -- 172.21.15.42:0/2178850326 learned_addr learned my addr 172.21.15.42:0/2178850326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e74ffe40 0x5636e76847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6eabe00700 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e767e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.935+0000 7f6ea9600700 1 -- 172.21.15.42:0/493495128 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6ea40e1ff0 con 0x7f6eac109860 2024-05-27T10:21:53.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6eb0a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6e900b2a70 0x7f6e900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea3400700 1 -- 172.21.15.42:0/2178850326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 msgr2=0x5636e767e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e767e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea3400700 1 -- 172.21.15.42:0/2178850326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e74ffe40 msgr2=0x5636e76847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e74ffe40 0x5636e76847f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea3400700 1 -- 172.21.15.42:0/2178850326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e7565420 con 0x5636e767edb0 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 0x5636e767f1c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa7e7c29222297e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6eb0a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6e900b2a70 0x7f6e900b4f80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40a92b0 tx=0x7f6ea4004ad0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.936+0000 7f6ea1600700 1 -- 172.21.15.42:0/2178850326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94012db0 con 0x5636e767edb0 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.937+0000 7f6ea1600700 1 -- 172.21.15.42:0/2178850326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e940044c0 con 0x5636e767edb0 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.937+0000 7f6ea1600700 1 -- 172.21.15.42:0/2178850326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400f450 con 0x5636e767edb0 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.937+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2178850326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 msgr2=0x5636e767f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.937+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 0x5636e767f1c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94009770 comp rx=0 tx=0).stop 2024-05-27T10:21:53.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2178850326 shutdown_connections 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e767edb0 0x5636e767f1c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e74ffe40 0x5636e76847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2178850326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e767e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2178850326 >> 172.21.15.42:0/2178850326 conn(0x5636e755ffc0 msgr2=0x5636e76798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2178850326 shutdown_connections 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2178850326 wait complete. 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.938+0000 7f6ec1620b80 1 -- start start 2024-05-27T10:21:53.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 Processor -- start 2024-05-27T10:21:53.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e7688510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e7688a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e7688f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5636e74a2fa0 con 0x5636e7684d30 2024-05-27T10:21:53.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636e753d760 con 0x5636e767edb0 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5636e768b800 con 0x5636e74ffe40 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.939+0000 7f6ec1620b80 10 client.0 _ll_register_callbacks cb 0x5636e7546c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e7688f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e7688f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44884/0 (socket says 172.21.15.42:44884) 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 -- 172.21.15.42:0/2797625586 learned_addr learned my addr 172.21.15.42:0/2797625586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e7688510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6eabe00700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e7688a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 -- 172.21.15.42:0/2797625586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 msgr2=0x5636e7688510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e7688510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 -- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 msgr2=0x5636e7688a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e7688a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.940+0000 7f6ea3400700 1 -- 172.21.15.42:0/2797625586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e74ec8d0 con 0x5636e7684d30 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6ea3400700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e7688f90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6e94012f90 tx=0x7f6e94017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d720e69e059cf17 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6e9a000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6e9a000700 1 -- 172.21.15.42:0/2797625586 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400f450 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e940044c0 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6e9a000700 1 -- 172.21.15.42:0/2797625586 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6e9400f5f0 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6e9a000700 1 -- 172.21.15.42:0/2797625586 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e9400f7f0 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6ec1620b80 10 client.16017 fetch_fsmap learned FSMap version 29 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636e756d060 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.941+0000 7f6e9a000700 1 -- 172.21.15.42:0/2797625586 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e940044c0 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6e9a000700 1 -- 172.21.15.42:0/2797625586 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6e9402e880 con 0x5636e7684d30 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6ec1620b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:21:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6ec1620b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6ec1620b80 20 client.16017 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e76985a0 0x5636e769a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6ec1620b80 4 client.16017 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e75aed90 con 0x5636e76985a0 2024-05-27T10:21:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.942+0000 7f6eabe00700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e76985a0 0x5636e769a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.943+0000 7f6eabe00700 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e76985a0 0x5636e769a9c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228eb78d052edc81 server_cookie=9ba8506d1fd1b35d in_seq=0 out_seq=0 2024-05-27T10:21:53.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.943+0000 7f6e9a000700 10 client.16017 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:53.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.945+0000 7f6e9a000700 1 -- 172.21.15.42:0/2797625586 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636e75aed90 con 0x5636e76985a0 2024-05-27T10:21:53.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.945+0000 7f6e9a000700 10 client.16017 handle_command_reply: tid=0 2024-05-27T10:21:53.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.945+0000 7f6ec1620b80 1 client.16017 shutdown 2024-05-27T10:21:53.949 INFO:tasks.workunit.client.0.smithi042.stdout:checking for setproctitle... no 2024-05-27T10:21:53.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 msgr2=0x5636e7688f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e7688f90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6e94012f90 tx=0x7f6e94017040 comp rx=0 tx=0).stop 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 shutdown_connections 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e76985a0 0x5636e769a9c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e7688f90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e7688510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/2797625586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e7688a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 >> 172.21.15.42:0/2797625586 conn(0x5636e755ffc0 msgr2=0x5636e7679580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6e9a000700 0 client.16017 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:53.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 shutdown_connections 2024-05-27T10:21:53.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 1 -- 172.21.15.42:0/2797625586 wait complete. 2024-05-27T10:21:53.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.946+0000 7f6ec1620b80 20 client.16017 trim_cache size 0 max 16384 2024-05-27T10:21:53.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 Processor -- start 2024-05-27T10:21:53.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 -- start start 2024-05-27T10:21:53.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 0x5636e767e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e768a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e7694aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1b3760 con 0x5636e74ffe40 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1b0ff0 con 0x5636e767edb0 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1aee60 con 0x5636e7684d30 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 0x5636e767e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 0x5636e767e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44894/0 (socket says 172.21.15.42:44894) 2024-05-27T10:21:53.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.971+0000 7f6ea3400700 1 -- 172.21.15.42:0/4249741507 learned_addr learned my addr 172.21.15.42:0/4249741507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6eabe00700 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e768a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6ea3400700 1 -- 172.21.15.42:0/4249741507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 msgr2=0x5636e7694aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e7694aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6ea3400700 1 -- 172.21.15.42:0/4249741507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 msgr2=0x5636e768a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e768a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6ea3400700 1 -- 172.21.15.42:0/4249741507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e7682880 con 0x5636e74ffe40 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.972+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 0x5636e767e870 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009d70 tx=0x7f6e9402ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c613093eff632bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:53.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.973+0000 7f6ea0c00700 1 -- 172.21.15.42:0/4249741507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94020440 con 0x5636e74ffe40 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.973+0000 7f6ea0c00700 1 -- 172.21.15.42:0/4249741507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e94032070 con 0x5636e74ffe40 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.973+0000 7f6ea0c00700 1 -- 172.21.15.42:0/4249741507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400fa80 con 0x5636e74ffe40 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.973+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4249741507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 msgr2=0x5636e767e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 0x5636e767e870 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009d70 tx=0x7f6e9402ee80 comp rx=0 tx=0).stop 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4249741507 shutdown_connections 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e7684d30 0x5636e7694aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e74ffe40 0x5636e767e870 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4249741507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e768a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4249741507 >> 172.21.15.42:0/4249741507 conn(0x5636e755ffc0 msgr2=0x5636e76884d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:53.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4249741507 shutdown_connections 2024-05-27T10:21:53.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4249741507 wait complete. 2024-05-27T10:21:53.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 -- start start 2024-05-27T10:21:53.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.974+0000 7f6ec1620b80 1 Processor -- start 2024-05-27T10:21:53.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e769dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e769af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e769b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1af4b0 con 0x5636e7684d30 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1b0920 con 0x5636e767edb0 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1afb80 con 0x5636e74ffe40 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ec1620b80 10 client.0 _ll_register_callbacks cb 0x5636e7546c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e769b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ea3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e769b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44906/0 (socket says 172.21.15.42:44906) 2024-05-27T10:21:53.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.975+0000 7f6ea3400700 1 -- 172.21.15.42:0/4028556938 learned_addr learned my addr 172.21.15.42:0/4028556938 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6eabe00700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e769af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e769dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 -- 172.21.15.42:0/4028556938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 msgr2=0x5636e769dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e769dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 -- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 msgr2=0x5636e769af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e769af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 -- 172.21.15.42:0/4028556938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e769bab0 con 0x5636e7684d30 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6eabe00700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e769af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:53.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6ea3400700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e769b470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009740 tx=0x7f6e94005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7199bd528255381 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.976+0000 7f6e9a000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6e9a000700 1 -- 172.21.15.42:0/4028556938 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e940054a0 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6e9a000700 1 -- 172.21.15.42:0/4028556938 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6e9402a070 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e94032070 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6e9a000700 1 -- 172.21.15.42:0/4028556938 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e9400f990 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6ec1620b80 10 client.16023 fetch_fsmap learned FSMap version 29 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636e7702bf0 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6e9a000700 1 -- 172.21.15.42:0/4028556938 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e94032070 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.977+0000 7f6e9a000700 1 -- 172.21.15.42:0/4028556938 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6e9402e910 con 0x5636e7684d30 2024-05-27T10:21:53.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6ec1620b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6ec1620b80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6ec1620b80 20 client.16023 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e77030c0 0x5636e77054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6ec1620b80 4 client.16023 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636e75a8570 con 0x5636e77030c0 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.978+0000 7f6eabe00700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e77030c0 0x5636e77054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.979+0000 7f6eabe00700 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e77030c0 0x5636e77054e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feeb9fb5c2da243b server_cookie=5e1cb4f0e4e507cc in_seq=0 out_seq=0 2024-05-27T10:21:53.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.979+0000 7f6e9a000700 10 client.16023 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:53.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.983+0000 7f6e9a000700 1 -- 172.21.15.42:0/4028556938 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5636e75a8570 con 0x5636e77030c0 2024-05-27T10:21:53.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.983+0000 7f6e9a000700 10 client.16023 handle_command_reply: tid=0 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.983+0000 7f6ec1620b80 1 client.16023 shutdown 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 msgr2=0x5636e769b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e769b470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009740 tx=0x7f6e94005760 comp rx=0 tx=0).stop 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 shutdown_connections 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636e77030c0 0x5636e77054e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636e7684d30 0x5636e769b470 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636e74ffe40 0x5636e769dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6e9a000700 0 client.16023 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:53.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 --2- 172.21.15.42:0/4028556938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636e767edb0 0x5636e769af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 >> 172.21.15.42:0/4028556938 conn(0x5636e755ffc0 msgr2=0x5636e7683e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 shutdown_connections 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 1 -- 172.21.15.42:0/4028556938 wait complete. 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.984+0000 7f6ec1620b80 20 client.16023 trim_cache size 0 max 16384 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-05-27T10:21:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "28fdbfa8-9597-4606-b729-fd1e3d1cc3f8": { 2024-05-27T10:21:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-05-27T10:21:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "28fdbfa8-9597-4606-b729-fd1e3d1cc3f8", 2024-05-27T10:21:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-05-27T10:21:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:21:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:21:53.986 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6e900b2a70 msgr2=0x7f6e900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6e900b2a70 0x7f6e900b4f80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40a92b0 tx=0x7f6ea4004ad0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 msgr2=0x7f6eac076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 0x7f6eac076530 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4007ee0 tx=0x7f6ea400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 shutdown_connections 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6e900b2a70 0x7f6e900b4f80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac109860 0x7f6eac076530 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1045e0 0x7f6eac075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 --2- 172.21.15.42:0/493495128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eac108f10 0x7f6eac075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:53.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 >> 172.21.15.42:0/493495128 conn(0x7f6eac100030 msgr2=0x7f6eac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:53.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 shutdown_connections 2024-05-27T10:21:53.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:53.985+0000 7f6eb2a00700 1 -- 172.21.15.42:0/493495128 wait complete. 2024-05-27T10:21:54.244 INFO:tasks.workunit.client.0.smithi042.stdout:checking for getopt_long... yes 2024-05-27T10:21:54.425 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-27T10:21:54.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:21:54.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:54 smithi104 ceph-mon[34812]: pgmap v98: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 137 KiB/s wr, 57 op/s 2024-05-27T10:21:54.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:54 smithi042 ceph-mon[31661]: pgmap v98: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 137 KiB/s wr, 57 op/s 2024-05-27T10:21:54.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:54 smithi143 ceph-mon[35545]: pgmap v98: 97 pgs: 97 active+clean; 4.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 137 KiB/s wr, 57 op/s 2024-05-27T10:21:54.601 INFO:tasks.workunit.client.0.smithi042.stdout:checking for strtoull... yes 2024-05-27T10:21:54.974 INFO:tasks.workunit.client.0.smithi042.stdout:checking for strtoq... yes 2024-05-27T10:21:55.171 INFO:tasks.cephfs.filesystem:scrub status for tag:28fdbfa8-9597-4606-b729-fd1e3d1cc3f8 - {'path': '/', 'tag': '28fdbfa8-9597-4606-b729-fd1e3d1cc3f8', 'options': 'recursive,force'} 2024-05-27T10:21:55.320 INFO:tasks.workunit.client.0.smithi042.stdout:checking for snprintf... yes 2024-05-27T10:21:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:21:55.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:55 smithi104 ceph-mon[34812]: scrub summary: idle 2024-05-27T10:21:55.487 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:55 smithi042 ceph-mon[31661]: scrub summary: idle 2024-05-27T10:21:55.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:55 smithi143 ceph-mon[35545]: scrub summary: idle 2024-05-27T10:21:55.698 INFO:tasks.workunit.client.0.smithi042.stdout:checking for vsnprintf... yes 2024-05-27T10:21:56.073 INFO:tasks.workunit.client.0.smithi042.stdout:checking for vfprintf... yes 2024-05-27T10:21:56.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:56 smithi042 ceph-mon[31661]: scrub summary: idle 2024-05-27T10:21:56.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:56 smithi042 ceph-mon[31661]: pgmap v99: 97 pgs: 97 active+clean; 4.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 210 KiB/s wr, 98 op/s 2024-05-27T10:21:56.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:56 smithi104 ceph-mon[34812]: scrub summary: idle 2024-05-27T10:21:56.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:56 smithi104 ceph-mon[34812]: pgmap v99: 97 pgs: 97 active+clean; 4.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 210 KiB/s wr, 98 op/s 2024-05-27T10:21:56.493 INFO:tasks.workunit.client.0.smithi042.stdout:checking for gethostname... yes 2024-05-27T10:21:56.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:56 smithi143 ceph-mon[35545]: scrub summary: idle 2024-05-27T10:21:56.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:56 smithi143 ceph-mon[35545]: pgmap v99: 97 pgs: 97 active+clean; 4.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 210 KiB/s wr, 98 op/s 2024-05-27T10:21:56.892 INFO:tasks.workunit.client.0.smithi042.stdout:checking for setlocale... yes 2024-05-27T10:21:57.248 INFO:tasks.workunit.client.0.smithi042.stdout:checking for sync... yes 2024-05-27T10:21:57.705 INFO:tasks.workunit.client.0.smithi042.stdout:checking whether snprintf is C99 conformant... done 2024-05-27T10:21:57.800 INFO:tasks.workunit.client.0.smithi042.stdout:configure: creating ./config.status 2024-05-27T10:21:58.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:21:58 smithi042 ceph-mon[31661]: pgmap v100: 97 pgs: 97 active+clean; 4.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 145 KiB/s wr, 73 op/s 2024-05-27T10:21:58.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:21:58 smithi104 ceph-mon[34812]: pgmap v100: 97 pgs: 97 active+clean; 4.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 145 KiB/s wr, 73 op/s 2024-05-27T10:21:58.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:21:58 smithi143 ceph-mon[35545]: pgmap v100: 97 pgs: 97 active+clean; 4.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 145 KiB/s wr, 73 op/s 2024-05-27T10:21:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 -- 172.21.15.42:0/3813944082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 msgr2=0x7fdbd4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 --2- 172.21.15.42:0/3813944082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4109320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00099c0 tx=0x7fdbc0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 -- 172.21.15.42:0/3813944082 shutdown_connections 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 --2- 172.21.15.42:0/3813944082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd4109860 0x7fdbd41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 --2- 172.21.15.42:0/3813944082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd41045e0 0x7fdbd41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 --2- 172.21.15.42:0/3813944082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4109320 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 -- 172.21.15.42:0/3813944082 >> 172.21.15.42:0/3813944082 conn(0x7fdbd4100030 msgr2=0x7fdbd4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 -- 172.21.15.42:0/3813944082 shutdown_connections 2024-05-27T10:21:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.815+0000 7fdbd9000700 1 -- 172.21.15.42:0/3813944082 wait complete. 2024-05-27T10:21:58.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 Processor -- start 2024-05-27T10:21:58.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 -- start start 2024-05-27T10:21:58.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd41045e0 0x7fdbd4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:58.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd4109860 0x7fdbd4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4114390 con 0x7fdbd4108f10 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4114210 con 0x7fdbd4109860 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.817+0000 7fdbd9000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4114510 con 0x7fdbd41045e0 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44912/0 (socket says 172.21.15.42:44912) 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 -- 172.21.15.42:0/1292337799 learned_addr learned my addr 172.21.15.42:0/1292337799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1e00700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd41045e0 0x7fdbd4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:58.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd2800700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd4109860 0x7fdbd4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 -- 172.21.15.42:0/1292337799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd41045e0 msgr2=0x7fdbd4075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd41045e0 0x7fdbd4075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 -- 172.21.15.42:0/1292337799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd4109860 msgr2=0x7fdbd4076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd4109860 0x7fdbd4076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1400700 1 -- 172.21.15.42:0/1292337799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd41a90f0 con 0x7fdbd4108f10 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd1e00700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd41045e0 0x7fdbd4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.818+0000 7fdbd2800700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd4109860 0x7fdbd4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:58.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.819+0000 7fdbd1400700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4075ff0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000b580 tx=0x7fdbc000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:58.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.819+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0027070 con 0x7fdbd4108f10 2024-05-27T10:21:58.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.819+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd41a9380 con 0x7fdbd4108f10 2024-05-27T10:21:58.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.819+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc00056d0 con 0x7fdbd4108f10 2024-05-27T10:21:58.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.819+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00173f0 con 0x7fdbd4108f10 2024-05-27T10:21:58.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.819+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41a97d0 con 0x7fdbd4108f10 2024-05-27T10:21:58.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.821+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd40512b0 con 0x7fdbd4108f10 2024-05-27T10:21:58.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.824+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdbc002a070 con 0x7fdbd4108f10 2024-05-27T10:21:58.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.826+0000 7fdbcaa00700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdbb80b29f0 0x7fdbb80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:58.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.826+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fdbc00e3770 con 0x7fdbd4108f10 2024-05-27T10:21:58.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.826+0000 7fdbd1e00700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdbb80b29f0 0x7fdbb80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:58.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.827+0000 7fdbd1e00700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdbb80b29f0 0x7fdbb80b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4005fd0 tx=0x7fdbc4005e30 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:58.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:58.829+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbc0013070 con 0x7fdbd4108f10 2024-05-27T10:21:58.966 INFO:tasks.workunit.client.0.smithi042.stdout:config.status: creating Makefile 2024-05-27T10:21:59.081 INFO:tasks.workunit.client.0.smithi042.stdout:config.status: creating src/Makefile 2024-05-27T10:21:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.155+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd402e5e0 con 0x7fdbd4108f10 2024-05-27T10:21:59.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.156+0000 7fdbcaa00700 1 -- 172.21.15.42:0/1292337799 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11881 (secure 0 0 0) 0x7fdbc00ab260 con 0x7fdbd4108f10 2024-05-27T10:21:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:21:59.156 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:21:52.648444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:21:59.156 INFO:tasks.workunit.client.0.smithi042.stdout:config.status: creating man/Makefile 2024-05-27T10:21:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 -- 172.21.15.42:0/2796812971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 msgr2=0x7fcc5c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 --2- 172.21.15.42:0/2796812971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c103be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc4802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 -- 172.21.15.42:0/2796812971 shutdown_connections 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 --2- 172.21.15.42:0/2796812971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc5c104250 0x7fcc5c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 --2- 172.21.15.42:0/2796812971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c103be0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 --2- 172.21.15.42:0/2796812971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc5c109ab0 0x7fcc5c1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 -- 172.21.15.42:0/2796812971 >> 172.21.15.42:0/2796812971 conn(0x7fcc5c079240 msgr2=0x7fcc5c079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 -- 172.21.15.42:0/2796812971 shutdown_connections 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.158+0000 7fcc5b200640 1 -- 172.21.15.42:0/2796812971 wait complete. 2024-05-27T10:21:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.159+0000 7fcc5b200640 1 Processor -- start 2024-05-27T10:21:59.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.159+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdbb80b29f0 msgr2=0x7fdbb80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.159+0000 7fdbd9000700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdbb80b29f0 0x7fdbb80b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4005fd0 tx=0x7fdbc4005e30 comp rx=0 tx=0).stop 2024-05-27T10:21:59.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.159+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 msgr2=0x7fdbd4075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.159+0000 7fdbd9000700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4075ff0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000b580 tx=0x7fdbc000f740 comp rx=0 tx=0).stop 2024-05-27T10:21:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 -- start start 2024-05-27T10:21:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc5c104250 0x7fcc5c076b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc5c109ab0 0x7fcc5c074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c0755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 shutdown_connections 2024-05-27T10:21:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdbb80b29f0 0x7fdbb80b4f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbd4109860 0x7fdbd4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbd41045e0 0x7fdbd4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 --2- 172.21.15.42:0/1292337799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd4108f10 0x7fdbd4075ff0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 >> 172.21.15.42:0/1292337799 conn(0x7fdbd4100030 msgr2=0x7fdbd410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c113ba0 con 0x7fcc5c109f50 2024-05-27T10:21:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c113a20 con 0x7fcc5c109ab0 2024-05-27T10:21:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fcc5b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c113d20 con 0x7fcc5c104250 2024-05-27T10:21:59.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 shutdown_connections 2024-05-27T10:21:59.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:21:59.160+0000 7fdbd9000700 1 -- 172.21.15.42:0/1292337799 wait complete. 2024-05-27T10:21:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c0755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c0755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44926/0 (socket says 172.21.15.42:44926) 2024-05-27T10:21:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 -- 172.21.15.42:0/3731517605 learned_addr learned my addr 172.21.15.42:0/3731517605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc59400640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc5c109ab0 0x7fcc5c074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc59e00640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc5c104250 0x7fcc5c076b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 -- 172.21.15.42:0/3731517605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc5c104250 msgr2=0x7fcc5c076b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc5c104250 0x7fcc5c076b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 -- 172.21.15.42:0/3731517605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc5c109ab0 msgr2=0x7fcc5c074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc5c109ab0 0x7fcc5c074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.161+0000 7fcc5a800640 1 -- 172.21.15.42:0/3731517605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c075db0 con 0x7fcc5c109f50 2024-05-27T10:21:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.162+0000 7fcc5a800640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c0755c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400eaa0 tx=0x7fcc5400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:59.163 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-05-27T10:21:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.162+0000 7fcc52a00640 1 -- 172.21.15.42:0/3731517605 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400cdb0 con 0x7fcc5c109f50 2024-05-27T10:21:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.163+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1ad0c0 con 0x7fcc5c109f50 2024-05-27T10:21:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.163+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1ad590 con 0x7fcc5c109f50 2024-05-27T10:21:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.163+0000 7fcc52a00640 1 -- 172.21.15.42:0/3731517605 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc54005020 con 0x7fcc5c109f50 2024-05-27T10:21:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.163+0000 7fcc52a00640 1 -- 172.21.15.42:0/3731517605 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc540116d0 con 0x7fcc5c109f50 2024-05-27T10:21:59.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.166+0000 7fcc67fdf740 1 Processor -- start 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.167+0000 7fcc52a00640 1 -- 172.21.15.42:0/3731517605 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcc54004b20 con 0x7fcc5c109f50 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 -- start start 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63973a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a63973fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 0x557a639744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557a638e4b90 con 0x557a635b04d0 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557a638d4730 con 0x557a635a82f0 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.168+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557a638d8770 con 0x557a635d11b0 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 0x557a639744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 0x557a639744e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44936/0 (socket says 172.21.15.42:44936) 2024-05-27T10:21:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 -- 172.21.15.42:0/2921341264 learned_addr learned my addr 172.21.15.42:0/2921341264 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59e00640 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a63973fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63973a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 -- 172.21.15.42:0/2921341264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 msgr2=0x557a63973a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63973a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 -- 172.21.15.42:0/2921341264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 msgr2=0x557a63973fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a63973fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.169+0000 7fcc59400640 1 -- 172.21.15.42:0/2921341264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6386d8b0 con 0x557a635b04d0 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc52a00640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc400b29d0 0x7fcc400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc59400640 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 0x557a639744e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3d5f64518fc83c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc52a00640 1 -- 172.21.15.42:0/3731517605 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcc540e2940 con 0x7fcc5c109f50 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc59e00640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc400b29d0 0x7fcc400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc47400640 1 -- 172.21.15.42:0/2921341264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48035040 con 0x557a635b04d0 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc47400640 1 -- 172.21.15.42:0/2921341264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc48003e00 con 0x557a635b04d0 2024-05-27T10:21:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.170+0000 7fcc47400640 1 -- 172.21.15.42:0/2921341264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4803ddb0 con 0x557a635b04d0 2024-05-27T10:21:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.171+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2921341264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 msgr2=0x557a639744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.171+0000 7fcc59e00640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc400b29d0 0x7fcc400b4ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x557a63974e10 tx=0x7fcc4c00b390 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.171+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 0x557a639744e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48009610 comp rx=0 tx=0).stop 2024-05-27T10:21:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2921341264 shutdown_connections 2024-05-27T10:21:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635b04d0 0x557a639744e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a63973fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2921341264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63973a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2921341264 >> 172.21.15.42:0/2921341264 conn(0x557a635c5300 msgr2=0x557a63970210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2921341264 shutdown_connections 2024-05-27T10:21:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2921341264 wait complete. 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.172+0000 7fcc67fdf740 1 -- start start 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 Processor -- start 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a6397dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a6397e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635d11b0 0x557a63989ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557a635ba790 con 0x557a635d11b0 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557a6387cbc0 con 0x557a635a82f0 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.173+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557a63980b30 con 0x557a635b04d0 2024-05-27T10:21:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.174+0000 7fcc67fdf740 10 client.0 _ll_register_callbacks cb 0x557a6385a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.174+0000 7fcc59400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a6397e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.174+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a6397dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.174+0000 7fcc5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635d11b0 0x557a63989ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.174+0000 7fcc59400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a6397e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:59476/0 (socket says 172.21.15.42:59476) 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.174+0000 7fcc59400640 1 -- 172.21.15.42:0/2721096903 learned_addr learned my addr 172.21.15.42:0/2721096903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc5a800640 1 -- 172.21.15.42:0/2721096903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 msgr2=0x557a6397e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc5a800640 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a6397e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc5a800640 1 -- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 msgr2=0x557a6397dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc5a800640 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a6397dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc5a800640 1 -- 172.21.15.42:0/2721096903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a635e9210 con 0x557a635d11b0 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc5a800640 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635d11b0 0x557a63989ec0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcc540e0080 tx=0x7fcc540a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939f9768569ea0ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc3be00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.175+0000 7fcc3be00640 1 -- 172.21.15.42:0/2721096903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5401f030 con 0x557a635d11b0 2024-05-27T10:21:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.176+0000 7fcc3be00640 1 -- 172.21.15.42:0/2721096903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcc540b25e0 con 0x557a635d11b0 2024-05-27T10:21:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.176+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc48003e00 con 0x557a635d11b0 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.176+0000 7fcc3be00640 1 -- 172.21.15.42:0/2721096903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc540adfc0 con 0x557a635d11b0 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.176+0000 7fcc67fdf740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.176+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a636b5620 con 0x557a635d11b0 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc3be00640 1 -- 172.21.15.42:0/2721096903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc48003e00 con 0x557a635d11b0 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc3be00640 1 -- 172.21.15.42:0/2721096903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fcc5400f040 con 0x557a635d11b0 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc67fdf740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc67fdf740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc67fdf740 20 client.16041 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a6358fd80 0x557a63990040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc67fdf740 4 client.16041 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:21:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.177+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a638e6ae0 con 0x557a6358fd80 2024-05-27T10:21:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.178+0000 7fcc59e00640 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a6358fd80 0x557a63990040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.179+0000 7fcc3be00640 10 client.16041 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.179+0000 7fcc59e00640 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a6358fd80 0x557a63990040 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab61320ec6b8a0ac server_cookie=50ba39d0216b7f87 in_seq=0 out_seq=0 2024-05-27T10:21:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc3be00640 1 -- 172.21.15.42:0/2721096903 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557a638e6ae0 con 0x557a6358fd80 2024-05-27T10:21:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc3be00640 10 client.16041 handle_command_reply: tid=0 2024-05-27T10:21:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 client.16041 shutdown 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635d11b0 msgr2=0x557a63989ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635d11b0 0x557a63989ec0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcc540e0080 tx=0x7fcc540a9890 comp rx=0 tx=0).stop 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 shutdown_connections 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635d11b0 0x557a63989ec0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a6358fd80 0x557a63990040 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a6397e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2721096903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635a82f0 0x557a6397dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 >> 172.21.15.42:0/2721096903 conn(0x557a635c5300 msgr2=0x557a63975780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.181+0000 7fcc3be00640 0 client.16041 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.182+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 shutdown_connections 2024-05-27T10:21:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.182+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2721096903 wait complete. 2024-05-27T10:21:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.182+0000 7fcc67fdf740 20 client.16041 trim_cache size 0 max 16384 2024-05-27T10:21:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.203+0000 7fcc67fdf740 1 Processor -- start 2024-05-27T10:21:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.203+0000 7fcc67fdf740 1 -- start start 2024-05-27T10:21:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63980390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635b04d0 0x557a639739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63989c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c010e90 con 0x557a635a82f0 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c00e760 con 0x557a635b04d0 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c00c5b0 con 0x557a635d11b0 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63980390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63980390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44950/0 (socket says 172.21.15.42:44950) 2024-05-27T10:21:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc59e00640 1 -- 172.21.15.42:0/2756033436 learned_addr learned my addr 172.21.15.42:0/2756033436 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.204+0000 7fcc5a800640 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63989c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59400640 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635b04d0 0x557a639739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59e00640 1 -- 172.21.15.42:0/2756033436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 msgr2=0x557a63989c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59e00640 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63989c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59e00640 1 -- 172.21.15.42:0/2756033436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635b04d0 msgr2=0x557a639739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59e00640 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635b04d0 0x557a639739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59e00640 1 -- 172.21.15.42:0/2756033436 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a639737f0 con 0x557a635a82f0 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc59e00640 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63980390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x557a63971550 tx=0x7fcc4c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8e5064284918bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.205+0000 7fcc46a00640 1 -- 172.21.15.42:0/2756033436 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c02d070 con 0x557a635a82f0 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.206+0000 7fcc46a00640 1 -- 172.21.15.42:0/2756033436 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc4c028840 con 0x557a635a82f0 2024-05-27T10:21:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.206+0000 7fcc46a00640 1 -- 172.21.15.42:0/2756033436 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c027980 con 0x557a635a82f0 2024-05-27T10:21:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2756033436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 msgr2=0x557a63980390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63980390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x557a63971550 tx=0x7fcc4c015000 comp rx=0 tx=0).stop 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2756033436 shutdown_connections 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635d11b0 0x557a63989c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635b04d0 0x557a639739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/2756033436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63980390 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2756033436 >> 172.21.15.42:0/2756033436 conn(0x557a635c5300 msgr2=0x557a6397df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2756033436 shutdown_connections 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.207+0000 7fcc67fdf740 1 -- 172.21.15.42:0/2756033436 wait complete. 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.208+0000 7fcc67fdf740 1 -- start start 2024-05-27T10:21:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.208+0000 7fcc67fdf740 1 Processor -- start 2024-05-27T10:21:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63981440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a63981980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635d11b0 0x557a639f0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c00cc20 con 0x557a635a82f0 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c00e090 con 0x557a635d11b0 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c00d2f0 con 0x557a635b04d0 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc67fdf740 10 client.0 _ll_register_callbacks cb 0x7fcc540a5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63981440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63981440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44956/0 (socket says 172.21.15.42:44956) 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.209+0000 7fcc59e00640 1 -- 172.21.15.42:0/262526701 learned_addr learned my addr 172.21.15.42:0/262526701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc5a800640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635d11b0 0x557a639f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59400640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a63981980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.211 INFO:tasks.workunit.client.0.smithi042.stdout:config.status: creating config.h 2024-05-27T10:21:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59e00640 1 -- 172.21.15.42:0/262526701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 msgr2=0x557a63981980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59e00640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a63981980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59e00640 1 -- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635d11b0 msgr2=0x557a639f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:21:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59e00640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635d11b0 0x557a639f0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59e00640 1 -- 172.21.15.42:0/262526701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a639f0bd0 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc5a800640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635d11b0 0x557a639f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc59e00640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63981440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c005860 tx=0x7fcc4c0274a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cf0081311a29ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.210+0000 7fcc3be00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.211+0000 7fcc3be00640 1 -- 172.21.15.42:0/262526701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c036060 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.211+0000 7fcc3be00640 1 -- 172.21.15.42:0/262526701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcc4c0289f0 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.211+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc4c028840 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.211+0000 7fcc3be00640 1 -- 172.21.15.42:0/262526701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc4c02d040 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.211+0000 7fcc67fdf740 10 client.16047 fetch_fsmap learned FSMap version 29 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.211+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a639f7eb0 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.212+0000 7fcc3be00640 1 -- 172.21.15.42:0/262526701 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc4c028840 con 0x557a635a82f0 2024-05-27T10:21:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc3be00640 1 -- 172.21.15.42:0/262526701 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fcc4c030070 con 0x557a635a82f0 2024-05-27T10:21:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc67fdf740 10 client.16047 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:21:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc67fdf740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:21:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc67fdf740 20 client.16047 populate_metadata read hostname 'smithi042' 2024-05-27T10:21:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a635651c0 0x557a639fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:21:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc67fdf740 4 client.16047 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:21:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.213+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a638e02c0 con 0x557a635651c0 2024-05-27T10:21:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.215+0000 7fcc59400640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a635651c0 0x557a639fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:21:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.217+0000 7fcc59400640 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a635651c0 0x557a639fa6a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ece528e9be48298 server_cookie=ded19d9aac4154da in_seq=0 out_seq=0 2024-05-27T10:21:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.217+0000 7fcc3be00640 10 client.16047 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc3be00640 1 -- 172.21.15.42:0/262526701 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557a638e02c0 con 0x557a635651c0 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc3be00640 10 client.16047 handle_command_reply: tid=0 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 client.16047 shutdown 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 msgr2=0x557a63981440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63981440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c005860 tx=0x7fcc4c0274a0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 shutdown_connections 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a635d11b0 0x557a639f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557a635651c0 0x557a639fa6a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a635b04d0 0x557a63981980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 --2- 172.21.15.42:0/262526701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557a635a82f0 0x557a63981440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 >> 172.21.15.42:0/262526701 conn(0x557a635c5300 msgr2=0x557a6397f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc3be00640 0 client.16047 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:21:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 shutdown_connections 2024-05-27T10:21:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.255+0000 7fcc67fdf740 1 -- 172.21.15.42:0/262526701 wait complete. 2024-05-27T10:21:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc67fdf740 20 client.16047 trim_cache size 0 max 16384 2024-05-27T10:21:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:21:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:21:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:21:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc400b29d0 msgr2=0x7fcc400b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc400b29d0 0x7fcc400b4ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x557a63974e10 tx=0x7fcc4c00b390 comp rx=0 tx=0).stop 2024-05-27T10:21:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 msgr2=0x7fcc5c0755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:21:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c0755c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400eaa0 tx=0x7fcc5400ef60 comp rx=0 tx=0).stop 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 shutdown_connections 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc400b29d0 0x7fcc400b4ee0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc5c109f50 0x7fcc5c0755c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.256+0000 7fcc5b200640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc5c109ab0 0x7fcc5c074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.257+0000 7fcc5b200640 1 --2- 172.21.15.42:0/3731517605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc5c104250 0x7fcc5c076b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.257+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 >> 172.21.15.42:0/3731517605 conn(0x7fcc5c079240 msgr2=0x7fcc5c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.257+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 shutdown_connections 2024-05-27T10:21:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:21:59.257+0000 7fcc5b200640 1 -- 172.21.15.42:0/3731517605 wait complete. 2024-05-27T10:21:59.508 INFO:tasks.workunit.client.0.smithi042.stdout:config.status: executing depfiles commands 2024-05-27T10:22:00.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-27T10:22:00.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-27T10:22:00.250 INFO:tasks.workunit.client.0.smithi042.stderr:+ make 2024-05-27T10:22:00.311 INFO:tasks.workunit.client.0.smithi042.stdout:make all-recursive 2024-05-27T10:22:00.336 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-27T10:22:00.350 INFO:tasks.workunit.client.0.smithi042.stdout:Making all in src 2024-05-27T10:22:00.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:00 smithi104 ceph-mon[34812]: pgmap v101: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 232 KiB/s wr, 112 op/s 2024-05-27T10:22:00.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:00 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1292337799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:00.461 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-05-27T10:22:00.461 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-05-27T10:22:00.461 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-05-27T10:22:00.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:00 smithi143 ceph-mon[35545]: pgmap v101: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 232 KiB/s wr, 112 op/s 2024-05-27T10:22:00.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:00 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1292337799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:00.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:00 smithi042 ceph-mon[31661]: pgmap v101: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 232 KiB/s wr, 112 op/s 2024-05-27T10:22:00.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:00 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1292337799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:00.707 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:00.795 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-05-27T10:22:00.795 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-05-27T10:22:01.520 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-05-27T10:22:01.520 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-05-27T10:22:02.310 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-05-27T10:22:02.311 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-05-27T10:22:02.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:02 smithi104 ceph-mon[34812]: pgmap v102: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 161 KiB/s wr, 80 op/s 2024-05-27T10:22:02.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:02 smithi143 ceph-mon[35545]: pgmap v102: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 161 KiB/s wr, 80 op/s 2024-05-27T10:22:02.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:02 smithi042 ceph-mon[31661]: pgmap v102: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 161 KiB/s wr, 80 op/s 2024-05-27T10:22:03.055 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-05-27T10:22:03.056 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-05-27T10:22:04.132 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-05-27T10:22:04.132 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-05-27T10:22:04.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:04 smithi104 ceph-mon[34812]: pgmap v103: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 161 KiB/s wr, 80 op/s 2024-05-27T10:22:04.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:04 smithi143 ceph-mon[35545]: pgmap v103: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 161 KiB/s wr, 80 op/s 2024-05-27T10:22:04.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:04 smithi042 ceph-mon[31661]: pgmap v103: 97 pgs: 97 active+clean; 5.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 161 KiB/s wr, 80 op/s 2024-05-27T10:22:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 -- 172.21.15.42:0/4227015578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c075af0 msgr2=0x7f6c0c075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 --2- 172.21.15.42:0/4227015578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c075af0 0x7f6c0c075f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009990 tx=0x7f6bfc009ca0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 -- 172.21.15.42:0/4227015578 shutdown_connections 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 --2- 172.21.15.42:0/4227015578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 --2- 172.21.15.42:0/4227015578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c075af0 0x7f6c0c075f00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 --2- 172.21.15.42:0/4227015578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c0c076440 0x7f6c0c1106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 -- 172.21.15.42:0/4227015578 >> 172.21.15.42:0/4227015578 conn(0x7f6c0c100030 msgr2=0x7f6c0c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 -- 172.21.15.42:0/4227015578 shutdown_connections 2024-05-27T10:22:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.643+0000 7f6c11400700 1 -- 172.21.15.42:0/4227015578 wait complete. 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.645+0000 7f6c11400700 1 Processor -- start 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.645+0000 7f6c11400700 1 -- start start 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.645+0000 7f6c11400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c0c075af0 0x7f6c0c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.645+0000 7f6c11400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c0c076440 0x7f6c0c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c11400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c11400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1165d0 con 0x7f6c0c110bf0 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c11400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c116450 con 0x7f6c0c076440 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c11400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c116750 con 0x7f6c0c075af0 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c0b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c0b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42746/0 (socket says 172.21.15.42:42746) 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c0b200700 1 -- 172.21.15.42:0/3063291068 learned_addr learned my addr 172.21.15.42:0/3063291068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c0a800700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c0c075af0 0x7f6c0c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.646+0000 7f6c09e00700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c0c076440 0x7f6c0c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c0b200700 1 -- 172.21.15.42:0/3063291068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c0c075af0 msgr2=0x7f6c0c19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c0b200700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c0c075af0 0x7f6c0c19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c0b200700 1 -- 172.21.15.42:0/3063291068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c0c076440 msgr2=0x7f6c0c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c0b200700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c0c076440 0x7f6c0c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c0b200700 1 -- 172.21.15.42:0/3063291068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c1a1e70 con 0x7f6c0c110bf0 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c0b200700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c1a17d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6c040094f0 tx=0x7f6c0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:04.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.647+0000 7f6c03400700 1 -- 172.21.15.42:0/3063291068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04004db0 con 0x7f6c0c110bf0 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.648+0000 7f6c03400700 1 -- 172.21.15.42:0/3063291068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c04009e20 con 0x7f6c0c110bf0 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.648+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c1a2160 con 0x7f6c0c110bf0 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.648+0000 7f6c03400700 1 -- 172.21.15.42:0/3063291068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04015400 con 0x7f6c0c110bf0 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.648+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c1a9a80 con 0x7f6c0c110bf0 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.652+0000 7f6c03400700 1 -- 172.21.15.42:0/3063291068 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c04005260 con 0x7f6c0c110bf0 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.653+0000 7f6c20028b80 1 Processor -- start 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 -- start start 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c03400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6bf00b29f0 0x7f6bf00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c03400700 1 -- 172.21.15.42:0/3063291068 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6c040e1fd0 con 0x7f6c0c110bf0 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f624e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f61ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 0x55d11f61f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c09e00700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6bf00b29f0 0x7f6bf00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d11f54cfd0 con 0x55d11f61f4d0 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d11f53cb70 con 0x55d11f625380 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.654+0000 7f6c20028b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d11f540bb0 con 0x55d11f49ffd0 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 0x55d11f61f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 0x55d11f61f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42762/0 (socket says 172.21.15.42:42762) 2024-05-27T10:22:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 -- 172.21.15.42:0/765628071 learned_addr learned my addr 172.21.15.42:0/765628071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0a800700 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f624e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c09e00700 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f61ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c09e00700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6bf00b29f0 0x7f6bf00b4f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c1034a0 tx=0x7f6bf80106b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 -- 172.21.15.42:0/765628071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 msgr2=0x55d11f624e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f624e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 -- 172.21.15.42:0/765628071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 msgr2=0x55d11f61ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f61ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c0b200700 1 -- 172.21.15.42:0/765628071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11f5055b0 con 0x55d11f61f4d0 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.655+0000 7f6c09e00700 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f61ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.656+0000 7f6c0b200700 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 0x55d11f61f8e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6c040055b0 tx=0x7f6c04005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76aa733d863f8a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.656+0000 7f6bf7e00700 1 -- 172.21.15.42:0/765628071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040dfe60 con 0x55d11f61f4d0 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.656+0000 7f6bf7e00700 1 -- 172.21.15.42:0/765628071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c040a92a0 con 0x55d11f61f4d0 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.656+0000 7f6bf7e00700 1 -- 172.21.15.42:0/765628071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040b1ed0 con 0x55d11f61f4d0 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 -- 172.21.15.42:0/765628071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 msgr2=0x55d11f61f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 0x55d11f61f8e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6c040055b0 tx=0x7f6c04005b90 comp rx=0 tx=0).stop 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 -- 172.21.15.42:0/765628071 shutdown_connections 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f61f4d0 0x55d11f61f8e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f624e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 --2- 172.21.15.42:0/765628071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f61ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 -- 172.21.15.42:0/765628071 >> 172.21.15.42:0/765628071 conn(0x55d11f500150 msgr2=0x55d11f61a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 -- 172.21.15.42:0/765628071 shutdown_connections 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.657+0000 7f6c20028b80 1 -- 172.21.15.42:0/765628071 wait complete. 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.658+0000 7f6c20028b80 1 -- start start 2024-05-27T10:22:04.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.658+0000 7f6c20028b80 1 Processor -- start 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f628b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f629060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d11f443130 con 0x55d11f49ffd0 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d11f4dd8f0 con 0x55d11f625380 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d11f62be50 con 0x55d11f61f4d0 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c20028b80 10 client.0 _ll_register_callbacks cb 0x55d11f4e6e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:04.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.659+0000 7f6c0a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f628b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f628b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42776/0 (socket says 172.21.15.42:42776) 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 -- 172.21.15.42:0/3631485625 learned_addr learned my addr 172.21.15.42:0/3631485625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c09e00700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f629060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0b200700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 -- 172.21.15.42:0/3631485625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 msgr2=0x55d11f629060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f629060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 -- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 msgr2=0x55d11f6295a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6295a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 -- 172.21.15.42:0/3631485625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d11f48ca60 con 0x55d11f49ffd0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6c0a800700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f628b20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc015000 tx=0x7f6bfc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1916244f14fb4738 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.660+0000 7f6bf4c00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.661+0000 7f6bf4c00700 1 -- 172.21.15.42:0/3631485625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc003bd0 con 0x55d11f49ffd0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.661+0000 7f6bf4c00700 1 -- 172.21.15.42:0/3631485625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6bfc003d70 con 0x55d11f49ffd0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.661+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c040a92a0 con 0x55d11f49ffd0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.661+0000 7f6bf4c00700 1 -- 172.21.15.42:0/3631485625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6bfc031050 con 0x55d11f49ffd0 2024-05-27T10:22:04.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.661+0000 7f6c20028b80 10 client.16059 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.661+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d11f50d1f0 con 0x55d11f49ffd0 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6bf4c00700 1 -- 172.21.15.42:0/3631485625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c040a92a0 con 0x55d11f49ffd0 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6bf4c00700 1 -- 172.21.15.42:0/3631485625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6bfc0237e0 con 0x55d11f49ffd0 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6c20028b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6c20028b80 10 client.16059 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6c20028b80 20 client.16059 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6c20028b80 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f638e00 0x55d11f63b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6c20028b80 4 client.16059 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:04.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.662+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d11f54ef20 con 0x55d11f638e00 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.663+0000 7f6c09e00700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f638e00 0x55d11f63b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.664+0000 7f6c09e00700 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f638e00 0x55d11f63b2a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cfe1901684a239 server_cookie=8b530f67c6e6c4df in_seq=0 out_seq=0 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.664+0000 7f6bf4c00700 10 client.16059 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.666+0000 7f6bf4c00700 1 -- 172.21.15.42:0/3631485625 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d11f54ef20 con 0x55d11f638e00 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.666+0000 7f6bf4c00700 10 client.16059 handle_command_reply: tid=0 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.666+0000 7f6c20028b80 1 client.16059 shutdown 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 msgr2=0x55d11f628b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f628b20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc015000 tx=0x7f6bfc009f90 comp rx=0 tx=0).stop 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 shutdown_connections 2024-05-27T10:22:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f638e00 0x55d11f63b2a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6295a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f628b20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 --2- 172.21.15.42:0/3631485625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f629060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6bf4c00700 0 client.16059 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 >> 172.21.15.42:0/3631485625 conn(0x55d11f500150 msgr2=0x55d11f61dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 shutdown_connections 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 1 -- 172.21.15.42:0/3631485625 wait complete. 2024-05-27T10:22:04.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.667+0000 7f6c20028b80 20 client.16059 trim_cache size 0 max 16384 2024-05-27T10:22:04.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.691+0000 7f6c20028b80 1 Processor -- start 2024-05-27T10:22:04.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.691+0000 7f6c20028b80 1 -- start start 2024-05-27T10:22:04.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f62ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f61f4d0 0x55d11f634eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 0x55d11f61e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c20028b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1b37c0 con 0x55d11f625380 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c20028b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1b1050 con 0x55d11f61f4d0 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c20028b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1aeec0 con 0x55d11f49ffd0 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c0b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 0x55d11f61e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c0b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 0x55d11f61e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42790/0 (socket says 172.21.15.42:42790) 2024-05-27T10:22:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c0b200700 1 -- 172.21.15.42:0/2986455646 learned_addr learned my addr 172.21.15.42:0/2986455646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:04.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c0a800700 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f62ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.692+0000 7f6c09e00700 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f61f4d0 0x55d11f634eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6c0b200700 1 -- 172.21.15.42:0/2986455646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 msgr2=0x55d11f62ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6c0b200700 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f62ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6c0b200700 1 -- 172.21.15.42:0/2986455646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f61f4d0 msgr2=0x55d11f634eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6c0b200700 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f61f4d0 0x55d11f634eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6c0b200700 1 -- 172.21.15.42:0/2986455646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11f62b3c0 con 0x55d11f625380 2024-05-27T10:22:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6c0b200700 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 0x55d11f61e4b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6c040dfc90 tx=0x7f6c040a91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fa751e1057f0a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.693+0000 7f6bf7400700 1 -- 172.21.15.42:0/2986455646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040dfe60 con 0x55d11f625380 2024-05-27T10:22:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.694+0000 7f6bf7400700 1 -- 172.21.15.42:0/2986455646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c040ad650 con 0x55d11f625380 2024-05-27T10:22:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.694+0000 7f6bf7400700 1 -- 172.21.15.42:0/2986455646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040ad380 con 0x55d11f625380 2024-05-27T10:22:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 -- 172.21.15.42:0/2986455646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 msgr2=0x55d11f61e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 0x55d11f61e4b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6c040dfc90 tx=0x7f6c040a91a0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 -- 172.21.15.42:0/2986455646 shutdown_connections 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f625380 0x55d11f61e4b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f49ffd0 0x55d11f62ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 --2- 172.21.15.42:0/2986455646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f61f4d0 0x55d11f634eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 -- 172.21.15.42:0/2986455646 >> 172.21.15.42:0/2986455646 conn(0x55d11f500150 msgr2=0x55d11f61dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 -- 172.21.15.42:0/2986455646 shutdown_connections 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.695+0000 7f6c20028b80 1 -- 172.21.15.42:0/2986455646 wait complete. 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.696+0000 7f6c20028b80 1 -- start start 2024-05-27T10:22:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.696+0000 7f6c20028b80 1 Processor -- start 2024-05-27T10:22:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.696+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f62e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.696+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f62e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c20028b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c20028b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1af510 con 0x55d11f49ffd0 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c20028b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1b0980 con 0x55d11f625380 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c20028b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1afbe0 con 0x55d11f61f4d0 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c20028b80 10 client.0 _ll_register_callbacks cb 0x55d11f4e6e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f62e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f62e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42794/0 (socket says 172.21.15.42:42794) 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 -- 172.21.15.42:0/953535441 learned_addr learned my addr 172.21.15.42:0/953535441 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c09e00700 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f62e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 -- 172.21.15.42:0/953535441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 msgr2=0x55d11f62e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f62e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 -- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 msgr2=0x55d11f6a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.697+0000 7f6c0a800700 1 -- 172.21.15.42:0/953535441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d11f62ef80 con 0x55d11f49ffd0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6c0a800700 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f62e190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009990 tx=0x7f6bfc00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa8aac9a42a3d95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6bf4c00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6bf4c00700 1 -- 172.21.15.42:0/953535441 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc039070 con 0x55d11f49ffd0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d11f548700 con 0x55d11f49ffd0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6bf4c00700 1 -- 172.21.15.42:0/953535441 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6bfc023470 con 0x55d11f49ffd0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6bf4c00700 1 -- 172.21.15.42:0/953535441 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6bfc045050 con 0x55d11f49ffd0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6c20028b80 10 client.16065 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.698+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d11f6a4ea0 con 0x55d11f49ffd0 2024-05-27T10:22:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6bf4c00700 1 -- 172.21.15.42:0/953535441 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d11f548700 con 0x55d11f49ffd0 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6bf4c00700 1 -- 172.21.15.42:0/953535441 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6bfc00f600 con 0x55d11f49ffd0 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6c20028b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6c20028b80 10 client.16065 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6c20028b80 20 client.16065 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6c20028b80 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f6a5680 0x55d11f6a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6c20028b80 4 client.16065 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-27T10:22:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.699+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55d11f3bf3e0 con 0x55d11f6a5680 2024-05-27T10:22:04.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.700+0000 7f6c09e00700 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f6a5680 0x55d11f6a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:04.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.701+0000 7f6c09e00700 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f6a5680 0x55d11f6a7aa0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7197e5479079186 server_cookie=b6872e226fb1c5e4 in_seq=0 out_seq=0 2024-05-27T10:22:04.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.701+0000 7f6bf4c00700 10 client.16065 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:04.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.934+0000 7f6bf4c00700 1 -- 172.21.15.42:0/953535441 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d11f3bf3e0 con 0x55d11f6a5680 2024-05-27T10:22:04.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.934+0000 7f6bf4c00700 10 client.16065 handle_command_reply: tid=0 2024-05-27T10:22:04.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.934+0000 7f6c20028b80 1 client.16065 shutdown 2024-05-27T10:22:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 msgr2=0x55d11f62e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f62e190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009990 tx=0x7f6bfc00f460 comp rx=0 tx=0).stop 2024-05-27T10:22:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 shutdown_connections 2024-05-27T10:22:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d11f6a5680 0x55d11f6a7aa0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d11f625380 0x55d11f6a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d11f49ffd0 0x55d11f62e190 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 --2- 172.21.15.42:0/953535441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d11f61f4d0 0x55d11f62e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.935+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 >> 172.21.15.42:0/953535441 conn(0x55d11f500150 msgr2=0x55d11f628900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.936+0000 7f6bf4c00700 0 client.16065 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.936+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 shutdown_connections 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.936+0000 7f6c20028b80 1 -- 172.21.15.42:0/953535441 wait complete. 2024-05-27T10:22:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.937+0000 7f6c20028b80 20 client.16065 trim_cache size 0 max 16384 2024-05-27T10:22:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:22:04.937 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 29, 2024-05-27T10:22:04.937 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:22:04.937 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 6, 2024-05-27T10:22:04.937 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:22:04.937 INFO:teuthology.orchestra.run.smithi042.stdout: "fae09024": { 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 6, 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 130.0, 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:22:04.938 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:22:04.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:04.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:04.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:04.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:04.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:04.939 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.938+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6bf00b29f0 msgr2=0x7f6bf00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.938+0000 7f6c11400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6bf00b29f0 0x7f6bf00b4f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c1034a0 tx=0x7f6bf80106b0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.938+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 msgr2=0x7f6c0c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.938+0000 7f6c11400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c1a17d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6c040094f0 tx=0x7f6c0400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 shutdown_connections 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6bf00b29f0 0x7f6bf00b4f00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c0c110bf0 0x7f6c0c1a17d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c0c075af0 0x7f6c0c19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 --2- 172.21.15.42:0/3063291068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c0c076440 0x7f6c0c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 >> 172.21.15.42:0/3063291068 conn(0x7f6c0c100030 msgr2=0x7f6c0c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 shutdown_connections 2024-05-27T10:22:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:04.939+0000 7f6c11400700 1 -- 172.21.15.42:0/3063291068 wait complete. 2024-05-27T10:22:06.111 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fae09024', quiesce took 11.7 seconds. Will release after: -1.6999999999999993 2024-05-27T10:22:06.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:22:06.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:06 smithi042 ceph-mon[31661]: pgmap v104: 97 pgs: 97 active+clean; 5.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 224 KiB/s wr, 113 op/s 2024-05-27T10:22:06.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:06 smithi104 ceph-mon[34812]: pgmap v104: 97 pgs: 97 active+clean; 5.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 224 KiB/s wr, 113 op/s 2024-05-27T10:22:06.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:06 smithi143 ceph-mon[35545]: pgmap v104: 97 pgs: 97 active+clean; 5.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 224 KiB/s wr, 113 op/s 2024-05-27T10:22:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:08.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:08 smithi104 ceph-mon[34812]: pgmap v105: 97 pgs: 97 active+clean; 5.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 151 KiB/s wr, 71 op/s 2024-05-27T10:22:08.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:08 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:22:08.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:08 smithi143 ceph-mon[35545]: pgmap v105: 97 pgs: 97 active+clean; 5.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 151 KiB/s wr, 71 op/s 2024-05-27T10:22:08.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:08 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:22:08.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:08 smithi042 ceph-mon[31661]: pgmap v105: 97 pgs: 97 active+clean; 5.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 151 KiB/s wr, 71 op/s 2024-05-27T10:22:08.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:08 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:22:09.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 -- 172.21.15.42:0/3800773578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0341043f0 msgr2=0x7fd034104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:09.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 --2- 172.21.15.42:0/3800773578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0341043f0 0x7fd034104800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd0240099c0 tx=0x7fd024009cd0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 -- 172.21.15.42:0/3800773578 shutdown_connections 2024-05-27T10:22:09.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 --2- 172.21.15.42:0/3800773578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 0x7fd034110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 --2- 172.21.15.42:0/3800773578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0341043f0 0x7fd034104800 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 --2- 172.21.15.42:0/3800773578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd034108d20 0x7fd034109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 -- 172.21.15.42:0/3800773578 >> 172.21.15.42:0/3800773578 conn(0x7fd034079d00 msgr2=0x7fd03407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:09.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 -- 172.21.15.42:0/3800773578 shutdown_connections 2024-05-27T10:22:09.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.247+0000 7fd039a00700 1 -- 172.21.15.42:0/3800773578 wait complete. 2024-05-27T10:22:09.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 Processor -- start 2024-05-27T10:22:09.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 -- start start 2024-05-27T10:22:09.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0341043f0 0x7fd034075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:09.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 0x7fd034075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:09.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 0x7fd034076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:09.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341141d0 con 0x7fd034108d20 2024-05-27T10:22:09.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd034114050 con 0x7fd034109670 2024-05-27T10:22:09.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.249+0000 7fd039a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd034114350 con 0x7fd0341043f0 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.250+0000 7fd033400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 0x7fd034075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.250+0000 7fd033400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 0x7fd034075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42828/0 (socket says 172.21.15.42:42828) 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.250+0000 7fd033400700 1 -- 172.21.15.42:0/2088724129 learned_addr learned my addr 172.21.15.42:0/2088724129 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.250+0000 7fd033e00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0341043f0 0x7fd034075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.250+0000 7fd039000700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 0x7fd034076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd033400700 1 -- 172.21.15.42:0/2088724129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0341043f0 msgr2=0x7fd034075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:09.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd033400700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0341043f0 0x7fd034075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd033400700 1 -- 172.21.15.42:0/2088724129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 msgr2=0x7fd034076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:09.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd033400700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 0x7fd034076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd033400700 1 -- 172.21.15.42:0/2088724129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0341a8f60 con 0x7fd034108d20 2024-05-27T10:22:09.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd039000700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 0x7fd034076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:09.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd033400700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 0x7fd034075ff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd02000d980 tx=0x7fd02000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020017070 con 0x7fd034108d20 2024-05-27T10:22:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd020009dc0 con 0x7fd034108d20 2024-05-27T10:22:09.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02000f3c0 con 0x7fd034108d20 2024-05-27T10:22:09.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.251+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0341a9250 con 0x7fd034108d20 2024-05-27T10:22:09.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.252+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a9790 con 0x7fd034108d20 2024-05-27T10:22:09.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.253+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0340512b0 con 0x7fd034108d20 2024-05-27T10:22:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.256+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd020009930 con 0x7fd034108d20 2024-05-27T10:22:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.258+0000 7fd030c00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0180b29f0 0x7fd0180b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.258+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fd0200e2110 con 0x7fd034108d20 2024-05-27T10:22:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.258+0000 7fd033e00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0180b29f0 0x7fd0180b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.259+0000 7fd033e00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0180b29f0 0x7fd0180b4f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd024000c00 tx=0x7fd024005fb0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:09.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.261+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0200a9ae0 con 0x7fd034108d20 2024-05-27T10:22:09.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:09 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:22:09.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:09 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:22:09.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:09 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:22:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.554+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd03402e5e0 con 0x7fd034108d20 2024-05-27T10:22:09.553 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:09 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:22:09.553 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:09 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:22:09.554 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:09 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:22:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.555+0000 7fd030c00700 1 -- 172.21.15.42:0/2088724129 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11881 (secure 0 0 0) 0x7fd0200a93e0 con 0x7fd034108d20 2024-05-27T10:22:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:22:09.555 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:21:52.648444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:22:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:09 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:22:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:09 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:22:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:09 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.558+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0180b29f0 msgr2=0x7fd0180b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.558+0000 7fd039a00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0180b29f0 0x7fd0180b4f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd024000c00 tx=0x7fd024005fb0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.558+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 msgr2=0x7fd034075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.558+0000 7fd039a00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 0x7fd034075ff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd02000d980 tx=0x7fd02000dc90 comp rx=0 tx=0).stop 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.558+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 shutdown_connections 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.558+0000 7fd039a00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0180b29f0 0x7fd0180b4f00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.559+0000 7fd039a00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd034109670 0x7fd034076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.559+0000 7fd039a00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0341043f0 0x7fd034075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.559+0000 7fd039a00700 1 --2- 172.21.15.42:0/2088724129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd034108d20 0x7fd034075ff0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.559+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 >> 172.21.15.42:0/2088724129 conn(0x7fd034079d00 msgr2=0x7fd034101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.559+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 shutdown_connections 2024-05-27T10:22:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:09.559+0000 7fd039a00700 1 -- 172.21.15.42:0/2088724129 wait complete. 2024-05-27T10:22:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-05-27T10:22:10.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:10 smithi042 ceph-mon[31661]: pgmap v106: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 166 KiB/s wr, 77 op/s 2024-05-27T10:22:10.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:10 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2088724129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:10.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:10 smithi104 ceph-mon[34812]: pgmap v106: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 166 KiB/s wr, 77 op/s 2024-05-27T10:22:10.449 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:10 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2088724129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:10.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:10 smithi143 ceph-mon[35545]: pgmap v106: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 166 KiB/s wr, 77 op/s 2024-05-27T10:22:10.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:10 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2088724129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:10.564 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id fae09024 --release --await' 2024-05-27T10:22:10.564 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id fae09024 --release --await 2024-05-27T10:22:11.036 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 -- 172.21.15.42:0/476971130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810c170 msgr2=0x7f4f68105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 --2- 172.21.15.42:0/476971130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810c170 0x7f4f68105db0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009900 tx=0x7f4f5402a3f0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 -- 172.21.15.42:0/476971130 shutdown_connections 2024-05-27T10:22:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 --2- 172.21.15.42:0/476971130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f68106420 0x7f4f681127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 --2- 172.21.15.42:0/476971130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810c170 0x7f4f68105db0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 --2- 172.21.15.42:0/476971130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f6810bcd0 0x7f4f68105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.486+0000 7f4f67e00640 1 -- 172.21.15.42:0/476971130 >> 172.21.15.42:0/476971130 conn(0x7f4f681013d0 msgr2=0x7f4f681037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.487+0000 7f4f67e00640 1 -- 172.21.15.42:0/476971130 shutdown_connections 2024-05-27T10:22:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.487+0000 7f4f67e00640 1 -- 172.21.15.42:0/476971130 wait complete. 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 Processor -- start 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 -- start start 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f68106420 0x7f4f681a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 0x7f4f681a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f6810c170 0x7f4f6819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68115da0 con 0x7f4f6810bcd0 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68115c20 con 0x7f4f68106420 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.488+0000 7f4f67e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68115f20 con 0x7f4f6810c170 2024-05-27T10:22:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 0x7f4f681a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 0x7f4f681a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42850/0 (socket says 172.21.15.42:42850) 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 -- 172.21.15.42:0/707566968 learned_addr learned my addr 172.21.15.42:0/707566968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66a00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f68106420 0x7f4f681a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f67400640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f6810c170 0x7f4f6819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 -- 172.21.15.42:0/707566968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f6810c170 msgr2=0x7f4f6819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f6810c170 0x7f4f6819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 -- 172.21.15.42:0/707566968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f68106420 msgr2=0x7f4f681a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f68106420 0x7f4f681a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.489+0000 7f4f66000640 1 -- 172.21.15.42:0/707566968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6819cd90 con 0x7f4f6810bcd0 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.490+0000 7f4f66000640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 0x7f4f681a2a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4f5402a910 tx=0x7f4f54003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.490+0000 7f4f5f400640 1 -- 172.21.15.42:0/707566968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5403f070 con 0x7f4f6810bcd0 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.490+0000 7f4f5f400640 1 -- 172.21.15.42:0/707566968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f54003e10 con 0x7f4f6810bcd0 2024-05-27T10:22:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.490+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6819cfc0 con 0x7f4f6810bcd0 2024-05-27T10:22:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.490+0000 7f4f5f400640 1 -- 172.21.15.42:0/707566968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54035030 con 0x7f4f6810bcd0 2024-05-27T10:22:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.490+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681a9280 con 0x7f4f6810bcd0 2024-05-27T10:22:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 Processor -- start 2024-05-27T10:22:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 -- start start 2024-05-27T10:22:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3563a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31982f0 0x55c2b3563fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31a04d0 0x55c2b35644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c2b34d4b90 con 0x55c2b31c11b0 2024-05-27T10:22:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c2b34c4730 con 0x55c2b31a04d0 2024-05-27T10:22:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.494+0000 7f4f745b1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c2b34c8770 con 0x55c2b31982f0 2024-05-27T10:22:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3563a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f5f400640 1 -- 172.21.15.42:0/707566968 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4f54042070 con 0x7f4f6810bcd0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3563a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42862/0 (socket says 172.21.15.42:42862) 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 -- 172.21.15.42:0/4034390824 learned_addr learned my addr 172.21.15.42:0/4034390824 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f66a00640 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31982f0 0x55c2b3563fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 -- 172.21.15.42:0/4034390824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31982f0 msgr2=0x55c2b3563fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31982f0 0x55c2b3563fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 -- 172.21.15.42:0/4034390824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31a04d0 msgr2=0x55c2b35644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31a04d0 0x55c2b35644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.495+0000 7f4f67400640 1 -- 172.21.15.42:0/4034390824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b345d8b0 con 0x55c2b31c11b0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.496+0000 7f4f67400640 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3563a60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009ee0 tx=0x7f4f60010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f8a0fa8a417ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.496+0000 7f4f53400640 1 -- 172.21.15.42:0/4034390824 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60016070 con 0x55c2b31c11b0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.496+0000 7f4f53400640 1 -- 172.21.15.42:0/4034390824 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f60004d20 con 0x55c2b31c11b0 2024-05-27T10:22:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.496+0000 7f4f53400640 1 -- 172.21.15.42:0/4034390824 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f600126b0 con 0x55c2b31c11b0 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.497+0000 7f4f5f400640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b29d0 0x7f4f4c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.497+0000 7f4f5f400640 1 -- 172.21.15.42:0/707566968 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4f54102dd0 con 0x7f4f6810bcd0 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.497+0000 7f4f66a00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b29d0 0x7f4f4c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.497+0000 7f4f745b1740 1 -- 172.21.15.42:0/4034390824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 msgr2=0x55c2b3563a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.497+0000 7f4f745b1740 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3563a60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009ee0 tx=0x7f4f60010740 comp rx=0 tx=0).stop 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 -- 172.21.15.42:0/4034390824 shutdown_connections 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31a04d0 0x55c2b35644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31982f0 0x55c2b3563fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 --2- 172.21.15.42:0/4034390824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3563a60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 -- 172.21.15.42:0/4034390824 >> 172.21.15.42:0/4034390824 conn(0x55c2b31b5300 msgr2=0x55c2b3560210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 -- 172.21.15.42:0/4034390824 shutdown_connections 2024-05-27T10:22:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f745b1740 1 -- 172.21.15.42:0/4034390824 wait complete. 2024-05-27T10:22:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.498+0000 7f4f66a00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b29d0 0x7f4f4c0b4ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55c2b3564e10 tx=0x7f4f5800b4d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 -- start start 2024-05-27T10:22:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 Processor -- start 2024-05-27T10:22:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b356dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b356e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3579f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c2b31aa790 con 0x55c2b31c11b0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c2b346cbc0 con 0x55c2b31982f0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.499+0000 7f4f745b1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c2b3570b90 con 0x55c2b31a04d0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f745b1740 10 client.0 _ll_register_callbacks cb 0x55c2b344a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3579f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3579f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42864/0 (socket says 172.21.15.42:42864) 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 -- 172.21.15.42:0/2448063363 learned_addr learned my addr 172.21.15.42:0/2448063363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f66a00640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b356dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f66000640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b356e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 -- 172.21.15.42:0/2448063363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 msgr2=0x55c2b356e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b356e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 -- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 msgr2=0x55c2b356dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b356dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.500+0000 7f4f67400640 1 -- 172.21.15.42:0/2448063363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b31d9210 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f67400640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3579f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4f60005a20 tx=0x7f4f60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deed93ff036bb9de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f47e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f47e00640 1 -- 172.21.15.42:0/2448063363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60028060 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f47e00640 1 -- 172.21.15.42:0/2448063363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4f600050e0 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f60004d20 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f47e00640 1 -- 172.21.15.42:0/2448063363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f60016070 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f745b1740 10 client.16083 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.501+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2b32a5620 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f47e00640 1 -- 172.21.15.42:0/2448063363 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f60004d20 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f47e00640 1 -- 172.21.15.42:0/2448063363 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4f6000f040 con 0x55c2b31c11b0 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f745b1740 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f745b1740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f745b1740 20 client.16083 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f745b1740 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b317fd80 0x55c2b357fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f745b1740 4 client.16083 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.502+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2b34d6ae0 con 0x55c2b317fd80 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.503+0000 7f4f66a00640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b317fd80 0x55c2b357fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.504+0000 7f4f66a00640 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b317fd80 0x55c2b357fe20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a179b53bf4f4f1c server_cookie=308caff606b6f79 in_seq=0 out_seq=0 2024-05-27T10:22:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.504+0000 7f4f47e00640 10 client.16083 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f47e00640 1 -- 172.21.15.42:0/2448063363 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c2b34d6ae0 con 0x55c2b317fd80 2024-05-27T10:22:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f47e00640 10 client.16083 handle_command_reply: tid=0 2024-05-27T10:22:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 client.16083 shutdown 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 msgr2=0x55c2b3579f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3579f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4f60005a20 tx=0x7f4f60004a40 comp rx=0 tx=0).stop 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 shutdown_connections 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31c11b0 0x55c2b3579f20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b317fd80 0x55c2b357fe20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b356e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 --2- 172.21.15.42:0/2448063363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b356dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f47e00640 0 client.16083 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 >> 172.21.15.42:0/2448063363 conn(0x55c2b31b5300 msgr2=0x55c2b3565780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 shutdown_connections 2024-05-27T10:22:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.506+0000 7f4f745b1740 1 -- 172.21.15.42:0/2448063363 wait complete. 2024-05-27T10:22:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.507+0000 7f4f745b1740 20 client.16083 trim_cache size 0 max 16384 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.523+0000 7f4f745b1740 1 Processor -- start 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.523+0000 7f4f745b1740 1 -- start start 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b3563a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 0x55c2b3579e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31c11b0 0x55c2b35677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f745b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681b4320 con 0x55c2b31a04d0 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f745b1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681b1bf0 con 0x55c2b31982f0 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f745b1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681afa40 con 0x55c2b31c11b0 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 0x55c2b3579e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 0x55c2b3579e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42874/0 (socket says 172.21.15.42:42874) 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 -- 172.21.15.42:0/1331095516 learned_addr learned my addr 172.21.15.42:0/1331095516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66a00640 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b3563a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f67400640 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31c11b0 0x55c2b35677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 -- 172.21.15.42:0/1331095516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31c11b0 msgr2=0x55c2b35677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31c11b0 0x55c2b35677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 -- 172.21.15.42:0/1331095516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 msgr2=0x55c2b3563a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b3563a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.524+0000 7f4f66000640 1 -- 172.21.15.42:0/1331095516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b3570a70 con 0x55c2b31a04d0 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.525+0000 7f4f67400640 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31c11b0 0x55c2b35677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.525+0000 7f4f66000640 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 0x55c2b3579e90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4f54100620 tx=0x7f4f540c9870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d96fc5930c8d981 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.525+0000 7f4f52a00640 1 -- 172.21.15.42:0/1331095516 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f540e4070 con 0x55c2b31a04d0 2024-05-27T10:22:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.525+0000 7f4f52a00640 1 -- 172.21.15.42:0/1331095516 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f540ca100 con 0x55c2b31a04d0 2024-05-27T10:22:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.525+0000 7f4f52a00640 1 -- 172.21.15.42:0/1331095516 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54049040 con 0x55c2b31a04d0 2024-05-27T10:22:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 -- 172.21.15.42:0/1331095516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 msgr2=0x55c2b3579e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 0x55c2b3579e90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4f54100620 tx=0x7f4f540c9870 comp rx=0 tx=0).stop 2024-05-27T10:22:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 -- 172.21.15.42:0/1331095516 shutdown_connections 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31c11b0 0x55c2b35677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31a04d0 0x55c2b3579e90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1331095516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31982f0 0x55c2b3563a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 -- 172.21.15.42:0/1331095516 >> 172.21.15.42:0/1331095516 conn(0x55c2b31b5300 msgr2=0x55c2b356fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 -- 172.21.15.42:0/1331095516 shutdown_connections 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.526+0000 7f4f745b1740 1 -- 172.21.15.42:0/1331095516 wait complete. 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 -- start start 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 Processor -- start 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 0x55c2b35df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b35dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31c11b0 0x55c2b3580410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681b00b0 con 0x55c2b31982f0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681b1520 con 0x55c2b31c11b0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681b0780 con 0x55c2b31a04d0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.527+0000 7f4f745b1740 10 client.0 _ll_register_callbacks cb 0x7f4f5800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 0x55c2b35df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 0x55c2b35df5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42890/0 (socket says 172.21.15.42:42890) 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 -- 172.21.15.42:0/1564083990 learned_addr learned my addr 172.21.15.42:0/1564083990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f67400640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31c11b0 0x55c2b3580410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66000640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b35dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 -- 172.21.15.42:0/1564083990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 msgr2=0x55c2b35dfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b35dfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 -- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31c11b0 msgr2=0x55c2b3580410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31c11b0 0x55c2b3580410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 -- 172.21.15.42:0/1564083990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b3580b00 con 0x55c2b31982f0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f67400640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31c11b0 0x55c2b3580410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f66a00640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 0x55c2b35df5f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4f58012fd0 tx=0x7f4f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e9ba0aa3c3d2c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.528+0000 7f4f47e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f47e00640 1 -- 172.21.15.42:0/1564083990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58029500 con 0x55c2b31982f0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f47e00640 1 -- 172.21.15.42:0/1564083990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f4f580296a0 con 0x55c2b31982f0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f540ca100 con 0x55c2b31982f0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f47e00640 1 -- 172.21.15.42:0/1564083990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f5800bd60 con 0x55c2b31982f0 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f745b1740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2b35e7c20 con 0x55c2b31982f0 2024-05-27T10:22:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.529+0000 7f4f47e00640 1 -- 172.21.15.42:0/1564083990 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f540ca100 con 0x55c2b31982f0 2024-05-27T10:22:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f47e00640 1 -- 172.21.15.42:0/1564083990 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4f5803cb80 con 0x55c2b31982f0 2024-05-27T10:22:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f745b1740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f745b1740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f745b1740 20 client.16089 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b31551c0 0x55c2b35ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f745b1740 4 client.16089 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2b34d02c0 con 0x55c2b31551c0 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.530+0000 7f4f66000640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b31551c0 0x55c2b35ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.531+0000 7f4f66000640 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b31551c0 0x55c2b35ea3c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a5fabb0549ee07 server_cookie=29c72f7d00e15f49 in_seq=0 out_seq=0 2024-05-27T10:22:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.531+0000 7f4f47e00640 10 client.16089 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f47e00640 1 -- 172.21.15.42:0/1564083990 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c2b34d02c0 con 0x55c2b31551c0 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f47e00640 10 client.16089 handle_command_reply: tid=0 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 client.16089 shutdown 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 msgr2=0x55c2b35df5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 0x55c2b35df5f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4f58012fd0 tx=0x7f4f580046c0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 shutdown_connections 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2b31c11b0 0x55c2b3580410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2b31551c0 0x55c2b35ea3c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2b31a04d0 0x55c2b35dfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 --2- 172.21.15.42:0/1564083990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2b31982f0 0x55c2b35df5f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 >> 172.21.15.42:0/1564083990 conn(0x55c2b31b5300 msgr2=0x55c2b356d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f47e00640 0 client.16089 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.568+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 shutdown_connections 2024-05-27T10:22:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.569+0000 7f4f745b1740 1 -- 172.21.15.42:0/1564083990 wait complete. 2024-05-27T10:22:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.569+0000 7f4f745b1740 20 client.16089 trim_cache size 0 max 16384 2024-05-27T10:22:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:22:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:22:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:22:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b29d0 msgr2=0x7f4f4c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b29d0 0x7f4f4c0b4ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55c2b3564e10 tx=0x7f4f5800b4d0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 msgr2=0x7f4f681a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 0x7f4f681a2a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4f5402a910 tx=0x7f4f54003c20 comp rx=0 tx=0).stop 2024-05-27T10:22:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 shutdown_connections 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f4f4c0b29d0 0x7f4f4c0b4ee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f6810c170 0x7f4f6819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f6810bcd0 0x7f4f681a2a20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 --2- 172.21.15.42:0/707566968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f68106420 0x7f4f681a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 >> 172.21.15.42:0/707566968 conn(0x7f4f681013d0 msgr2=0x7f4f681017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 shutdown_connections 2024-05-27T10:22:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:12.570+0000 7f4f67e00640 1 -- 172.21.15.42:0/707566968 wait complete. 2024-05-27T10:22:13.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:12 smithi143 ceph-mon[35545]: pgmap v107: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-05-27T10:22:13.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:12 smithi042 ceph-mon[31661]: pgmap v107: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-05-27T10:22:13.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:12 smithi104 ceph-mon[34812]: pgmap v107: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-05-27T10:22:13.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 -- 172.21.15.42:0/2247690166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f309010b860 msgr2=0x7f30901131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 --2- 172.21.15.42:0/2247690166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f309010b860 0x7f30901131c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f308800b0d0 tx=0x7f308800b3e0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 -- 172.21.15.42:0/2247690166 shutdown_connections 2024-05-27T10:22:13.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 --2- 172.21.15.42:0/2247690166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f309010b860 0x7f30901131c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 --2- 172.21.15.42:0/2247690166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3090106630 0x7f3090106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 --2- 172.21.15.42:0/2247690166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f309010af10 0x7f309010b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 -- 172.21.15.42:0/2247690166 >> 172.21.15.42:0/2247690166 conn(0x7f3090102020 msgr2=0x7f3090104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:13.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 -- 172.21.15.42:0/2247690166 shutdown_connections 2024-05-27T10:22:13.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.814+0000 7f3096e00700 1 -- 172.21.15.42:0/2247690166 wait complete. 2024-05-27T10:22:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.815+0000 7f3096e00700 1 Processor -- start 2024-05-27T10:22:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.815+0000 7f3096e00700 1 -- start start 2024-05-27T10:22:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3096e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 0x7f30901a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3096e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f309010af10 0x7f30901a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3096e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309010b860 0x7f309019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3096e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3090116340 con 0x7f3090106630 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3096e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30901161c0 con 0x7f309010af10 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3096e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f30901164c0 con 0x7f309010b860 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 0x7f30901a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 0x7f30901a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35858/0 (socket says 172.21.15.42:35858) 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.816+0000 7f3095a00700 1 -- 172.21.15.42:0/1162660077 learned_addr learned my addr 172.21.15.42:0/1162660077 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3096400700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309010b860 0x7f309019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095000700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f309010af10 0x7f30901a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095a00700 1 -- 172.21.15.42:0/1162660077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309010b860 msgr2=0x7f309019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095a00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309010b860 0x7f309019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095a00700 1 -- 172.21.15.42:0/1162660077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f309010af10 msgr2=0x7f30901a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095a00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f309010af10 0x7f30901a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095a00700 1 -- 172.21.15.42:0/1162660077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309019d1e0 con 0x7f3090106630 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3096400700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309010b860 0x7f309019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.817+0000 7f3095a00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 0x7f30901a28c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f308400d950 tx=0x7f308400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.818+0000 7f308e000700 1 -- 172.21.15.42:0/1162660077 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084016070 con 0x7f3090106630 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.818+0000 7f308e000700 1 -- 172.21.15.42:0/1162660077 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3084010460 con 0x7f3090106630 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.818+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309019d4d0 con 0x7f3090106630 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.818+0000 7f308e000700 1 -- 172.21.15.42:0/1162660077 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308400f620 con 0x7f3090106630 2024-05-27T10:22:13.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.818+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30901a9770 con 0x7f3090106630 2024-05-27T10:22:13.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.822+0000 7f308e000700 1 -- 172.21.15.42:0/1162660077 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3084019070 con 0x7f3090106630 2024-05-27T10:22:13.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.823+0000 7f30a7018b80 1 Processor -- start 2024-05-27T10:22:13.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.823+0000 7f30a7018b80 1 -- start start 2024-05-27T10:22:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.823+0000 7f308e000700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f30780b2a70 0x7f30780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 0x5591508fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55915077bf40 0x5591508fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150907c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f30a7018b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559150828f40 con 0x559150867af0 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f30a7018b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559150818ae0 con 0x559150905840 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f30a7018b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55915081cb20 con 0x55915077bf40 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f308e000700 1 -- 172.21.15.42:0/1162660077 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f30840e21c0 con 0x7f3090106630 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 0x5591508fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 0x5591508fac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35862/0 (socket says 172.21.15.42:35862) 2024-05-27T10:22:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f3095a00700 1 -- 172.21.15.42:0/174355129 learned_addr learned my addr 172.21.15.42:0/174355129 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f3095000700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55915077bf40 0x5591508fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.824+0000 7f3096400700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f30780b2a70 0x7f30780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095a00700 1 -- 172.21.15.42:0/174355129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55915077bf40 msgr2=0x5591508fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095a00700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55915077bf40 0x5591508fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095a00700 1 -- 172.21.15.42:0/174355129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 msgr2=0x559150907c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3096400700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150907c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095a00700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150907c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095a00700 1 -- 172.21.15.42:0/174355129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591507e1520 con 0x559150867af0 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095000700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55915077bf40 0x5591508fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:13.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3096400700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150907c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.825+0000 7f3095a00700 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 0x5591508fac40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3084005950 tx=0x7f30840a8d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea18c06955a11b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.826+0000 7f3082a00700 1 -- 172.21.15.42:0/174355129 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084020070 con 0x559150867af0 2024-05-27T10:22:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.826+0000 7f3096400700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f30780b2a70 0x7f30780b4f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3088009040 tx=0x7f3088008fb0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.826+0000 7f3082a00700 1 -- 172.21.15.42:0/174355129 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30840dde50 con 0x559150867af0 2024-05-27T10:22:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.826+0000 7f3082a00700 1 -- 172.21.15.42:0/174355129 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840dfb80 con 0x559150867af0 2024-05-27T10:22:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 -- 172.21.15.42:0/174355129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 msgr2=0x5591508fac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 0x5591508fac40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3084005950 tx=0x7f30840a8d30 comp rx=0 tx=0).stop 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 -- 172.21.15.42:0/174355129 shutdown_connections 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150907c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559150867af0 0x5591508fac40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 --2- 172.21.15.42:0/174355129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55915077bf40 0x5591508fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 -- 172.21.15.42:0/174355129 >> 172.21.15.42:0/174355129 conn(0x559150815490 msgr2=0x5591508f6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 -- 172.21.15.42:0/174355129 shutdown_connections 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 -- 172.21.15.42:0/174355129 wait complete. 2024-05-27T10:22:13.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.827+0000 7f30a7018b80 1 -- start start 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 Processor -- start 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150902810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150867af0 0x559150902d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150905840 0x559150903290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55915071f0a0 con 0x55915077bf40 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591507b9860 con 0x559150867af0 2024-05-27T10:22:13.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.828+0000 7f30a7018b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559150908350 con 0x559150905840 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f30a7018b80 10 client.0 _ll_register_callbacks cb 0x5591507c2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150902810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150902810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35874/0 (socket says 172.21.15.42:35874) 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 -- 172.21.15.42:0/3487413979 learned_addr learned my addr 172.21.15.42:0/3487413979 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3096400700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150905840 0x559150903290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095000700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150867af0 0x559150902d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 -- 172.21.15.42:0/3487413979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150905840 msgr2=0x559150903290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150905840 0x559150903290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 -- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150867af0 msgr2=0x559150902d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150867af0 0x559150902d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.829+0000 7f3095a00700 1 -- 172.21.15.42:0/3487413979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591507689d0 con 0x55915077bf40 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f3095a00700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150902810 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3084010fd0 tx=0x7f30840a9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8416d3f8dd13c25d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f3077400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f3077400700 1 -- 172.21.15.42:0/3487413979 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084020070 con 0x55915077bf40 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f3077400700 1 -- 172.21.15.42:0/3487413979 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f30840dfb80 con 0x55915077bf40 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30840dde50 con 0x55915077bf40 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f3077400700 1 -- 172.21.15.42:0/3487413979 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30840dfd80 con 0x55915077bf40 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f30a7018b80 10 client.16101 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.830+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591507e9160 con 0x55915077bf40 2024-05-27T10:22:13.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f3077400700 1 -- 172.21.15.42:0/3487413979 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30840dde50 con 0x55915077bf40 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f3077400700 1 -- 172.21.15.42:0/3487413979 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f30840a3bb0 con 0x55915077bf40 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f30a7018b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f30a7018b80 10 client.16101 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f30a7018b80 20 client.16101 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f30a7018b80 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559150915070 0x559150917490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f30a7018b80 4 client.16101 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.831+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55915082ae90 con 0x559150915070 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.832+0000 7f3095000700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559150915070 0x559150917490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.833+0000 7f3095000700 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559150915070 0x559150917490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68684aa98c967cb server_cookie=5063409a86c11537 in_seq=0 out_seq=0 2024-05-27T10:22:13.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.833+0000 7f3077400700 10 client.16101 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:13.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f3077400700 1 -- 172.21.15.42:0/3487413979 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55915082ae90 con 0x559150915070 2024-05-27T10:22:13.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f3077400700 10 client.16101 handle_command_reply: tid=0 2024-05-27T10:22:13.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 client.16101 shutdown 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 msgr2=0x559150902810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150902810 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3084010fd0 tx=0x7f30840a9540 comp rx=0 tx=0).stop 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 shutdown_connections 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559150915070 0x559150917490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150905840 0x559150903290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150902810 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 --2- 172.21.15.42:0/3487413979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150867af0 0x559150902d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 >> 172.21.15.42:0/3487413979 conn(0x559150815490 msgr2=0x5591508fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f3077400700 0 client.16101 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 shutdown_connections 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.835+0000 7f30a7018b80 1 -- 172.21.15.42:0/3487413979 wait complete. 2024-05-27T10:22:13.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.836+0000 7f30a7018b80 20 client.16101 trim_cache size 0 max 16384 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 Processor -- start 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 -- start start 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150904cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150902d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150903240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30901b35f0 con 0x55915077bf40 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30901b0e80 con 0x559150905840 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.857+0000 7f30a7018b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f30901aecf0 con 0x559150867af0 2024-05-27T10:22:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150904cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150904cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35886/0 (socket says 172.21.15.42:35886) 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 -- 172.21.15.42:0/445251868 learned_addr learned my addr 172.21.15.42:0/445251868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095000700 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150902d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3096400700 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150903240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 -- 172.21.15.42:0/445251868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 msgr2=0x559150902d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150902d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 -- 172.21.15.42:0/445251868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 msgr2=0x559150903240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150903240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.858+0000 7f3095a00700 1 -- 172.21.15.42:0/445251868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591508f80c0 con 0x55915077bf40 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.859+0000 7f3095a00700 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150904cb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f30840a5510 tx=0x7f30840cd360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72e4610413dcd6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.859+0000 7f3082000700 1 -- 172.21.15.42:0/445251868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840c3070 con 0x55915077bf40 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.859+0000 7f3082000700 1 -- 172.21.15.42:0/445251868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30840dde50 con 0x55915077bf40 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.859+0000 7f3082000700 1 -- 172.21.15.42:0/445251868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840a96f0 con 0x55915077bf40 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 -- 172.21.15.42:0/445251868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 msgr2=0x559150904cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150904cb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f30840a5510 tx=0x7f30840cd360 comp rx=0 tx=0).stop 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 -- 172.21.15.42:0/445251868 shutdown_connections 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x559150903240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150904cb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 --2- 172.21.15.42:0/445251868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150902d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 -- 172.21.15.42:0/445251868 >> 172.21.15.42:0/445251868 conn(0x559150815490 msgr2=0x5591509025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:13.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 -- 172.21.15.42:0/445251868 shutdown_connections 2024-05-27T10:22:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.860+0000 7f30a7018b80 1 -- 172.21.15.42:0/445251868 wait complete. 2024-05-27T10:22:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.861+0000 7f30a7018b80 1 -- start start 2024-05-27T10:22:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.861+0000 7f30a7018b80 1 Processor -- start 2024-05-27T10:22:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150918590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150918ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x55915097e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30901af340 con 0x55915077bf40 2024-05-27T10:22:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30901b07b0 con 0x559150905840 2024-05-27T10:22:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f30901afa10 con 0x559150867af0 2024-05-27T10:22:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f30a7018b80 10 client.0 _ll_register_callbacks cb 0x5591507c2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150918590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f3095a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150918590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35890/0 (socket says 172.21.15.42:35890) 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f3095a00700 1 -- 172.21.15.42:0/762810285 learned_addr learned my addr 172.21.15.42:0/762810285 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f3095000700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150918ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.862+0000 7f3096400700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x55915097e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3095a00700 1 -- 172.21.15.42:0/762810285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 msgr2=0x559150918ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3095a00700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150918ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3095a00700 1 -- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 msgr2=0x55915097e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3095a00700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x55915097e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3095a00700 1 -- 172.21.15.42:0/762810285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915097ee20 con 0x55915077bf40 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3095a00700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150918590 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f30840a9420 tx=0x7f30840a3e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660c53edcea54f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3077400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3077400700 1 -- 172.21.15.42:0/762810285 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840c3040 con 0x55915077bf40 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f3077400700 1 -- 172.21.15.42:0/762810285 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f30840ca390 con 0x55915077bf40 2024-05-27T10:22:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.863+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30840dde50 con 0x55915077bf40 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f3077400700 1 -- 172.21.15.42:0/762810285 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30840d2070 con 0x55915077bf40 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 10 client.16107 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55915097ee20 con 0x55915077bf40 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f3077400700 1 -- 172.21.15.42:0/762810285 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30840dde50 con 0x55915077bf40 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f3077400700 1 -- 172.21.15.42:0/762810285 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f30840ca070 con 0x55915077bf40 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 10 client.16107 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 20 client.16107 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5591509802d0 0x5591509826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 4 client.16107 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fae09024", "release": true, "await": true}] 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.864+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fae09024", "release": true, "await": true}) -- 0x559150824670 con 0x5591509802d0 2024-05-27T10:22:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.865+0000 7f3095000700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5591509802d0 0x5591509826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:13.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.866+0000 7f3095000700 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5591509802d0 0x5591509826f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad435de88e89dd9f server_cookie=a05f22858a2cfcc8 in_seq=0 out_seq=0 2024-05-27T10:22:13.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.866+0000 7f3077400700 10 client.16107 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.875+0000 7f3077400700 1 -- 172.21.15.42:0/762810285 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559150824670 con 0x5591509802d0 2024-05-27T10:22:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.875+0000 7f3077400700 10 client.16107 handle_command_reply: tid=0 2024-05-27T10:22:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 client.16107 shutdown 2024-05-27T10:22:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 msgr2=0x559150918590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150918590 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f30840a9420 tx=0x7f30840a3e90 comp rx=0 tx=0).stop 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 shutdown_connections 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5591509802d0 0x5591509826f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559150905840 0x55915097e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55915077bf40 0x559150918590 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 --2- 172.21.15.42:0/762810285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559150867af0 0x559150918ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 >> 172.21.15.42:0/762810285 conn(0x559150815490 msgr2=0x559150903020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f3077400700 0 client.16107 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 shutdown_connections 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 1 -- 172.21.15.42:0/762810285 wait complete. 2024-05-27T10:22:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.876+0000 7f30a7018b80 20 client.16107 trim_cache size 0 max 16384 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 29, 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 8, 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: "fae09024": { 2024-05-27T10:22:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 8, 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 130.0, 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:22:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:13.881 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:22:13.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f30780b2a70 msgr2=0x7f30780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f30780b2a70 0x7f30780b4f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3088009040 tx=0x7f3088008fb0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 msgr2=0x7f30901a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:13.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 0x7f30901a28c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f308400d950 tx=0x7f308400dd10 comp rx=0 tx=0).stop 2024-05-27T10:22:13.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 shutdown_connections 2024-05-27T10:22:13.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f30780b2a70 0x7f30780b4f80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309010b860 0x7f309019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3090106630 0x7f30901a28c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 --2- 172.21.15.42:0/1162660077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f309010af10 0x7f30901a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 >> 172.21.15.42:0/1162660077 conn(0x7f3090102020 msgr2=0x7f309010a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.877+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 shutdown_connections 2024-05-27T10:22:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:13.878+0000 7f3096e00700 1 -- 172.21.15.42:0/1162660077 wait complete. 2024-05-27T10:22:13.948 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-05-27T10:22:13.948 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-05-27T10:22:15.040 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-05-27T10:22:15.040 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-05-27T10:22:15.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:14 smithi143 ceph-mon[35545]: pgmap v108: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-05-27T10:22:15.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:14 smithi042 ceph-mon[31661]: pgmap v108: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-05-27T10:22:15.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:14 smithi104 ceph-mon[34812]: pgmap v108: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-05-27T10:22:15.503 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fae09024', total seconds elapsed: 21.1 2024-05-27T10:22:16.079 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-05-27T10:22:16.079 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-05-27T10:22:17.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:16 smithi143 ceph-mon[35545]: pgmap v109: 97 pgs: 97 active+clean; 5.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 85 KiB/s wr, 44 op/s 2024-05-27T10:22:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:16 smithi042 ceph-mon[31661]: pgmap v109: 97 pgs: 97 active+clean; 5.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 85 KiB/s wr, 44 op/s 2024-05-27T10:22:17.099 INFO:tasks.workunit.client.0.smithi042.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-05-27T10:22:17.099 INFO:tasks.workunit.client.0.smithi042.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-05-27T10:22:17.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:16 smithi104 ceph-mon[34812]: pgmap v109: 97 pgs: 97 active+clean; 5.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 85 KiB/s wr, 44 op/s 2024-05-27T10:22:17.758 INFO:tasks.workunit.client.0.smithi042.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-05-27T10:22:18.167 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-05-27T10:22:18.171 INFO:tasks.workunit.client.0.smithi042.stdout:Making all in man 2024-05-27T10:22:18.251 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-05-27T10:22:18.251 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Nothing to be done for 'all'. 2024-05-27T10:22:18.251 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-05-27T10:22:18.298 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-27T10:22:18.314 INFO:tasks.workunit.client.0.smithi042.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-27T10:22:18.318 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-27T10:22:18.323 INFO:tasks.workunit.client.0.smithi042.stderr:+ cd src 2024-05-27T10:22:18.334 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir blogtest_in 2024-05-27T10:22:18.348 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 'running blogbench' 2024-05-27T10:22:18.348 INFO:tasks.workunit.client.0.smithi042.stdout:running blogbench 2024-05-27T10:22:18.348 INFO:tasks.workunit.client.0.smithi042.stderr:+ ./blogbench -d blogtest_in 2024-05-27T10:22:18.352 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-05-27T10:22:18.352 INFO:tasks.workunit.client.0.smithi042.stdout:Frequency = 10 secs 2024-05-27T10:22:18.352 INFO:tasks.workunit.client.0.smithi042.stdout:Scratch dir = [blogtest_in] 2024-05-27T10:22:18.707 INFO:tasks.workunit.client.0.smithi042.stdout:Spawning 3 writers... 2024-05-27T10:22:18.708 INFO:tasks.workunit.client.0.smithi042.stdout:Spawning 1 rewriters... 2024-05-27T10:22:18.708 INFO:tasks.workunit.client.0.smithi042.stdout:Spawning 5 commenters... 2024-05-27T10:22:18.708 INFO:tasks.workunit.client.0.smithi042.stdout:Spawning 100 readers... 2024-05-27T10:22:18.714 INFO:tasks.workunit.client.0.smithi042.stdout:Benchmarking for 30 iterations. 2024-05-27T10:22:18.714 INFO:tasks.workunit.client.0.smithi042.stdout:The test will run during 5 minutes. 2024-05-27T10:22:18.714 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-05-27T10:22:18.715 INFO:tasks.workunit.client.0.smithi042.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-05-27T10:22:19.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:18 smithi143 ceph-mon[35545]: pgmap v110: 97 pgs: 97 active+clean; 5.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-05-27T10:22:19.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:18 smithi042 ceph-mon[31661]: pgmap v110: 97 pgs: 97 active+clean; 5.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-05-27T10:22:19.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:18 smithi104 ceph-mon[34812]: pgmap v110: 97 pgs: 97 active+clean; 5.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-05-27T10:22:21.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:20 smithi143 ceph-mon[35545]: pgmap v111: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 87 KiB/s wr, 22 op/s 2024-05-27T10:22:21.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:20 smithi042 ceph-mon[31661]: pgmap v111: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 87 KiB/s wr, 22 op/s 2024-05-27T10:22:21.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:20 smithi104 ceph-mon[34812]: pgmap v111: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 87 KiB/s wr, 22 op/s 2024-05-27T10:22:23.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:22 smithi143 ceph-mon[35545]: pgmap v112: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 72 KiB/s wr, 15 op/s 2024-05-27T10:22:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:22 smithi042 ceph-mon[31661]: pgmap v112: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 72 KiB/s wr, 15 op/s 2024-05-27T10:22:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:22 smithi104 ceph-mon[34812]: pgmap v112: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 72 KiB/s wr, 15 op/s 2024-05-27T10:22:25.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:24 smithi143 ceph-mon[35545]: pgmap v113: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 72 KiB/s wr, 15 op/s 2024-05-27T10:22:25.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:24 smithi042 ceph-mon[31661]: pgmap v113: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 72 KiB/s wr, 15 op/s 2024-05-27T10:22:25.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T10:22:25.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:24 smithi104 ceph-mon[34812]: pgmap v113: 97 pgs: 97 active+clean; 6.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 72 KiB/s wr, 15 op/s 2024-05-27T10:22:25.840 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.935+0000 7f58d4c00640 1 -- 172.21.15.42:0/549741700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c809d250 msgr2=0x7f58c809d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.935+0000 7f58d4c00640 1 --2- 172.21.15.42:0/549741700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c809d250 0x7f58c809d660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009900 tx=0x7f58c402a3f0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.936+0000 7f58d4c00640 1 -- 172.21.15.42:0/549741700 shutdown_connections 2024-05-27T10:22:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.936+0000 7f58d4c00640 1 --2- 172.21.15.42:0/549741700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 0x7f58c80a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.936+0000 7f58d4c00640 1 --2- 172.21.15.42:0/549741700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c80a17e0 0x7f58c80a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.936+0000 7f58d4c00640 1 --2- 172.21.15.42:0/549741700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c809d250 0x7f58c809d660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.936+0000 7f58d4c00640 1 -- 172.21.15.42:0/549741700 >> 172.21.15.42:0/549741700 conn(0x7f58c8009e60 msgr2=0x7f58c800a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.940+0000 7f58d4c00640 1 -- 172.21.15.42:0/549741700 shutdown_connections 2024-05-27T10:22:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.940+0000 7f58d4c00640 1 -- 172.21.15.42:0/549741700 wait complete. 2024-05-27T10:22:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.941+0000 7f58d4c00640 1 Processor -- start 2024-05-27T10:22:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.942+0000 7f58d4c00640 1 -- start start 2024-05-27T10:22:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.943+0000 7f58d4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c809d250 0x7f58c8139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.943+0000 7f58d4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c80a17e0 0x7f58c8139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.943+0000 7f58d4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 0x7f58c81339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.943+0000 7f58d4c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80ad0d0 con 0x7f58c80a22a0 2024-05-27T10:22:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.943+0000 7f58d4c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80acf50 con 0x7f58c809d250 2024-05-27T10:22:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.943+0000 7f58d4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80ad250 con 0x7f58c80a17e0 2024-05-27T10:22:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.944+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 0x7f58c81339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.944+0000 7f58cf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c809d250 0x7f58c8139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.944+0000 7f58cf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c809d250 0x7f58c8139750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45674/0 (socket says 172.21.15.42:45674) 2024-05-27T10:22:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.944+0000 7f58cf400640 1 -- 172.21.15.42:0/2187137946 learned_addr learned my addr 172.21.15.42:0/2187137946 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.945+0000 7f58cea00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c80a17e0 0x7f58c8139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.945+0000 7f58cfe00640 1 -- 172.21.15.42:0/2187137946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c80a17e0 msgr2=0x7f58c8139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.945+0000 7f58cfe00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c80a17e0 0x7f58c8139c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.945+0000 7f58cfe00640 1 -- 172.21.15.42:0/2187137946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c809d250 msgr2=0x7f58c8139750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.945+0000 7f58cfe00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c809d250 0x7f58c8139750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.945+0000 7f58cfe00640 1 -- 172.21.15.42:0/2187137946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c8134050 con 0x7f58c80a22a0 2024-05-27T10:22:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.946+0000 7f58cfe00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 0x7f58c81339b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f58d0069170 tx=0x7f58d007b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.946+0000 7f58c3e00640 1 -- 172.21.15.42:0/2187137946 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0083070 con 0x7f58c80a22a0 2024-05-27T10:22:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.946+0000 7f58c3e00640 1 -- 172.21.15.42:0/2187137946 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58d006c380 con 0x7f58c80a22a0 2024-05-27T10:22:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.946+0000 7f58c3e00640 1 -- 172.21.15.42:0/2187137946 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d007f700 con 0x7f58c80a22a0 2024-05-27T10:22:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.946+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c8134340 con 0x7f58c80a22a0 2024-05-27T10:22:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.947+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c81405d0 con 0x7f58c80a22a0 2024-05-27T10:22:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.951+0000 7f58dd190740 1 Processor -- start 2024-05-27T10:22:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.955+0000 7f58c3e00640 1 -- 172.21.15.42:0/2187137946 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f58d00697b0 con 0x7f58c80a22a0 2024-05-27T10:22:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.955+0000 7f58dd190740 1 -- start start 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58dd190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b46a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58dd190740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b46fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58dd190740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b47510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58dd190740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560374ab7b90 con 0x56037477b2f0 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58dd190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560374aa7730 con 0x5603747a41b0 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58dd190740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560374aab770 con 0x5603747834d0 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.956+0000 7f58cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b46fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b46fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59580/0 (socket says 172.21.15.42:59580) 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 -- 172.21.15.42:0/3423998766 learned_addr learned my addr 172.21.15.42:0/3423998766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cea00640 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b46a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cea00640 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b47510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 -- 172.21.15.42:0/3423998766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 msgr2=0x560374b47510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58c3e00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f58b80b2ab0 0x7f58b80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b47510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 -- 172.21.15.42:0/3423998766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 msgr2=0x560374b46a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b46a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cf400640 1 -- 172.21.15.42:0/3423998766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560374a408b0 con 0x56037477b2f0 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58cea00640 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b47510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.957+0000 7f58c3e00640 1 -- 172.21.15.42:0/2187137946 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f58d014f4a0 con 0x7f58c80a22a0 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.958+0000 7f58cfe00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f58b80b2ab0 0x7f58b80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.958+0000 7f58cf400640 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b46fd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f58c40040d0 tx=0x7f58c4004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cb765ce2643f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.959+0000 7f58cfe00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f58b80b2ab0 0x7f58b80b4fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58d0050b30 tx=0x7f58d0088000 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.959+0000 7f58b7e00640 1 -- 172.21.15.42:0/3423998766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4040070 con 0x56037477b2f0 2024-05-27T10:22:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.959+0000 7f58b7e00640 1 -- 172.21.15.42:0/3423998766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c40054b0 con 0x56037477b2f0 2024-05-27T10:22:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 -- 172.21.15.42:0/3423998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 msgr2=0x560374b46fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b46fd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f58c40040d0 tx=0x7f58c4004100 comp rx=0 tx=0).stop 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 -- 172.21.15.42:0/3423998766 shutdown_connections 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b47510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b46fd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 --2- 172.21.15.42:0/3423998766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b46a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 -- 172.21.15.42:0/3423998766 >> 172.21.15.42:0/3423998766 conn(0x560374798300 msgr2=0x560374b43240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 -- 172.21.15.42:0/3423998766 shutdown_connections 2024-05-27T10:22:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.961+0000 7f58dd190740 1 -- 172.21.15.42:0/3423998766 wait complete. 2024-05-27T10:22:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.962+0000 7f58dd190740 1 -- start start 2024-05-27T10:22:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.963+0000 7f58dd190740 1 Processor -- start 2024-05-27T10:22:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b51130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56037478d790 con 0x56037477b2f0 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560374a4fbc0 con 0x5603747a41b0 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560374b53bb0 con 0x5603747834d0 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.964+0000 7f58dd190740 10 client.0 _ll_register_callbacks cb 0x560374a2d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.965+0000 7f58cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b50bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.965+0000 7f58cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b50bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59588/0 (socket says 172.21.15.42:59588) 2024-05-27T10:22:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.965+0000 7f58cea00640 1 -- 172.21.15.42:0/1528068446 learned_addr learned my addr 172.21.15.42:0/1528068446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.965+0000 7f58cf400640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b51130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.966+0000 7f58cea00640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.967+0000 7f58cf400640 1 -- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 msgr2=0x560374b5cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.967+0000 7f58cf400640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b5cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.967+0000 7f58cf400640 1 -- 172.21.15.42:0/1528068446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 msgr2=0x560374b50bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.967+0000 7f58cf400640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b50bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.967+0000 7f58cf400640 1 -- 172.21.15.42:0/1528068446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603747bc210 con 0x5603747834d0 2024-05-27T10:22:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.967+0000 7f58cea00640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:22:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.968+0000 7f58cf400640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b51130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x560374b47e40 tx=0x7f58c4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e098972d817bb42b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.968+0000 7f58b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:22:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.968+0000 7f58b4c00640 1 -- 172.21.15.42:0/1528068446 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4035040 con 0x5603747834d0 2024-05-27T10:22:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.969+0000 7f58b4c00640 1 -- 172.21.15.42:0/1528068446 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f58c40050f0 con 0x5603747834d0 2024-05-27T10:22:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.969+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c40054b0 con 0x5603747834d0 2024-05-27T10:22:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.970+0000 7f58b4c00640 1 -- 172.21.15.42:0/1528068446 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58c40052f0 con 0x5603747834d0 2024-05-27T10:22:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.970+0000 7f58dd190740 10 client.25741 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.970+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560374888620 con 0x5603747834d0 2024-05-27T10:22:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.971+0000 7f58b4c00640 1 -- 172.21.15.42:0/1528068446 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c4009520 con 0x5603747834d0 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.971+0000 7f58b4c00640 1 -- 172.21.15.42:0/1528068446 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f58c403c650 con 0x5603747834d0 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.971+0000 7f58dd190740 10 client.25741 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.971+0000 7f58dd190740 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.972+0000 7f58dd190740 20 client.25741 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.972+0000 7f58dd190740 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560374762d80 0x560374b62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.972+0000 7f58dd190740 4 client.25741 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.972+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58c40054b0 con 0x560374762d80 2024-05-27T10:22:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.973+0000 7f58cea00640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560374762d80 0x560374b62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.974+0000 7f58cea00640 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560374762d80 0x560374b62cc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5a68b595b7d5c2 server_cookie=5550a305ecbc8fd8 in_seq=0 out_seq=0 2024-05-27T10:22:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.974+0000 7f58b4c00640 10 client.25741 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.976+0000 7f58b4c00640 1 -- 172.21.15.42:0/1528068446 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f58c40054b0 con 0x560374762d80 2024-05-27T10:22:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.976+0000 7f58b4c00640 10 client.25741 handle_command_reply: tid=0 2024-05-27T10:22:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 client.25741 shutdown 2024-05-27T10:22:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 msgr2=0x560374b51130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b51130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x560374b47e40 tx=0x7f58c4009260 comp rx=0 tx=0).stop 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 shutdown_connections 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374b5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x560374762d80 0x560374b62cc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b51130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 --2- 172.21.15.42:0/1528068446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56037477b2f0 0x560374b50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58b4c00640 0 client.25741 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.977+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 >> 172.21.15.42:0/1528068446 conn(0x560374798300 msgr2=0x560374b487b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.978+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 shutdown_connections 2024-05-27T10:22:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.978+0000 7f58dd190740 1 -- 172.21.15.42:0/1528068446 wait complete. 2024-05-27T10:22:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:25.978+0000 7f58dd190740 20 client.25741 trim_cache size 0 max 16384 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 Processor -- start 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 -- start start 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56037477b2f0 0x560374b53320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747a41b0 0x560374b5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f58c814b460 con 0x5603747a41b0 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8148d30 con 0x56037477b2f0 2024-05-27T10:22:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.004+0000 7f58dd190740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8146b80 con 0x5603747834d0 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.005+0000 7f58cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.005+0000 7f58cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:35452/0 (socket says 172.21.15.42:35452) 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.005+0000 7f58cf400640 1 -- 172.21.15.42:0/2500176042 learned_addr learned my addr 172.21.15.42:0/2500176042 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.005+0000 7f58cea00640 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56037477b2f0 0x560374b53320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.005+0000 7f58cea00640 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747a41b0 0x560374b5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.006+0000 7f58cea00640 1 -- 172.21.15.42:0/2500176042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 msgr2=0x560374b474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.006+0000 7f58cea00640 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.006+0000 7f58cea00640 1 -- 172.21.15.42:0/2500176042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56037477b2f0 msgr2=0x560374b53320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.006+0000 7f58cea00640 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56037477b2f0 0x560374b53320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.006+0000 7f58cea00640 1 -- 172.21.15.42:0/2500176042 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560374b63030 con 0x5603747a41b0 2024-05-27T10:22:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.007+0000 7f58cea00640 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747a41b0 0x560374b5cd30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x560374a4fcd0 tx=0x7f58bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf5b7a19bff5a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.008+0000 7f58b7400640 1 -- 172.21.15.42:0/2500176042 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc02a040 con 0x5603747a41b0 2024-05-27T10:22:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.008+0000 7f58b7400640 1 -- 172.21.15.42:0/2500176042 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58bc0120d0 con 0x5603747a41b0 2024-05-27T10:22:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.008+0000 7f58b7400640 1 -- 172.21.15.42:0/2500176042 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc024720 con 0x5603747a41b0 2024-05-27T10:22:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.010+0000 7f58dd190740 1 -- 172.21.15.42:0/2500176042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747a41b0 msgr2=0x560374b5cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.010+0000 7f58dd190740 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747a41b0 0x560374b5cd30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x560374a4fcd0 tx=0x7f58bc0046c0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.010+0000 7f58dd190740 1 -- 172.21.15.42:0/2500176042 shutdown_connections 2024-05-27T10:22:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.011+0000 7f58dd190740 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747a41b0 0x560374b5cd30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.011+0000 7f58dd190740 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5603747834d0 0x560374b474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.011+0000 7f58dd190740 1 --2- 172.21.15.42:0/2500176042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56037477b2f0 0x560374b53320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.011+0000 7f58dd190740 1 -- 172.21.15.42:0/2500176042 >> 172.21.15.42:0/2500176042 conn(0x560374798300 msgr2=0x560374b50eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.011+0000 7f58dd190740 1 -- 172.21.15.42:0/2500176042 shutdown_connections 2024-05-27T10:22:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.011+0000 7f58dd190740 1 -- 172.21.15.42:0/2500176042 wait complete. 2024-05-27T10:22:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.012+0000 7f58dd190740 1 -- start start 2024-05-27T10:22:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.012+0000 7f58dd190740 1 Processor -- start 2024-05-27T10:22:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56037477b2f0 0x560374b54500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 0x560374b54a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374bc3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81471f0 con 0x5603747834d0 2024-05-27T10:22:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8148660 con 0x5603747a41b0 2024-05-27T10:22:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81478c0 con 0x56037477b2f0 2024-05-27T10:22:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58dd190740 10 client.0 _ll_register_callbacks cb 0x7f58bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 0x560374b54a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 0x560374b54a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59608/0 (socket says 172.21.15.42:59608) 2024-05-27T10:22:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.013+0000 7f58cf400640 1 -- 172.21.15.42:0/389206538 learned_addr learned my addr 172.21.15.42:0/389206538 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cea00640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374bc3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cf400640 1 -- 172.21.15.42:0/389206538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56037477b2f0 msgr2=0x560374b54500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cf400640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56037477b2f0 0x560374b54500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cf400640 1 -- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 msgr2=0x560374bc3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cf400640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374bc3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cf400640 1 -- 172.21.15.42:0/389206538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560374bc36a0 con 0x5603747834d0 2024-05-27T10:22:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.014+0000 7f58cea00640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374bc3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.015+0000 7f58cf400640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 0x560374b54a40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f58c403cac0 tx=0x7f58c4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be41e21ab5d2ea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.015+0000 7f58b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.016+0000 7f58b4c00640 1 -- 172.21.15.42:0/389206538 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4035070 con 0x5603747834d0 2024-05-27T10:22:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.016+0000 7f58b4c00640 1 -- 172.21.15.42:0/389206538 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f58c40050f0 con 0x5603747834d0 2024-05-27T10:22:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.016+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58bc0120d0 con 0x5603747834d0 2024-05-27T10:22:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.017+0000 7f58b4c00640 1 -- 172.21.15.42:0/389206538 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58c4005490 con 0x5603747834d0 2024-05-27T10:22:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.017+0000 7f58dd190740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.017+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560374bca9a0 con 0x5603747834d0 2024-05-27T10:22:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58b4c00640 1 -- 172.21.15.42:0/389206538 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58bc0120d0 con 0x5603747834d0 2024-05-27T10:22:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58b4c00640 1 -- 172.21.15.42:0/389206538 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f58c404f420 con 0x5603747834d0 2024-05-27T10:22:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58dd190740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58dd190740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58dd190740 20 client.16125 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58dd190740 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5603747381c0 0x560374bcd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58dd190740 4 client.16125 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:22:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.018+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560374ab9ae0 con 0x5603747381c0 2024-05-27T10:22:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.019+0000 7f58cea00640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5603747381c0 0x560374bcd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.021+0000 7f58cea00640 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5603747381c0 0x560374bcd240 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb52edb375c5cb3 server_cookie=ca127885dd06c880 in_seq=0 out_seq=0 2024-05-27T10:22:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.021+0000 7f58b4c00640 10 client.16125 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.052+0000 7f58b4c00640 1 -- 172.21.15.42:0/389206538 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560374ab9ae0 con 0x5603747381c0 2024-05-27T10:22:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.052+0000 7f58b4c00640 10 client.16125 handle_command_reply: tid=0 2024-05-27T10:22:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 client.16125 shutdown 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 msgr2=0x560374b54a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 0x560374b54a40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f58c403cac0 tx=0x7f58c4005850 comp rx=0 tx=0).stop 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 shutdown_connections 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603747a41b0 0x560374bc3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603747834d0 0x560374b54a40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5603747381c0 0x560374bcd240 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 --2- 172.21.15.42:0/389206538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56037477b2f0 0x560374b54500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 >> 172.21.15.42:0/389206538 conn(0x560374798300 msgr2=0x560374b50be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58b4c00640 0 client.16125 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.053+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 shutdown_connections 2024-05-27T10:22:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.054+0000 7f58dd190740 1 -- 172.21.15.42:0/389206538 wait complete. 2024-05-27T10:22:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.054+0000 7f58dd190740 20 client.16125 trim_cache size 0 max 16384 2024-05-27T10:22:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:22:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:22:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:22:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f58b80b2ab0 msgr2=0x7f58b80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f58b80b2ab0 0x7f58b80b4fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58d0050b30 tx=0x7f58d0088000 comp rx=0 tx=0).stop 2024-05-27T10:22:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 msgr2=0x7f58c81339b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 0x7f58c81339b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f58d0069170 tx=0x7f58d007b6a0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 shutdown_connections 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f58b80b2ab0 0x7f58b80b4fc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c80a22a0 0x7f58c81339b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58c80a17e0 0x7f58c8139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 --2- 172.21.15.42:0/2187137946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58c809d250 0x7f58c8139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 >> 172.21.15.42:0/2187137946 conn(0x7f58c8009e60 msgr2=0x7f58c809bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.055+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 shutdown_connections 2024-05-27T10:22:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:26.056+0000 7f58d4c00640 1 -- 172.21.15.42:0/2187137946 wait complete. 2024-05-27T10:22:26.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:26 smithi042 ceph-mon[31661]: pgmap v114: 97 pgs: 97 active+clean; 19 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-05-27T10:22:27.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:26 smithi143 ceph-mon[35545]: pgmap v114: 97 pgs: 97 active+clean; 19 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-05-27T10:22:27.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:26 smithi104 ceph-mon[34812]: pgmap v114: 97 pgs: 97 active+clean; 19 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 -- 172.21.15.42:0/1844967328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee4105e20 msgr2=0x7fdee410e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 --2- 172.21.15.42:0/1844967328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee4105e20 0x7fdee410e370 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fded0009990 tx=0x7fded0009ca0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 -- 172.21.15.42:0/1844967328 shutdown_connections 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 --2- 172.21.15.42:0/1844967328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdee410e900 0x7fdee4110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 --2- 172.21.15.42:0/1844967328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee41054d0 0x7fdee41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 --2- 172.21.15.42:0/1844967328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee4105e20 0x7fdee410e370 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 -- 172.21.15.42:0/1844967328 >> 172.21.15.42:0/1844967328 conn(0x7fdee40fedb0 msgr2=0x7fdee41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.535+0000 7fdeec000700 1 -- 172.21.15.42:0/1844967328 shutdown_connections 2024-05-27T10:22:28.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.536+0000 7fdeec000700 1 -- 172.21.15.42:0/1844967328 wait complete. 2024-05-27T10:22:28.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.537+0000 7fdeec000700 1 Processor -- start 2024-05-27T10:22:28.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.538+0000 7fdeec000700 1 -- start start 2024-05-27T10:22:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeec000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdee41054d0 0x7fdee410dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeec000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee4105e20 0x7fdee410b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeec000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee410e900 0x7fdee410b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeec000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4104450 con 0x7fdee410e900 2024-05-27T10:22:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeec000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41042d0 con 0x7fdee4105e20 2024-05-27T10:22:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeec000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41045d0 con 0x7fdee41054d0 2024-05-27T10:22:28.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee4105e20 0x7fdee410b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee4105e20 0x7fdee410b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45734/0 (socket says 172.21.15.42:45734) 2024-05-27T10:22:28.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdee8c00700 1 -- 172.21.15.42:0/2149826567 learned_addr learned my addr 172.21.15.42:0/2149826567 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:28.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdee9600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdee41054d0 0x7fdee410dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.539+0000 7fdeea000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee410e900 0x7fdee410b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.540+0000 7fdeea000700 1 -- 172.21.15.42:0/2149826567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdee41054d0 msgr2=0x7fdee410dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.540+0000 7fdeea000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdee41054d0 0x7fdee410dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.540+0000 7fdeea000700 1 -- 172.21.15.42:0/2149826567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee4105e20 msgr2=0x7fdee410b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.540+0000 7fdeea000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee4105e20 0x7fdee410b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.540+0000 7fdeea000700 1 -- 172.21.15.42:0/2149826567 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee410bc50 con 0x7fdee410e900 2024-05-27T10:22:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.541+0000 7fdeea000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee410e900 0x7fdee410b660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c370 tx=0x7fdedc00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:28.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.542+0000 7fdee2000700 1 -- 172.21.15.42:0/2149826567 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc004bb0 con 0x7fdee410e900 2024-05-27T10:22:28.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.542+0000 7fdee2000700 1 -- 172.21.15.42:0/2149826567 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdedc004d50 con 0x7fdee410e900 2024-05-27T10:22:28.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.542+0000 7fdee2000700 1 -- 172.21.15.42:0/2149826567 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc004020 con 0x7fdee410e900 2024-05-27T10:22:28.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.542+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee410bee0 con 0x7fdee410e900 2024-05-27T10:22:28.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.542+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee40744e0 con 0x7fdee410e900 2024-05-27T10:22:28.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.547+0000 7fdee2000700 1 -- 172.21.15.42:0/2149826567 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdedc01b030 con 0x7fdee410e900 2024-05-27T10:22:28.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.549+0000 7fdee2000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdecc0b2b40 0x7fdecc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.549+0000 7fdee2000700 1 -- 172.21.15.42:0/2149826567 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fdedc0e1f80 con 0x7fdee410e900 2024-05-27T10:22:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.549+0000 7fdefac11b80 1 Processor -- start 2024-05-27T10:22:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 -- start start 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa033e40 0x5631aa1b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 0x5631aa1b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdee9600700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdecc0b2b40 0x7fdecc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b2db0 0x5631aa1b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5631aa0e0e40 con 0x5631aa1b8d30 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5631aa0d09e0 con 0x5631aa033e40 2024-05-27T10:22:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.550+0000 7fdefac11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5631aa0d4a20 con 0x5631aa1b2db0 2024-05-27T10:22:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.551+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 0x5631aa1b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.551+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 0x5631aa1b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59628/0 (socket says 172.21.15.42:59628) 2024-05-27T10:22:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.551+0000 7fdee8c00700 1 -- 172.21.15.42:0/3798799720 learned_addr learned my addr 172.21.15.42:0/3798799720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.551+0000 7fdeea000700 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b2db0 0x5631aa1b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee8c00700 1 -- 172.21.15.42:0/3798799720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b2db0 msgr2=0x5631aa1b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee8c00700 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b2db0 0x5631aa1b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee8c00700 1 -- 172.21.15.42:0/3798799720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa033e40 msgr2=0x5631aa1b87f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:22:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee8c00700 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa033e40 0x5631aa1b87f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee8c00700 1 -- 172.21.15.42:0/3798799720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631aa099420 con 0x5631aa1b8d30 2024-05-27T10:22:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee9600700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdecc0b2b40 0x7fdecc0b5050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fded8009c60 tx=0x7fded8009260 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.552+0000 7fdee8c00700 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 0x5631aa1b2870 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fded00096e0 tx=0x7fded0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4555446aaca99400 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:28.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.553+0000 7fded6a00700 1 -- 172.21.15.42:0/3798799720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0027070 con 0x5631aa1b8d30 2024-05-27T10:22:28.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.553+0000 7fded6a00700 1 -- 172.21.15.42:0/3798799720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded0003b10 con 0x5631aa1b8d30 2024-05-27T10:22:28.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.553+0000 7fded6a00700 1 -- 172.21.15.42:0/3798799720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded000bd10 con 0x5631aa1b8d30 2024-05-27T10:22:28.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.554+0000 7fdefac11b80 1 -- 172.21.15.42:0/3798799720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 msgr2=0x5631aa1b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.554+0000 7fdefac11b80 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 0x5631aa1b2870 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fded00096e0 tx=0x7fded0009710 comp rx=0 tx=0).stop 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 -- 172.21.15.42:0/3798799720 shutdown_connections 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b2db0 0x5631aa1b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa033e40 0x5631aa1b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 --2- 172.21.15.42:0/3798799720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b8d30 0x5631aa1b2870 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 -- 172.21.15.42:0/3798799720 >> 172.21.15.42:0/3798799720 conn(0x5631aa093fc0 msgr2=0x5631aa1ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 -- 172.21.15.42:0/3798799720 shutdown_connections 2024-05-27T10:22:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.555+0000 7fdefac11b80 1 -- 172.21.15.42:0/3798799720 wait complete. 2024-05-27T10:22:28.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.556+0000 7fdefac11b80 1 -- start start 2024-05-27T10:22:28.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.556+0000 7fdefac11b80 1 Processor -- start 2024-05-27T10:22:28.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.556+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa033e40 0x5631aa1bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 0x5631aa1bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdefac11b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5631a9fd6fa0 con 0x5631aa1b2db0 2024-05-27T10:22:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdefac11b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5631aa071760 con 0x5631aa1b8d30 2024-05-27T10:22:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdefac11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5631aa1bf7a0 con 0x5631aa033e40 2024-05-27T10:22:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdefac11b80 10 client.0 _ll_register_callbacks cb 0x5631aa07ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 0x5631aa1bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 0x5631aa1bcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59638/0 (socket says 172.21.15.42:59638) 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.557+0000 7fdee8c00700 1 -- 172.21.15.42:0/2256838189 learned_addr learned my addr 172.21.15.42:0/2256838189 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdee9600700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa033e40 0x5631aa1bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdeea000700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdee8c00700 1 -- 172.21.15.42:0/2256838189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa033e40 msgr2=0x5631aa1bc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdee8c00700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa033e40 0x5631aa1bc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdee8c00700 1 -- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 msgr2=0x5631aa1bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdee8c00700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdee8c00700 1 -- 172.21.15.42:0/2256838189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631aa0208d0 con 0x5631aa1b2db0 2024-05-27T10:22:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.558+0000 7fdeea000700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.559+0000 7fdecb400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.559+0000 7fdee8c00700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 0x5631aa1bcb60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fded0003ed0 tx=0x7fded000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1218e0f3f687666 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.559+0000 7fdecb400700 1 -- 172.21.15.42:0/2256838189 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0031060 con 0x5631aa1b2db0 2024-05-27T10:22:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.560+0000 7fdecb400700 1 -- 172.21.15.42:0/2256838189 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fded0003880 con 0x5631aa1b2db0 2024-05-27T10:22:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.560+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0003b10 con 0x5631aa1b2db0 2024-05-27T10:22:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.560+0000 7fdecb400700 1 -- 172.21.15.42:0/2256838189 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fded0027040 con 0x5631aa1b2db0 2024-05-27T10:22:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.560+0000 7fdefac11b80 10 client.16137 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.560+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631aa0a1060 con 0x5631aa1b2db0 2024-05-27T10:22:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.562+0000 7fdecb400700 1 -- 172.21.15.42:0/2256838189 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded0003b10 con 0x5631aa1b2db0 2024-05-27T10:22:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdecb400700 1 -- 172.21.15.42:0/2256838189 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fded0022a30 con 0x5631aa1b2db0 2024-05-27T10:22:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdefac11b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdefac11b80 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdefac11b80 20 client.16137 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa1cc640 0x5631aa1cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdefac11b80 4 client.16137 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.563+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631aa0e2d90 con 0x5631aa1cc640 2024-05-27T10:22:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.564+0000 7fdee9600700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa1cc640 0x5631aa1cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.565+0000 7fdecb400700 10 client.16137 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.565+0000 7fdee9600700 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa1cc640 0x5631aa1cea60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d702aa9885b26a server_cookie=bbcfe060a9eb4ce1 in_seq=0 out_seq=0 2024-05-27T10:22:28.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdecb400700 1 -- 172.21.15.42:0/2256838189 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5631aa0e2d90 con 0x5631aa1cc640 2024-05-27T10:22:28.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdecb400700 10 client.16137 handle_command_reply: tid=0 2024-05-27T10:22:28.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 client.16137 shutdown 2024-05-27T10:22:28.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 msgr2=0x5631aa1bcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 0x5631aa1bcb60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fded0003ed0 tx=0x7fded000b7c0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 shutdown_connections 2024-05-27T10:22:28.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa1cc640 0x5631aa1cea60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa033e40 0x5631aa1bc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2256838189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa1b2db0 0x5631aa1bcb60 secure :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fded0003ed0 tx=0x7fded000b7c0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 >> 172.21.15.42:0/2256838189 conn(0x5631aa093fc0 msgr2=0x5631aa1ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.567+0000 7fdecb400700 0 client.16137 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.568+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 shutdown_connections 2024-05-27T10:22:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.568+0000 7fdefac11b80 1 -- 172.21.15.42:0/2256838189 wait complete. 2024-05-27T10:22:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.569+0000 7fdefac11b80 20 client.16137 trim_cache size 0 max 16384 2024-05-27T10:22:28.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.603+0000 7fdefac11b80 1 Processor -- start 2024-05-27T10:22:28.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 -- start start 2024-05-27T10:22:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b8d30 0x5631aa1c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1bec30 0x5631aa1bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41c0280 con 0x5631aa033e40 2024-05-27T10:22:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bdb10 con 0x5631aa1bec30 2024-05-27T10:22:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdefac11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bb960 con 0x5631aa1b8d30 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.604+0000 7fdee9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.605+0000 7fdee9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59648/0 (socket says 172.21.15.42:59648) 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.605+0000 7fdee9600700 1 -- 172.21.15.42:0/923825238 learned_addr learned my addr 172.21.15.42:0/923825238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.605+0000 7fdee8c00700 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b8d30 0x5631aa1c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.605+0000 7fdeea000700 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1bec30 0x5631aa1bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.605+0000 7fdee9600700 1 -- 172.21.15.42:0/923825238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b8d30 msgr2=0x5631aa1c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.606+0000 7fdee9600700 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b8d30 0x5631aa1c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.606+0000 7fdee9600700 1 -- 172.21.15.42:0/923825238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1bec30 msgr2=0x5631aa1bc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.606+0000 7fdee9600700 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1bec30 0x5631aa1bc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.606+0000 7fdee9600700 1 -- 172.21.15.42:0/923825238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631aa1b7d80 con 0x5631aa033e40 2024-05-27T10:22:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.606+0000 7fdeea000700 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1bec30 0x5631aa1bc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:28.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.606+0000 7fdee9600700 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1b2870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fded8011ab0 tx=0x7fded8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f3bcbf8a0c3a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:28.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.607+0000 7fded6000700 1 -- 172.21.15.42:0/923825238 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded801e320 con 0x5631aa033e40 2024-05-27T10:22:28.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.607+0000 7fded6000700 1 -- 172.21.15.42:0/923825238 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded802a030 con 0x5631aa033e40 2024-05-27T10:22:28.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.607+0000 7fded6000700 1 -- 172.21.15.42:0/923825238 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8026bf0 con 0x5631aa033e40 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 -- 172.21.15.42:0/923825238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 msgr2=0x5631aa1b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1b2870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fded8011ab0 tx=0x7fded8014000 comp rx=0 tx=0).stop 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 -- 172.21.15.42:0/923825238 shutdown_connections 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1bec30 0x5631aa1bc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1b2870 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 --2- 172.21.15.42:0/923825238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1b8d30 0x5631aa1c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.608+0000 7fdefac11b80 1 -- 172.21.15.42:0/923825238 >> 172.21.15.42:0/923825238 conn(0x5631aa093fc0 msgr2=0x5631aa1bb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:28.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.609+0000 7fdefac11b80 1 -- 172.21.15.42:0/923825238 shutdown_connections 2024-05-27T10:22:28.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.609+0000 7fdefac11b80 1 -- 172.21.15.42:0/923825238 wait complete. 2024-05-27T10:22:28.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.610+0000 7fdefac11b80 1 -- start start 2024-05-27T10:22:28.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.610+0000 7fdefac11b80 1 Processor -- start 2024-05-27T10:22:28.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.612+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.612+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdefac11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1bec30 0x5631aa235eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdefac11b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bbfd0 con 0x5631aa033e40 2024-05-27T10:22:28.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdefac11b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bd440 con 0x5631aa1b8d30 2024-05-27T10:22:28.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdefac11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bc6a0 con 0x5631aa1bec30 2024-05-27T10:22:28.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdefac11b80 10 client.0 _ll_register_callbacks cb 0x5631aa07ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:28.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdee8c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45756/0 (socket says 172.21.15.42:45756) 2024-05-27T10:22:28.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdee8c00700 1 -- 172.21.15.42:0/2361550432 learned_addr learned my addr 172.21.15.42:0/2361550432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:28.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdee9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.613+0000 7fdeea000700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1bec30 0x5631aa235eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.614+0000 7fdeea000700 1 -- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 msgr2=0x5631aa1d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.614+0000 7fdeea000700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.614+0000 7fdeea000700 1 -- 172.21.15.42:0/2361550432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 msgr2=0x5631aa1cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.615+0000 7fdeea000700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.615+0000 7fdeea000700 1 -- 172.21.15.42:0/2361550432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631aa2363f0 con 0x5631aa1bec30 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.615+0000 7fdee9600700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.615+0000 7fdeea000700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1bec30 0x5631aa235eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0df6c0 tx=0x7fdedc0a9820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f84d0f9d37ee77 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:28.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.615+0000 7fdecb400700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.616+0000 7fdecb400700 1 -- 172.21.15.42:0/2361550432 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0a99e0 con 0x5631aa1bec30 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.616+0000 7fdecb400700 1 -- 172.21.15.42:0/2361550432 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fdedc0a58d0 con 0x5631aa1bec30 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.616+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded802a030 con 0x5631aa1bec30 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.616+0000 7fdecb400700 1 -- 172.21.15.42:0/2361550432 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdedc0a92e0 con 0x5631aa1bec30 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.616+0000 7fdefac11b80 10 client.25762 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.616+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631aa2375d0 con 0x5631aa1bec30 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.617+0000 7fdecb400700 1 -- 172.21.15.42:0/2361550432 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded802a030 con 0x5631aa1bec30 2024-05-27T10:22:28.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.617+0000 7fdecb400700 1 -- 172.21.15.42:0/2361550432 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdedc0a4c50 con 0x5631aa1bec30 2024-05-27T10:22:28.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.619+0000 7fdefac11b80 10 client.25762 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:28.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.619+0000 7fdefac11b80 10 client.25762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:28.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.619+0000 7fdefac11b80 20 client.25762 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:28.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.619+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa237c30 0x5631aa23a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:28.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.619+0000 7fdefac11b80 4 client.25762 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T10:22:28.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.619+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631aa0dc570 con 0x5631aa237c30 2024-05-27T10:22:28.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.620+0000 7fdee9600700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa237c30 0x5631aa23a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:28.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.621+0000 7fdee9600700 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa237c30 0x5631aa23a050 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f273bb0f4e42bf4 server_cookie=b4157ab50c02dd3b in_seq=0 out_seq=0 2024-05-27T10:22:28.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.621+0000 7fdecb400700 10 client.25762 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:28.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.622+0000 7fdecb400700 1 -- 172.21.15.42:0/2361550432 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5631aa0dc570 con 0x5631aa237c30 2024-05-27T10:22:28.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.622+0000 7fdecb400700 10 client.25762 handle_command_reply: tid=0 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.622+0000 7fdefac11b80 1 client.25762 shutdown 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.622+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1bec30 msgr2=0x5631aa235eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.622+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1bec30 0x5631aa235eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0df6c0 tx=0x7fdedc0a9820 comp rx=0 tx=0).stop 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 shutdown_connections 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5631aa237c30 0x5631aa23a050 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631aa1bec30 0x5631aa235eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631aa033e40 0x5631aa1cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 --2- 172.21.15.42:0/2361550432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631aa1b8d30 0x5631aa1d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 >> 172.21.15.42:0/2361550432 conn(0x5631aa093fc0 msgr2=0x5631aa1b6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:28.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdecb400700 0 client.25762 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:28.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 shutdown_connections 2024-05-27T10:22:28.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.623+0000 7fdefac11b80 1 -- 172.21.15.42:0/2361550432 wait complete. 2024-05-27T10:22:28.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.624+0000 7fdefac11b80 20 client.25762 trim_cache size 0 max 16384 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdecc0b2b40 msgr2=0x7fdecc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdecc0b2b40 0x7fdecc0b5050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fded8009c60 tx=0x7fded8009260 comp rx=0 tx=0).stop 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee410e900 msgr2=0x7fdee410b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:28.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee410e900 0x7fdee410b660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c370 tx=0x7fdedc00c680 comp rx=0 tx=0).stop 2024-05-27T10:22:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 shutdown_connections 2024-05-27T10:22:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdecc0b2b40 0x7fdecc0b5050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee410e900 0x7fdee410b660 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdee41054d0 0x7fdee410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 --2- 172.21.15.42:0/2149826567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdee4105e20 0x7fdee410b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.625+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 >> 172.21.15.42:0/2149826567 conn(0x7fdee40fedb0 msgr2=0x7fdee4112f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.626+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 shutdown_connections 2024-05-27T10:22:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:28.626+0000 7fdeec000700 1 -- 172.21.15.42:0/2149826567 wait complete. 2024-05-27T10:22:28.714 INFO:tasks.workunit.client.0.smithi042.stdout: 2 6365 191 5070 202 3914 251 2024-05-27T10:22:29.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:28 smithi143 ceph-mon[35545]: pgmap v115: 97 pgs: 97 active+clean; 19 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-05-27T10:22:29.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:28 smithi042 ceph-mon[31661]: pgmap v115: 97 pgs: 97 active+clean; 19 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-05-27T10:22:29.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:28 smithi104 ceph-mon[34812]: pgmap v115: 97 pgs: 97 active+clean; 19 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-05-27T10:22:29.884 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T10:22:29.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:22:30.465 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:30.999 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:30 smithi042 ceph-mon[31661]: pgmap v116: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-05-27T10:22:31.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:30 smithi143 ceph-mon[35545]: pgmap v116: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-05-27T10:22:31.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:30 smithi104 ceph-mon[34812]: pgmap v116: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-05-27T10:22:32.984 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:32 smithi042 ceph-mon[31661]: pgmap v117: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-27T10:22:33.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:32 smithi143 ceph-mon[35545]: pgmap v117: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-27T10:22:33.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 -- 172.21.15.42:0/148501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 msgr2=0x7f332c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:33.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 --2- 172.21.15.42:0/148501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c1038c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f331c0099c0 tx=0x7f331c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 -- 172.21.15.42:0/148501780 shutdown_connections 2024-05-27T10:22:33.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 --2- 172.21.15.42:0/148501780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f332c108730 0x7f332c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 --2- 172.21.15.42:0/148501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c1038c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 --2- 172.21.15.42:0/148501780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c107de0 0x7f332c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 -- 172.21.15.42:0/148501780 >> 172.21.15.42:0/148501780 conn(0x7f332c0fedb0 msgr2=0x7f332c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:33.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.136+0000 7f3331800700 1 -- 172.21.15.42:0/148501780 shutdown_connections 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.137+0000 7f3331800700 1 -- 172.21.15.42:0/148501780 wait complete. 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.138+0000 7f3331800700 1 Processor -- start 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 -- start start 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c10fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f332c107de0 0x7f332c10cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c108730 0x7f332c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:33.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f332c112150 con 0x7f332c1034b0 2024-05-27T10:22:33.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f332c111fd0 con 0x7f332c108730 2024-05-27T10:22:33.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f3331800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1122d0 con 0x7f332c107de0 2024-05-27T10:22:33.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f332aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c10fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:33.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f332aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c10fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50314/0 (socket says 172.21.15.42:50314) 2024-05-27T10:22:33.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.139+0000 7f332aa00700 1 -- 172.21.15.42:0/3795425496 learned_addr learned my addr 172.21.15.42:0/3795425496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:33.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332aa00700 1 -- 172.21.15.42:0/3795425496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f332c107de0 msgr2=0x7f332c10cf90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T10:22:33.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332aa00700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f332c107de0 0x7f332c10cf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332aa00700 1 -- 172.21.15.42:0/3795425496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c108730 msgr2=0x7f332c10d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:33.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332b400700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c108730 0x7f332c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:33.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332aa00700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c108730 0x7f332c10d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332aa00700 1 -- 172.21.15.42:0/3795425496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c10dc10 con 0x7f332c1034b0 2024-05-27T10:22:33.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.140+0000 7f332b400700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c108730 0x7f332c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:33.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.141+0000 7f332aa00700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c10fe40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f331c00b580 tx=0x7f331c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:33.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.141+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c027070 con 0x7f332c1034b0 2024-05-27T10:22:33.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.141+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c1ad830 con 0x7f332c1034b0 2024-05-27T10:22:33.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.142+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c1adc20 con 0x7f332c1034b0 2024-05-27T10:22:33.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.142+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f331c00fdd0 con 0x7f332c1034b0 2024-05-27T10:22:33.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.142+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c022cc0 con 0x7f332c1034b0 2024-05-27T10:22:33.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.144+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c005330 con 0x7f332c1034b0 2024-05-27T10:22:33.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.148+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f331c022470 con 0x7f332c1034b0 2024-05-27T10:22:33.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.150+0000 7f3323400700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f33100b2a70 0x7f33100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:33.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.150+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f331c0e3d60 con 0x7f332c1034b0 2024-05-27T10:22:33.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.151+0000 7f332a000700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f33100b2a70 0x7f33100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:33.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.153+0000 7f332a000700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f33100b2a70 0x7f33100b4f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f332c106990 tx=0x7f331800b3f0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:33.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.154+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f331c030070 con 0x7f332c1034b0 2024-05-27T10:22:33.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:32 smithi104 ceph-mon[34812]: pgmap v117: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-27T10:22:33.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.618+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f330c005620 con 0x7f332c1034b0 2024-05-27T10:22:33.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.620+0000 7f3323400700 1 -- 172.21.15.42:0/3795425496 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11881 (secure 0 0 0) 0x7f331c013070 con 0x7f332c1034b0 2024-05-27T10:22:33.621 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:22:33.621 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:21:52.648444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:22:33.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f33100b2a70 msgr2=0x7f33100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f33100b2a70 0x7f33100b4f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f332c106990 tx=0x7f331800b3f0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 msgr2=0x7f332c10fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c10fe40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f331c00b580 tx=0x7f331c00f510 comp rx=0 tx=0).stop 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 shutdown_connections 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f33100b2a70 0x7f33100b4f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f332c108730 0x7f332c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c1034b0 0x7f332c10fe40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 --2- 172.21.15.42:0/3795425496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f332c107de0 0x7f332c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.624+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 >> 172.21.15.42:0/3795425496 conn(0x7f332c0fedb0 msgr2=0x7f332c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.625+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 shutdown_connections 2024-05-27T10:22:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:33.625+0000 7f3331800700 1 -- 172.21.15.42:0/3795425496 wait complete. 2024-05-27T10:22:33.630 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-05-27T10:22:34.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:33 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3795425496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:34.075 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-27T10:22:34.075 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-27T10:22:34.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:22:34.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:33 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3795425496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:34.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:33 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3795425496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:34.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T10:22:34.786 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:34.959 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:34 smithi042 ceph-mon[31661]: pgmap v118: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-27T10:22:35.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:34 smithi143 ceph-mon[35545]: pgmap v118: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-27T10:22:35.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:34 smithi104 ceph-mon[34812]: pgmap v118: 97 pgs: 97 active+clean; 22 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-27T10:22:36.628 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:37.043 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:36 smithi042 ceph-mon[31661]: pgmap v119: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-27T10:22:37.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:36 smithi104 ceph-mon[34812]: pgmap v119: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-27T10:22:37.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:36 smithi143 ceph-mon[35545]: pgmap v119: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-27T10:22:38.715 INFO:tasks.workunit.client.0.smithi042.stdout: 3 9961 91 7876 61 1387 209 2024-05-27T10:22:38.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 -- 172.21.15.42:0/3214952256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0108730 msgr2=0x7fbec0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:38.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 --2- 172.21.15.42:0/3214952256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0108730 0x7fbec0110090 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00099c0 tx=0x7fbeb0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 -- 172.21.15.42:0/3214952256 shutdown_connections 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 --2- 172.21.15.42:0/3214952256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0108730 0x7fbec0110090 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 --2- 172.21.15.42:0/3214952256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 --2- 172.21.15.42:0/3214952256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0107de0 0x7fbec01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 -- 172.21.15.42:0/3214952256 >> 172.21.15.42:0/3214952256 conn(0x7fbec00fedb0 msgr2=0x7fbec01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.408+0000 7fbec7a00700 1 -- 172.21.15.42:0/3214952256 shutdown_connections 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.409+0000 7fbec7a00700 1 -- 172.21.15.42:0/3214952256 wait complete. 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.410+0000 7fbec7a00700 1 Processor -- start 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.410+0000 7fbec7a00700 1 -- start start 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec7a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec7a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0107de0 0x7fbec019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec7a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0108730 0x7fbec01a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec7a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbec01143b0 con 0x7fbec0107de0 2024-05-27T10:22:38.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec7a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0114230 con 0x7fbec0108730 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec7a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0114530 con 0x7fbec01034b0 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.411+0000 7fbec5000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.412+0000 7fbec5000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:47320/0 (socket says 172.21.15.42:47320) 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.412+0000 7fbec5000700 1 -- 172.21.15.42:0/2510977647 learned_addr learned my addr 172.21.15.42:0/2510977647 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.412+0000 7fbebfe00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0107de0 0x7fbec019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.412+0000 7fbec5a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0108730 0x7fbec01a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbec5000700 1 -- 172.21.15.42:0/2510977647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0108730 msgr2=0x7fbec01a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbec5000700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0108730 0x7fbec01a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbec5000700 1 -- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0107de0 msgr2=0x7fbec019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbec5000700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0107de0 0x7fbec019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbec5000700 1 -- 172.21.15.42:0/2510977647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec01a1e20 con 0x7fbec01034b0 2024-05-27T10:22:38.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbec5a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0108730 0x7fbec01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.413+0000 7fbebfe00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0107de0 0x7fbec019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.414+0000 7fbec5000700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec019d000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800c9b0 tx=0x7fbeb800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.414+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8007b00 con 0x7fbec01034b0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.414+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec01a2110 con 0x7fbec01034b0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.415+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeb8007ca0 con 0x7fbec01034b0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.415+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8018660 con 0x7fbec01034b0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.416+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec01a9ae0 con 0x7fbec01034b0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.418+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec00512b0 con 0x7fbec01034b0 2024-05-27T10:22:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.422+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbeb8046990 con 0x7fbec01034b0 2024-05-27T10:22:38.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.424+0000 7fbebd600700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbea40b29f0 0x7fbea40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:38.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.424+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbeb80e2790 con 0x7fbec01034b0 2024-05-27T10:22:38.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.425+0000 7fbebfe00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbea40b29f0 0x7fbea40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:38.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.426+0000 7fbebfe00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbea40b29f0 0x7fbea40b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009550 tx=0x7fbea80091b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:38.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.436+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbeb8014030 con 0x7fbec01034b0 2024-05-27T10:22:38.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.921+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbec019e360 con 0x7fbec01034b0 2024-05-27T10:22:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:22:38.925 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:21:52.648444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:22:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.924+0000 7fbebd600700 1 -- 172.21.15.42:0/2510977647 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11881 (secure 0 0 0) 0x7fbec019e360 con 0x7fbec01034b0 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.927+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbea40b29f0 msgr2=0x7fbea40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.927+0000 7fbec7a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbea40b29f0 0x7fbea40b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009550 tx=0x7fbea80091b0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.927+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 msgr2=0x7fbec019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.927+0000 7fbec7a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec019d000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800c9b0 tx=0x7fbeb800ccc0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 shutdown_connections 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbea40b29f0 0x7fbea40b4f00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbec0108730 0x7fbec01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec01034b0 0x7fbec019d000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 --2- 172.21.15.42:0/2510977647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbec0107de0 0x7fbec019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 >> 172.21.15.42:0/2510977647 conn(0x7fbec00fedb0 msgr2=0x7fbec0109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 shutdown_connections 2024-05-27T10:22:38.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:38.928+0000 7fbec7a00700 1 -- 172.21.15.42:0/2510977647 wait complete. 2024-05-27T10:22:38.931 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-05-27T10:22:39.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:38 smithi042 ceph-mon[31661]: pgmap v120: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 575 KiB/s wr, 52 op/s 2024-05-27T10:22:39.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:38 smithi104 ceph-mon[34812]: pgmap v120: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 575 KiB/s wr, 52 op/s 2024-05-27T10:22:39.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:38 smithi143 ceph-mon[35545]: pgmap v120: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 575 KiB/s wr, 52 op/s 2024-05-27T10:22:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 -- 172.21.15.42:0/1924149119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14102b90 msgr2=0x7fbb14102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 --2- 172.21.15.42:0/1924149119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14102b90 0x7fbb14102fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009860 tx=0x7fbb0402a340 comp rx=0 tx=0).stop 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 -- 172.21.15.42:0/1924149119 shutdown_connections 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 --2- 172.21.15.42:0/1924149119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14108010 0x7fbb141084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 --2- 172.21.15.42:0/1924149119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 0x7fbb141079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 --2- 172.21.15.42:0/1924149119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14102b90 0x7fbb14102fa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 -- 172.21.15.42:0/1924149119 >> 172.21.15.42:0/1924149119 conn(0x7fbb140782d0 msgr2=0x7fbb14078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 -- 172.21.15.42:0/1924149119 shutdown_connections 2024-05-27T10:22:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.362+0000 7fbb19800640 1 -- 172.21.15.42:0/1924149119 wait complete. 2024-05-27T10:22:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.363+0000 7fbb19800640 1 Processor -- start 2024-05-27T10:22:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.364+0000 7fbb19800640 1 -- start start 2024-05-27T10:22:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.365+0000 7fbb19800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14102b90 0x7fbb14110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.365+0000 7fbb19800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 0x7fbb14110590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.365+0000 7fbb19800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14108010 0x7fbb14110ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14102b90 0x7fbb14110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14102b90 0x7fbb14110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:57338/0 (socket says 172.21.15.42:57338) 2024-05-27T10:22:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb12a00640 1 -- 172.21.15.42:0/1458580565 learned_addr learned my addr 172.21.15.42:0/1458580565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1411a2e0 con 0x7fbb141075e0 2024-05-27T10:22:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1411a160 con 0x7fbb14102b90 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1411a460 con 0x7fbb14108010 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.366+0000 7fbb12000640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 0x7fbb14110590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb13400640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14108010 0x7fbb14110ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb12000640 1 -- 172.21.15.42:0/1458580565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14108010 msgr2=0x7fbb14110ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb12000640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14108010 0x7fbb14110ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb12000640 1 -- 172.21.15.42:0/1458580565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14102b90 msgr2=0x7fbb14110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb12000640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14102b90 0x7fbb14110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb12000640 1 -- 172.21.15.42:0/1458580565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb14114a00 con 0x7fbb141075e0 2024-05-27T10:22:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.367+0000 7fbb13400640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14108010 0x7fbb14110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.368+0000 7fbb12000640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 0x7fbb14110590 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d9c0 tx=0x7fbb0000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.368+0000 7fbb0b400640 1 -- 172.21.15.42:0/1458580565 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00004230 con 0x7fbb141075e0 2024-05-27T10:22:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.368+0000 7fbb0b400640 1 -- 172.21.15.42:0/1458580565 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb00009dd0 con 0x7fbb141075e0 2024-05-27T10:22:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.368+0000 7fbb0b400640 1 -- 172.21.15.42:0/1458580565 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00003e00 con 0x7fbb141075e0 2024-05-27T10:22:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.369+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb14114cf0 con 0x7fbb141075e0 2024-05-27T10:22:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.369+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141151f0 con 0x7fbb141075e0 2024-05-27T10:22:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.375+0000 7fbb207d8740 1 Processor -- start 2024-05-27T10:22:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.376+0000 7fbb207d8740 1 -- start start 2024-05-27T10:22:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.377+0000 7fbb0b400640 1 -- 172.21.15.42:0/1458580565 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbb000035f0 con 0x7fbb141075e0 2024-05-27T10:22:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.377+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 0x55f39a55c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.377+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1912f0 0x55f39a55cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.377+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a55d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb207d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f39a4cdb90 con 0x55f39a1994d0 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb207d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f39a4bd730 con 0x55f39a1912f0 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb207d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f39a4c1770 con 0x55f39a1ba1b0 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb13400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 0x55f39a55c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb13400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 0x55f39a55c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:47344/0 (socket says 172.21.15.42:47344) 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb13400640 1 -- 172.21.15.42:0/2241347820 learned_addr learned my addr 172.21.15.42:0/2241347820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.378+0000 7fbb12000640 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a55d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb13400640 1 -- 172.21.15.42:0/2241347820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1912f0 msgr2=0x55f39a55cda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb13400640 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1912f0 0x55f39a55cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb13400640 1 -- 172.21.15.42:0/2241347820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 msgr2=0x55f39a55d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb13400640 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a55d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb13400640 1 -- 172.21.15.42:0/2241347820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39a4568b0 con 0x55f39a1ba1b0 2024-05-27T10:22:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb12000640 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a55d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.379+0000 7fbb13400640 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 0x55f39a55c860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009f30 tx=0x7fbb0c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c447079c157060ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.380+0000 7fbaffe00640 1 -- 172.21.15.42:0/2241347820 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c016070 con 0x55f39a1ba1b0 2024-05-27T10:22:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.380+0000 7fbaffe00640 1 -- 172.21.15.42:0/2241347820 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb0c01baf0 con 0x55f39a1ba1b0 2024-05-27T10:22:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.380+0000 7fbaffe00640 1 -- 172.21.15.42:0/2241347820 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c012b20 con 0x55f39a1ba1b0 2024-05-27T10:22:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.380+0000 7fbb0b400640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbaf80b29d0 0x7fbaf80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.381+0000 7fbb12a00640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbaf80b29d0 0x7fbaf80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.381+0000 7fbb207d8740 1 -- 172.21.15.42:0/2241347820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 msgr2=0x55f39a55c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.381+0000 7fbb207d8740 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 0x55f39a55c860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009f30 tx=0x7fbb0c010800 comp rx=0 tx=0).stop 2024-05-27T10:22:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb0b400640 1 -- 172.21.15.42:0/1458580565 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbb000e24d0 con 0x7fbb141075e0 2024-05-27T10:22:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb12a00640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbaf80b29d0 0x7fbaf80b4ee0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402a930 tx=0x7fbb040058b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 -- 172.21.15.42:0/2241347820 shutdown_connections 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a55d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1912f0 0x55f39a55cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 --2- 172.21.15.42:0/2241347820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1ba1b0 0x55f39a55c860 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 -- 172.21.15.42:0/2241347820 >> 172.21.15.42:0/2241347820 conn(0x55f39a1ae300 msgr2=0x55f39a559010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 -- 172.21.15.42:0/2241347820 shutdown_connections 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.382+0000 7fbb207d8740 1 -- 172.21.15.42:0/2241347820 wait complete. 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.383+0000 7fbb207d8740 1 -- start start 2024-05-27T10:22:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.383+0000 7fbb207d8740 1 Processor -- start 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a566a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1994d0 0x55f39a566fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1ba1b0 0x55f39a572cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f39a1a3790 con 0x55f39a1ba1b0 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f39a465bc0 con 0x55f39a1994d0 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f39a569990 con 0x55f39a1912f0 2024-05-27T10:22:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.384+0000 7fbb207d8740 10 client.0 _ll_register_callbacks cb 0x55f39a443400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.385+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a566a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.385+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a566a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:47348/0 (socket says 172.21.15.42:47348) 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.385+0000 7fbb12a00640 1 -- 172.21.15.42:0/319123392 learned_addr learned my addr 172.21.15.42:0/319123392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.385+0000 7fbb12000640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1994d0 0x55f39a566fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.385+0000 7fbb13400640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1ba1b0 0x55f39a572cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.386+0000 7fbb13400640 1 -- 172.21.15.42:0/319123392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 msgr2=0x55f39a566a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.386+0000 7fbb13400640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a566a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.386+0000 7fbb13400640 1 -- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1994d0 msgr2=0x55f39a566fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.386+0000 7fbb13400640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1994d0 0x55f39a566fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.386+0000 7fbb13400640 1 -- 172.21.15.42:0/319123392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39a1d2210 con 0x55f39a1ba1b0 2024-05-27T10:22:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.386+0000 7fbb13400640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1ba1b0 0x55f39a572cc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006010 tx=0x7fbb0c01beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bd1c414c20beff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.387+0000 7fbafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.387+0000 7fbafcc00640 1 -- 172.21.15.42:0/319123392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c029060 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.387+0000 7fbafcc00640 1 -- 172.21.15.42:0/319123392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbb0c004b40 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.387+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c01baf0 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.387+0000 7fbafcc00640 1 -- 172.21.15.42:0/319123392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c016070 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 10 client.16167 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f39a29e620 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbafcc00640 1 -- 172.21.15.42:0/319123392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb0c01baf0 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbafcc00640 1 -- 172.21.15.42:0/319123392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbb0c00f040 con 0x55f39a1ba1b0 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 20 client.16167 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a178d80 0x55f39a578bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 4 client.16167 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.388+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f39a4cfae0 con 0x55f39a178d80 2024-05-27T10:22:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.389+0000 7fbb12a00640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a178d80 0x55f39a578bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.390+0000 7fbb12a00640 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a178d80 0x55f39a578bb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b389ab2158606189 server_cookie=b6fad8bac16d1ff0 in_seq=0 out_seq=0 2024-05-27T10:22:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.390+0000 7fbafcc00640 10 client.16167 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.393+0000 7fbafcc00640 1 -- 172.21.15.42:0/319123392 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f39a4cfae0 con 0x55f39a178d80 2024-05-27T10:22:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.393+0000 7fbafcc00640 10 client.16167 handle_command_reply: tid=0 2024-05-27T10:22:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.393+0000 7fbb207d8740 1 client.16167 shutdown 2024-05-27T10:22:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.393+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1ba1b0 msgr2=0x55f39a572cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.393+0000 7fbb207d8740 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1ba1b0 0x55f39a572cc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006010 tx=0x7fbb0c01beb0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 shutdown_connections 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1ba1b0 0x55f39a572cc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1994d0 0x55f39a566fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a178d80 0x55f39a578bb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 --2- 172.21.15.42:0/319123392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a566a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 >> 172.21.15.42:0/319123392 conn(0x55f39a1ae300 msgr2=0x55f39a55c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbafcc00640 0 client.16167 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 shutdown_connections 2024-05-27T10:22:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.394+0000 7fbb207d8740 1 -- 172.21.15.42:0/319123392 wait complete. 2024-05-27T10:22:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.395+0000 7fbb207d8740 20 client.16167 trim_cache size 0 max 16384 2024-05-27T10:22:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.422+0000 7fbb207d8740 1 Processor -- start 2024-05-27T10:22:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.422+0000 7fbb207d8740 1 -- start start 2024-05-27T10:22:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.423+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1912f0 0x55f39a569190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.423+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 0x55f39a558330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a572c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb207d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141c1090 con 0x55f39a1912f0 2024-05-27T10:22:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb207d8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141be960 con 0x55f39a1ba1b0 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb207d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141bc7b0 con 0x55f39a1994d0 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb12000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 0x55f39a558330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1912f0 0x55f39a569190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb12000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 0x55f39a558330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:47360/0 (socket says 172.21.15.42:47360) 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.424+0000 7fbb12000640 1 -- 172.21.15.42:0/1356531038 learned_addr learned my addr 172.21.15.42:0/1356531038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb13400640 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a572c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb12000640 1 -- 172.21.15.42:0/1356531038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 msgr2=0x55f39a572c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb12000640 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a572c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb12000640 1 -- 172.21.15.42:0/1356531038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1912f0 msgr2=0x55f39a569190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb12000640 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1912f0 0x55f39a569190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb12000640 1 -- 172.21.15.42:0/1356531038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39a55dc10 con 0x55f39a1994d0 2024-05-27T10:22:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.425+0000 7fbb12000640 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 0x55f39a558330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb000dfcd0 tx=0x7fbb000a5040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d9be0d9d3326a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.426+0000 7fbaff400640 1 -- 172.21.15.42:0/1356531038 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000a9310 con 0x55f39a1994d0 2024-05-27T10:22:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.426+0000 7fbaff400640 1 -- 172.21.15.42:0/1356531038 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb000a9f00 con 0x55f39a1994d0 2024-05-27T10:22:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.426+0000 7fbaff400640 1 -- 172.21.15.42:0/1356531038 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000a5a90 con 0x55f39a1994d0 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 -- 172.21.15.42:0/1356531038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 msgr2=0x55f39a558330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 0x55f39a558330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb000dfcd0 tx=0x7fbb000a5040 comp rx=0 tx=0).stop 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 -- 172.21.15.42:0/1356531038 shutdown_connections 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a572c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1994d0 0x55f39a558330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 --2- 172.21.15.42:0/1356531038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1912f0 0x55f39a569190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.427+0000 7fbb207d8740 1 -- 172.21.15.42:0/1356531038 >> 172.21.15.42:0/1356531038 conn(0x55f39a1ae300 msgr2=0x55f39a5662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.428+0000 7fbb207d8740 1 -- 172.21.15.42:0/1356531038 shutdown_connections 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.428+0000 7fbb207d8740 1 -- 172.21.15.42:0/1356531038 wait complete. 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.428+0000 7fbb207d8740 1 -- start start 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.428+0000 7fbb207d8740 1 Processor -- start 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a578d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a579250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb207d8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a5deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb12000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a579250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb12000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a579250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50378/0 (socket says 172.21.15.42:50378) 2024-05-27T10:22:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb12000640 1 -- 172.21.15.42:0/601182765 learned_addr learned my addr 172.21.15.42:0/601182765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.429+0000 7fbb207d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141bce20 con 0x55f39a1994d0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141be290 con 0x55f39a1ba1b0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141bd4f0 con 0x55f39a1912f0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb12a00640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a578d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb207d8740 10 client.0 _ll_register_callbacks cb 0x7fbb0c009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb13400640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a5deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb12000640 1 -- 172.21.15.42:0/601182765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 msgr2=0x55f39a578d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb12000640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a578d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb12000640 1 -- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 msgr2=0x55f39a5deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb12000640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a5deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.430+0000 7fbb12000640 1 -- 172.21.15.42:0/601182765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39a579af0 con 0x55f39a1994d0 2024-05-27T10:22:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.431+0000 7fbb12000640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a579250 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb000a9500 tx=0x7fbb000a5d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e01e792a60e9085 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.431+0000 7fbafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.431+0000 7fbafcc00640 1 -- 172.21.15.42:0/601182765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000e0060 con 0x55f39a1994d0 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.431+0000 7fbafcc00640 1 -- 172.21.15.42:0/601182765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fbb000a5df0 con 0x55f39a1994d0 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.431+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb000a9f00 con 0x55f39a1994d0 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.432+0000 7fbafcc00640 1 -- 172.21.15.42:0/601182765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb000adbb0 con 0x55f39a1994d0 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.432+0000 7fbb207d8740 10 client.16173 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.432+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f39a5e0590 con 0x55f39a1994d0 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.433+0000 7fbafcc00640 1 -- 172.21.15.42:0/601182765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb000a9f00 con 0x55f39a1994d0 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.433+0000 7fbafcc00640 1 -- 172.21.15.42:0/601182765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbb00009940 con 0x55f39a1994d0 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb207d8740 10 client.16173 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb207d8740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb207d8740 20 client.16173 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb207d8740 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a14e1c0 0x55f39a5e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb207d8740 4 client.16173 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:22:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f39a4c92c0 con 0x55f39a14e1c0 2024-05-27T10:22:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.434+0000 7fbb12a00640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a14e1c0 0x55f39a5e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.435+0000 7fbb12a00640 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a14e1c0 0x55f39a5e2c50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216b1bd078321fb3 server_cookie=7f0dc22cc458f294 in_seq=0 out_seq=0 2024-05-27T10:22:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.435+0000 7fbafcc00640 10 client.16173 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.460+0000 7fbafcc00640 1 -- 172.21.15.42:0/601182765 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f39a4c92c0 con 0x55f39a14e1c0 2024-05-27T10:22:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.460+0000 7fbafcc00640 10 client.16173 handle_command_reply: tid=0 2024-05-27T10:22:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.460+0000 7fbb207d8740 1 client.16173 shutdown 2024-05-27T10:22:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.460+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 msgr2=0x55f39a579250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.460+0000 7fbb207d8740 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a579250 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb000a9500 tx=0x7fbb000a5d40 comp rx=0 tx=0).stop 2024-05-27T10:22:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 shutdown_connections 2024-05-27T10:22:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbb207d8740 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f39a1ba1b0 0x55f39a5deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbb207d8740 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f39a1994d0 0x55f39a579250 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbb207d8740 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f39a14e1c0 0x55f39a5e2c50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbb207d8740 1 --2- 172.21.15.42:0/601182765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39a1912f0 0x55f39a578d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 >> 172.21.15.42:0/601182765 conn(0x55f39a1ae300 msgr2=0x55f39a568180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.461+0000 7fbafcc00640 0 client.16173 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.462+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 shutdown_connections 2024-05-27T10:22:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.462+0000 7fbb207d8740 1 -- 172.21.15.42:0/601182765 wait complete. 2024-05-27T10:22:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.463+0000 7fbb207d8740 20 client.16173 trim_cache size 0 max 16384 2024-05-27T10:22:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:22:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:22:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbaf80b29d0 msgr2=0x7fbaf80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbaf80b29d0 0x7fbaf80b4ee0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402a930 tx=0x7fbb040058b0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 msgr2=0x7fbb14110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 0x7fbb14110590 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d9c0 tx=0x7fbb0000de80 comp rx=0 tx=0).stop 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 shutdown_connections 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fbaf80b29d0 0x7fbaf80b4ee0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb14108010 0x7fbb14110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb141075e0 0x7fbb14110590 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 --2- 172.21.15.42:0/1458580565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb14102b90 0x7fbb14110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.466+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 >> 172.21.15.42:0/1458580565 conn(0x7fbb140782d0 msgr2=0x7fbb14105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.467+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 shutdown_connections 2024-05-27T10:22:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:39.467+0000 7fbb19800640 1 -- 172.21.15.42:0/1458580565 wait complete. 2024-05-27T10:22:39.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 -- 172.21.15.42:0/2395828867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c1045e0 msgr2=0x7f7b2c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 --2- 172.21.15.42:0/2395828867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c1049f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009990 tx=0x7f7b20009ca0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 -- 172.21.15.42:0/2395828867 shutdown_connections 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 --2- 172.21.15.42:0/2395828867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 --2- 172.21.15.42:0/2395828867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c1049f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 --2- 172.21.15.42:0/2395828867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c108f10 0x7f7b2c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.889+0000 7f7b33400700 1 -- 172.21.15.42:0/2395828867 >> 172.21.15.42:0/2395828867 conn(0x7f7b2c100030 msgr2=0x7f7b2c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.890+0000 7f7b33400700 1 -- 172.21.15.42:0/2395828867 shutdown_connections 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.890+0000 7f7b33400700 1 -- 172.21.15.42:0/2395828867 wait complete. 2024-05-27T10:22:39.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.891+0000 7f7b33400700 1 Processor -- start 2024-05-27T10:22:39.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.892+0000 7f7b33400700 1 -- start start 2024-05-27T10:22:39.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.892+0000 7f7b33400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b33400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c108f10 0x7f7b2c19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b33400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b33400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1165a0 con 0x7f7b2c109860 2024-05-27T10:22:39.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b33400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c116420 con 0x7f7b2c108f10 2024-05-27T10:22:39.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b33400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c116720 con 0x7f7b2c1045e0 2024-05-27T10:22:39.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b31400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b30a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b30a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c19cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:47376/0 (socket says 172.21.15.42:47376) 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b31400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50388/0 (socket says 172.21.15.42:50388) 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b30a00700 1 -- 172.21.15.42:0/3044365569 learned_addr learned my addr 172.21.15.42:0/3044365569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.893+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c108f10 0x7f7b2c19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.894+0000 7f7b31400700 1 -- 172.21.15.42:0/3044365569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c1045e0 msgr2=0x7f7b2c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.894+0000 7f7b31400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.894+0000 7f7b31400700 1 -- 172.21.15.42:0/3044365569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c108f10 msgr2=0x7f7b2c19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.894+0000 7f7b31400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c108f10 0x7f7b2c19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.894+0000 7f7b31400700 1 -- 172.21.15.42:0/3044365569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c1a1d90 con 0x7f7b2c109860 2024-05-27T10:22:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.894+0000 7f7b31400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1a17a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400cc50 tx=0x7f7b2400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.895+0000 7f7b29600700 1 -- 172.21.15.42:0/3044365569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24016020 con 0x7f7b2c109860 2024-05-27T10:22:39.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.895+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c1a2080 con 0x7f7b2c109860 2024-05-27T10:22:39.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.895+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1a25c0 con 0x7f7b2c109860 2024-05-27T10:22:39.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.895+0000 7f7b29600700 1 -- 172.21.15.42:0/3044365569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b24012420 con 0x7f7b2c109860 2024-05-27T10:22:39.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.895+0000 7f7b29600700 1 -- 172.21.15.42:0/3044365569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240115e0 con 0x7f7b2c109860 2024-05-27T10:22:39.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.902+0000 7f7b4202eb80 1 Processor -- start 2024-05-27T10:22:39.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.902+0000 7f7b29600700 1 -- 172.21.15.42:0/3044365569 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7b24019020 con 0x7f7b2c109860 2024-05-27T10:22:39.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.903+0000 7f7b4202eb80 1 -- start start 2024-05-27T10:22:39.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.904+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096269f0 0x5601096b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.904+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.904+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 0x5601096c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.904+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5601095e7e40 con 0x5601096b9e70 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.904+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5601095d79e0 con 0x56010953ae40 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.904+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5601095dba20 con 0x5601096269f0 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.905+0000 7f7b2be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 0x5601096c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.905+0000 7f7b2be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 0x5601096c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50404/0 (socket says 172.21.15.42:50404) 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.905+0000 7f7b2be00700 1 -- 172.21.15.42:0/1997180219 learned_addr learned my addr 172.21.15.42:0/1997180219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.905+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096269f0 0x5601096b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.905+0000 7f7b29600700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7b140b2a70 0x7f7b140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b2be00700 1 -- 172.21.15.42:0/1997180219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096269f0 msgr2=0x5601096b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096269f0 0x5601096b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b2be00700 1 -- 172.21.15.42:0/1997180219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 msgr2=0x5601096b9870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:22:39.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096b9870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b2be00700 1 -- 172.21.15.42:0/1997180219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601095a0420 con 0x5601096b9e70 2024-05-27T10:22:39.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b29600700 1 -- 172.21.15.42:0/3044365569 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7b240e2220 con 0x7f7b2c109860 2024-05-27T10:22:39.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.906+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 0x5601096c6110 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009e90 tx=0x7f7b18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19034415e70cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.907+0000 7f7b31400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7b140b2a70 0x7f7b140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.907+0000 7f7b1e000700 1 -- 172.21.15.42:0/1997180219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18004050 con 0x5601096b9e70 2024-05-27T10:22:39.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.907+0000 7f7b1e000700 1 -- 172.21.15.42:0/1997180219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b18003680 con 0x5601096b9e70 2024-05-27T10:22:39.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.907+0000 7f7b1e000700 1 -- 172.21.15.42:0/1997180219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18010690 con 0x5601096b9e70 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.908+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1997180219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 msgr2=0x5601096c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.908+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 0x5601096c6110 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009e90 tx=0x7f7b18009260 comp rx=0 tx=0).stop 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b31400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7b140b2a70 0x7f7b140b4f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400afd0 tx=0x7f7b24006040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1997180219 shutdown_connections 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096b9e70 0x5601096c6110 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096269f0 0x5601096b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1997180219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1997180219 >> 172.21.15.42:0/1997180219 conn(0x5601095d4390 msgr2=0x5601096b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.909+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1997180219 shutdown_connections 2024-05-27T10:22:39.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.910+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1997180219 wait complete. 2024-05-27T10:22:39.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.910+0000 7f7b4202eb80 1 -- start start 2024-05-27T10:22:39.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.911+0000 7f7b4202eb80 1 Processor -- start 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5601094ddfa0 con 0x5601096269f0 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560109578760 con 0x56010953ae40 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5601096c6b80 con 0x5601096b9e70 2024-05-27T10:22:39.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.912+0000 7f7b4202eb80 10 client.0 _ll_register_callbacks cb 0x560109581c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:57410/0 (socket says 172.21.15.42:57410) 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 -- 172.21.15.42:0/1931344909 learned_addr learned my addr 172.21.15.42:0/1931344909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b30a00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 -- 172.21.15.42:0/1931344909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 msgr2=0x5601096c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.913+0000 7f7b2be00700 1 -- 172.21.15.42:0/1931344909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 msgr2=0x5601096c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.914+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.914+0000 7f7b2be00700 1 -- 172.21.15.42:0/1931344909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601095278d0 con 0x56010953ae40 2024-05-27T10:22:39.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.914+0000 7f7b30a00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:39.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.914+0000 7f7b2be00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096c3600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005850 tx=0x7f7b18005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b109eee7423b282 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.914+0000 7f7b12a00700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:22:39.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.915+0000 7f7b12a00700 1 -- 172.21.15.42:0/1931344909 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18028040 con 0x56010953ae40 2024-05-27T10:22:39.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.915+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b18003680 con 0x56010953ae40 2024-05-27T10:22:39.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.915+0000 7f7b12a00700 1 -- 172.21.15.42:0/1931344909 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7b18010430 con 0x56010953ae40 2024-05-27T10:22:39.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.915+0000 7f7b12a00700 1 -- 172.21.15.42:0/1931344909 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b18010630 con 0x56010953ae40 2024-05-27T10:22:39.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.916+0000 7f7b4202eb80 10 client.35750 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:39.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.916+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601095a8060 con 0x56010953ae40 2024-05-27T10:22:39.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.916+0000 7f7b12a00700 1 -- 172.21.15.42:0/1931344909 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b18003680 con 0x56010953ae40 2024-05-27T10:22:39.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b12a00700 1 -- 172.21.15.42:0/1931344909 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7b1801f660 con 0x56010953ae40 2024-05-27T10:22:39.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b4202eb80 10 client.35750 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:39.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b4202eb80 10 client.35750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:22:39.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b4202eb80 20 client.35750 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:39.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x5601096d3980 0x5601096d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b4202eb80 4 client.35750 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:39.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.917+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601095e9d90 con 0x5601096d3980 2024-05-27T10:22:39.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.918+0000 7f7b30a00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x5601096d3980 0x5601096d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.919+0000 7f7b12a00700 10 client.35750 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:22:39.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.919+0000 7f7b30a00700 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x5601096d3980 0x5601096d5da0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bcf7cbeb6d130fc server_cookie=fb42895dad8aa30a in_seq=0 out_seq=0 2024-05-27T10:22:39.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.921+0000 7f7b12a00700 1 -- 172.21.15.42:0/1931344909 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5601095e9d90 con 0x5601096d3980 2024-05-27T10:22:39.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.921+0000 7f7b12a00700 10 client.35750 handle_command_reply: tid=0 2024-05-27T10:22:39.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.921+0000 7f7b4202eb80 1 client.35750 shutdown 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 msgr2=0x5601096c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096c3600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005850 tx=0x7f7b18005780 comp rx=0 tx=0).stop 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 shutdown_connections 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x5601096d3980 0x5601096d5da0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096c3600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/1931344909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 >> 172.21.15.42:0/1931344909 conn(0x5601095d4390 msgr2=0x5601096b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.922+0000 7f7b12a00700 0 client.35750 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:22:39.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.923+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 shutdown_connections 2024-05-27T10:22:39.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.923+0000 7f7b4202eb80 1 -- 172.21.15.42:0/1931344909 wait complete. 2024-05-27T10:22:39.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.923+0000 7f7b4202eb80 20 client.35750 trim_cache size 0 max 16384 2024-05-27T10:22:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 Processor -- start 2024-05-27T10:22:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 -- start start 2024-05-27T10:22:39.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56010953ae40 0x5601096bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5601096b9e70 0x5601096c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c00fd50 con 0x5601096269f0 2024-05-27T10:22:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c00d5e0 con 0x5601096b9e70 2024-05-27T10:22:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.950+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c00b430 con 0x56010953ae40 2024-05-27T10:22:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50428/0 (socket says 172.21.15.42:50428) 2024-05-27T10:22:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 -- 172.21.15.42:0/3701126843 learned_addr learned my addr 172.21.15.42:0/3701126843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56010953ae40 0x5601096bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5601096b9e70 0x5601096c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 -- 172.21.15.42:0/3701126843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56010953ae40 msgr2=0x5601096bee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56010953ae40 0x5601096bee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 -- 172.21.15.42:0/3701126843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5601096b9e70 msgr2=0x5601096c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.951+0000 7f7b30a00700 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5601096b9e70 0x5601096c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.952+0000 7f7b30a00700 1 -- 172.21.15.42:0/3701126843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601096b1240 con 0x5601096269f0 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.952+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56010953ae40 0x5601096bee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.952+0000 7f7b30a00700 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cfc80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b650 tx=0x7f7b20005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f61eb874dd93a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.952+0000 7f7b1d600700 1 -- 172.21.15.42:0/3701126843 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20035070 con 0x5601096269f0 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.953+0000 7f7b1d600700 1 -- 172.21.15.42:0/3701126843 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b20030dd0 con 0x5601096269f0 2024-05-27T10:22:39.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.953+0000 7f7b1d600700 1 -- 172.21.15.42:0/3701126843 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000bad0 con 0x5601096269f0 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3701126843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 msgr2=0x5601096cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cfc80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b650 tx=0x7f7b20005040 comp rx=0 tx=0).stop 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3701126843 shutdown_connections 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5601096b9e70 0x5601096c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56010953ae40 0x5601096bee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3701126843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cfc80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3701126843 >> 172.21.15.42:0/3701126843 conn(0x5601095d4390 msgr2=0x5601096c1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3701126843 shutdown_connections 2024-05-27T10:22:39.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.954+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3701126843 wait complete. 2024-05-27T10:22:39.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.955+0000 7f7b4202eb80 1 -- start start 2024-05-27T10:22:39.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.955+0000 7f7b4202eb80 1 Processor -- start 2024-05-27T10:22:39.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c00baa0 con 0x5601096269f0 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c00cf10 con 0x56010953ae40 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c00c170 con 0x5601096b9e70 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.956+0000 7f7b4202eb80 10 client.0 _ll_register_callbacks cb 0x560109581c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b2be00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50440/0 (socket says 172.21.15.42:50440) 2024-05-27T10:22:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 -- 172.21.15.42:0/3147405294 learned_addr learned my addr 172.21.15.42:0/3147405294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:39.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 -- 172.21.15.42:0/3147405294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 msgr2=0x5601096c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 -- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 msgr2=0x5601096cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:39.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b30a00700 1 -- 172.21.15.42:0/3147405294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601096c43f0 con 0x5601096269f0 2024-05-27T10:22:39.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.957+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:39.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.958+0000 7f7b30a00700 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cc670 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7b20033040 tx=0x7f7b20003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4dc0ae3813bf19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:39.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.958+0000 7f7b12a00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:39.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.958+0000 7f7b12a00700 1 -- 172.21.15.42:0/3147405294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2003e060 con 0x5601096269f0 2024-05-27T10:22:39.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.958+0000 7f7b12a00700 1 -- 172.21.15.42:0/3147405294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f7b20030940 con 0x5601096269f0 2024-05-27T10:22:39.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.959+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b20030dd0 con 0x5601096269f0 2024-05-27T10:22:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.959+0000 7f7b12a00700 1 -- 172.21.15.42:0/3147405294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b20035040 con 0x5601096269f0 2024-05-27T10:22:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.959+0000 7f7b4202eb80 10 client.16188 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.959+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56010973da80 con 0x5601096269f0 2024-05-27T10:22:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b12a00700 1 -- 172.21.15.42:0/3147405294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b20030dd0 con 0x5601096269f0 2024-05-27T10:22:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b12a00700 1 -- 172.21.15.42:0/3147405294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7b2000f460 con 0x5601096269f0 2024-05-27T10:22:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b4202eb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b4202eb80 10 client.16188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T10:22:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b4202eb80 20 client.16188 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56010973dfa0 0x5601097403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b4202eb80 4 client.16188 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:22:39.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.960+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 --> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601095e3570 con 0x56010973dfa0 2024-05-27T10:22:39.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.961+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56010973dfa0 0x5601097403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.962+0000 7f7b2be00700 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56010973dfa0 0x5601097403c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b0dbfd39b23af28 server_cookie=73b042cc232371e1 in_seq=0 out_seq=0 2024-05-27T10:22:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.963+0000 7f7b12a00700 10 client.16188 ms_handle_connect on v2:172.21.15.104:6834/3501163811 2024-05-27T10:22:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.963+0000 7f7b12a00700 1 -- 172.21.15.42:0/3147405294 <== mds.2 v2:172.21.15.104:6834/3501163811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601095e3570 con 0x56010973dfa0 2024-05-27T10:22:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.963+0000 7f7b12a00700 10 client.16188 handle_command_reply: tid=0 2024-05-27T10:22:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.963+0000 7f7b4202eb80 1 client.16188 shutdown 2024-05-27T10:22:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 msgr2=0x5601096cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cc670 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7b20033040 tx=0x7f7b20003e50 comp rx=0 tx=0).stop 2024-05-27T10:22:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 shutdown_connections 2024-05-27T10:22:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:6834/3501163811,v1:172.21.15.104:6835/3501163811] conn(0x56010973dfa0 0x5601097403c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5601096b9e70 0x5601096c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b12a00700 0 client.16188 ms_handle_reset on v2:172.21.15.104:6834/3501163811 2024-05-27T10:22:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56010953ae40 0x5601096cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 --2- 172.21.15.42:0/3147405294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601096269f0 0x5601096cc670 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.964+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 >> 172.21.15.42:0/3147405294 conn(0x5601095d4390 msgr2=0x5601096c2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.966 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:22:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.965+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 shutdown_connections 2024-05-27T10:22:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.965+0000 7f7b4202eb80 1 -- 172.21.15.42:0/3147405294 wait complete. 2024-05-27T10:22:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.965+0000 7f7b4202eb80 20 client.16188 trim_cache size 0 max 16384 2024-05-27T10:22:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.966+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7b140b2a70 msgr2=0x7f7b140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.966+0000 7f7b33400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7b140b2a70 0x7f7b140b4f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400afd0 tx=0x7f7b24006040 comp rx=0 tx=0).stop 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.966+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 msgr2=0x7f7b2c1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.966+0000 7f7b33400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1a17a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400cc50 tx=0x7f7b2400cf60 comp rx=0 tx=0).stop 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 shutdown_connections 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f7b140b2a70 0x7f7b140b4f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b2c109860 0x7f7b2c1a17a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b2c1045e0 0x7f7b2c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 --2- 172.21.15.42:0/3044365569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b2c108f10 0x7f7b2c19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 >> 172.21.15.42:0/3044365569 conn(0x7f7b2c100030 msgr2=0x7f7b2c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 shutdown_connections 2024-05-27T10:22:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:39.967+0000 7f7b33400700 1 -- 172.21.15.42:0/3044365569 wait complete. 2024-05-27T10:22:40.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:39 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2510977647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:40.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:39 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2510977647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:40.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:39 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2510977647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:22:40.331 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-27T10:22:40.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-27T10:22:41.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:40 smithi042 ceph-mon[31661]: pgmap v121: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-05-27T10:22:41.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:40 smithi104 ceph-mon[34812]: pgmap v121: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-05-27T10:22:41.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:40 smithi143 ceph-mon[35545]: pgmap v121: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-05-27T10:22:41.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T10:22:41.457 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:42.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:42 smithi042 ceph-mon[31661]: pgmap v122: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 706 KiB/s wr, 56 op/s 2024-05-27T10:22:42.966 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:43.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:42 smithi104 ceph-mon[34812]: pgmap v122: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 706 KiB/s wr, 56 op/s 2024-05-27T10:22:43.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:42 smithi143 ceph-mon[35545]: pgmap v122: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 706 KiB/s wr, 56 op/s 2024-05-27T10:22:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.939+0000 7fcd5c800700 1 -- 172.21.15.42:0/2414466205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd54078630 msgr2=0x7fcd54114ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:45.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.939+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2414466205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd54078630 0x7fcd54114ca0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcd440098f0 tx=0x7fcd44009c00 comp rx=0 tx=0).stop 2024-05-27T10:22:45.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 -- 172.21.15.42:0/2414466205 shutdown_connections 2024-05-27T10:22:45.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2414466205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd541151e0 0x7fcd54117600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2414466205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 0x7fcd540780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2414466205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd54078630 0x7fcd54114ca0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 -- 172.21.15.42:0/2414466205 >> 172.21.15.42:0/2414466205 conn(0x7fcd54102080 msgr2=0x7fcd541044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:45.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 -- 172.21.15.42:0/2414466205 shutdown_connections 2024-05-27T10:22:45.019 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:44 smithi042 ceph-mon[31661]: pgmap v123: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 706 KiB/s wr, 56 op/s 2024-05-27T10:22:45.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.940+0000 7fcd5c800700 1 -- 172.21.15.42:0/2414466205 wait complete. 2024-05-27T10:22:45.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.941+0000 7fcd5c800700 1 Processor -- start 2024-05-27T10:22:45.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.942+0000 7fcd5c800700 1 -- start start 2024-05-27T10:22:45.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.942+0000 7fcd5c800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 0x7fcd54075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.942+0000 7fcd5c800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd54078630 0x7fcd5406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.942+0000 7fcd5c800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd541151e0 0x7fcd5406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd5c800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5411a9a0 con 0x7fcd541151e0 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd5c800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5411a820 con 0x7fcd54077ce0 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd5c800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5411ab20 con 0x7fcd54078630 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd59e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 0x7fcd54075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd59e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 0x7fcd54075390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:46758/0 (socket says 172.21.15.42:46758) 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd59e00700 1 -- 172.21.15.42:0/2677554569 learned_addr learned my addr 172.21.15.42:0/2677554569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd59400700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd54078630 0x7fcd5406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.943+0000 7fcd5a800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd541151e0 0x7fcd5406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.944+0000 7fcd59400700 1 -- 172.21.15.42:0/2677554569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 msgr2=0x7fcd54075390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.944+0000 7fcd59400700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 0x7fcd54075390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.944+0000 7fcd59400700 1 -- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd541151e0 msgr2=0x7fcd5406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.944+0000 7fcd59400700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd541151e0 0x7fcd5406eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.944+0000 7fcd59400700 1 -- 172.21.15.42:0/2677554569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5406f2c0 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.944+0000 7fcd59400700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd54078630 0x7fcd5406e4c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.945+0000 7fcd4ea00700 1 -- 172.21.15.42:0/2677554569 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44024020 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.945+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd54072f80 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.945+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd54073480 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.945+0000 7fcd4ea00700 1 -- 172.21.15.42:0/2677554569 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd440094c0 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.945+0000 7fcd4ea00700 1 -- 172.21.15.42:0/2677554569 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440052a0 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.951+0000 7fcd4ea00700 1 -- 172.21.15.42:0/2677554569 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcd44027070 con 0x7fcd54078630 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.951+0000 7fcd6b370b80 1 Processor -- start 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.952+0000 7fcd6b370b80 1 -- start start 2024-05-27T10:22:45.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.952+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc013fd0 0x55b5fc198e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.953+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc192f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.953+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.953+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc0c0fd0 con 0x55b5fc013fd0 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.953+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc0b0b70 con 0x55b5fc199380 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.953+0000 7fcd4ea00700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcd3c0b2b30 0x7fcd3c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.953+0000 7fcd4ea00700 1 -- 172.21.15.42:0/2677554569 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcd440e2f50 con 0x7fcd54078630 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.954+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc0b4bb0 con 0x55b5fc1934d0 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.954+0000 7fcd5a800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcd3c0b2b30 0x7fcd3c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.954+0000 7fcd59400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.954+0000 7fcd59400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:46632/0 (socket says 172.21.15.42:46632) 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.954+0000 7fcd59400700 1 -- 172.21.15.42:0/1857733634 learned_addr learned my addr 172.21.15.42:0/1857733634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.954+0000 7fcd59e00700 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc192f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd5a800700 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc013fd0 0x55b5fc198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd59400700 1 -- 172.21.15.42:0/1857733634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 msgr2=0x55b5fc192f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd59400700 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc192f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd59400700 1 -- 172.21.15.42:0/1857733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc013fd0 msgr2=0x55b5fc198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd59400700 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc013fd0 0x55b5fc198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd59400700 1 -- 172.21.15.42:0/1857733634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fc0795b0 con 0x55b5fc1934d0 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.955+0000 7fcd5a800700 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc013fd0 0x55b5fc198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:45.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.956+0000 7fcd59400700 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1938e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd440a6330 tx=0x7fcd44009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4a18391fd071863 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:45.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.956+0000 7fcd5a800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcd3c0b2b30 0x7fcd3c0b5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55b5fc194180 tx=0x7fcd500099e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:45.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.956+0000 7fcd43400700 1 -- 172.21.15.42:0/1857733634 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440e0b20 con 0x55b5fc1934d0 2024-05-27T10:22:45.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.957+0000 7fcd43400700 1 -- 172.21.15.42:0/1857733634 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd440aa2b0 con 0x55b5fc1934d0 2024-05-27T10:22:45.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 -- 172.21.15.42:0/1857733634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 msgr2=0x55b5fc1938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:45.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1938e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd440a6330 tx=0x7fcd44009c00 comp rx=0 tx=0).stop 2024-05-27T10:22:45.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 -- 172.21.15.42:0/1857733634 shutdown_connections 2024-05-27T10:22:45.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1938e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc013fd0 0x55b5fc198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/1857733634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc192f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.958+0000 7fcd6b370b80 1 -- 172.21.15.42:0/1857733634 >> 172.21.15.42:0/1857733634 conn(0x55b5fc074150 msgr2=0x55b5fc18e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.959+0000 7fcd6b370b80 1 -- 172.21.15.42:0/1857733634 shutdown_connections 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.959+0000 7fcd6b370b80 1 -- 172.21.15.42:0/1857733634 wait complete. 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.961+0000 7fcd6b370b80 1 -- start start 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.961+0000 7fcd6b370b80 1 Processor -- start 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc199380 0x55b5fc19d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b5fbfb7130 con 0x55b5fc1934d0 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc0518f0 con 0x55b5fc013fd0 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc19fc10 con 0x55b5fc199380 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.963+0000 7fcd6b370b80 10 client.0 _ll_register_callbacks cb 0x55b5fc05ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55830/0 (socket says 172.21.15.42:55830) 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 -- 172.21.15.42:0/3730265222 learned_addr learned my addr 172.21.15.42:0/3730265222 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd5a800700 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc199380 0x55b5fc19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 -- 172.21.15.42:0/3730265222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc199380 msgr2=0x55b5fc19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc199380 0x55b5fc19d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 -- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 msgr2=0x55b5fc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.964+0000 7fcd59400700 1 -- 172.21.15.42:0/3730265222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fc000a60 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.965+0000 7fcd59400700 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19c9a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcd440b2ff0 tx=0x7fcd4400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb703b859ce99431 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.965+0000 7fcd37e00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.966+0000 7fcd37e00700 1 -- 172.21.15.42:0/3730265222 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440c3070 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.966+0000 7fcd37e00700 1 -- 172.21.15.42:0/3730265222 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcd440a9720 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.966+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd440aa2b0 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.966+0000 7fcd37e00700 1 -- 172.21.15.42:0/3730265222 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd440e0b20 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.966+0000 7fcd6b370b80 10 client.16197 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.966+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5fc0811f0 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd37e00700 1 -- 172.21.15.42:0/3730265222 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd440aa2b0 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd37e00700 1 -- 172.21.15.42:0/3730265222 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fcd4401f070 con 0x55b5fc1934d0 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd6b370b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:45.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd6b370b80 10 client.16197 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd6b370b80 20 client.16197 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc1ac7c0 0x55b5fc1aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd6b370b80 4 client.16197 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.967+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5fc0c2f20 con 0x55b5fc1ac7c0 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.968+0000 7fcd59e00700 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc1ac7c0 0x55b5fc1aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.969+0000 7fcd59e00700 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc1ac7c0 0x55b5fc1aebe0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37da25a34652485a server_cookie=7116a721c8147213 in_seq=0 out_seq=0 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.969+0000 7fcd37e00700 10 client.16197 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.971+0000 7fcd37e00700 1 -- 172.21.15.42:0/3730265222 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b5fc0c2f20 con 0x55b5fc1ac7c0 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.971+0000 7fcd37e00700 10 client.16197 handle_command_reply: tid=0 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.971+0000 7fcd6b370b80 1 client.16197 shutdown 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 msgr2=0x55b5fc19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19c9a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcd440b2ff0 tx=0x7fcd4400bed0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 shutdown_connections 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc1ac7c0 0x55b5fc1aebe0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc199380 0x55b5fc19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3730265222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19c9a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd37e00700 0 client.16197 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.972+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 >> 172.21.15.42:0/3730265222 conn(0x55b5fc074150 msgr2=0x55b5fc191da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.973+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 shutdown_connections 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.973+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3730265222 wait complete. 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:44.974+0000 7fcd6b370b80 20 client.16197 trim_cache size 0 max 16384 2024-05-27T10:22:45.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.008+0000 7fcd6b370b80 1 Processor -- start 2024-05-27T10:22:45.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 -- start start 2024-05-27T10:22:45.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800fe70 con 0x55b5fc1934d0 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800d700 con 0x55b5fc199380 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.009+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800b550 con 0x55b5fc013fd0 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.010+0000 7fcd59400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.010+0000 7fcd5a800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.010+0000 7fcd5a800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:46774/0 (socket says 172.21.15.42:46774) 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.010+0000 7fcd5a800700 1 -- 172.21.15.42:0/3501640900 learned_addr learned my addr 172.21.15.42:0/3501640900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.010+0000 7fcd59e00700 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.011+0000 7fcd59400700 1 -- 172.21.15.42:0/3501640900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc013fd0 msgr2=0x55b5fc1a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.011+0000 7fcd59400700 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.011+0000 7fcd59400700 1 -- 172.21.15.42:0/3501640900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 msgr2=0x55b5fc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.011+0000 7fcd59400700 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.011+0000 7fcd59400700 1 -- 172.21.15.42:0/3501640900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fc19f240 con 0x55b5fc1934d0 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.011+0000 7fcd5a800700 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.012+0000 7fcd59400700 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19ed00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcd440aa240 tx=0x7fcd4400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a89e7409d5e3b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.012+0000 7fcd42a00700 1 -- 172.21.15.42:0/3501640900 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440b3080 con 0x55b5fc1934d0 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.013+0000 7fcd42a00700 1 -- 172.21.15.42:0/3501640900 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd440ae660 con 0x55b5fc1934d0 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.013+0000 7fcd42a00700 1 -- 172.21.15.42:0/3501640900 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440a91e0 con 0x55b5fc1934d0 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3501640900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 msgr2=0x55b5fc19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19ed00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcd440aa240 tx=0x7fcd4400bb90 comp rx=0 tx=0).stop 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3501640900 shutdown_connections 2024-05-27T10:22:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc199380 0x55b5fc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/3501640900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc1934d0 0x55b5fc19ed00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.014+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3501640900 >> 172.21.15.42:0/3501640900 conn(0x55b5fc074150 msgr2=0x55b5fc197a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.015+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3501640900 shutdown_connections 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.015+0000 7fcd6b370b80 1 -- 172.21.15.42:0/3501640900 wait complete. 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.015+0000 7fcd6b370b80 1 -- start start 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.016+0000 7fcd6b370b80 1 Processor -- start 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.017+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.017+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.017+0000 7fcd6b370b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc199380 0x55b5fc1a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd59e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800bbc0 con 0x55b5fc199380 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800d030 con 0x55b5fc013fd0 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd6b370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800c290 con 0x55b5fc1934d0 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd6b370b80 10 client.0 _ll_register_callbacks cb 0x55b5fc05ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd5a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc199380 0x55b5fc1a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd59e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:46788/0 (socket says 172.21.15.42:46788) 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd59e00700 1 -- 172.21.15.42:0/2981871797 learned_addr learned my addr 172.21.15.42:0/2981871797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.018+0000 7fcd59400700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.019+0000 7fcd59400700 1 -- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 msgr2=0x55b5fc1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.019+0000 7fcd59400700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.019+0000 7fcd59400700 1 -- 172.21.15.42:0/2981871797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc199380 msgr2=0x55b5fc1a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.019+0000 7fcd59400700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc199380 0x55b5fc1a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.019+0000 7fcd59400700 1 -- 172.21.15.42:0/2981871797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fc1a2b10 con 0x55b5fc1934d0 2024-05-27T10:22:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.020+0000 7fcd59e00700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.020+0000 7fcd59400700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1a1ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd440aa240 tx=0x7fcd44009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6885eec8c530d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.020+0000 7fcd37e00700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.021+0000 7fcd37e00700 1 -- 172.21.15.42:0/2981871797 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440ae120 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.021+0000 7fcd37e00700 1 -- 172.21.15.42:0/2981871797 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcd440ae2c0 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.021+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd440ae660 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.022+0000 7fcd37e00700 1 -- 172.21.15.42:0/2981871797 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd440c3070 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.022+0000 7fcd6b370b80 10 client.25816 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.022+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5fc1a2b10 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.022+0000 7fcd37e00700 1 -- 172.21.15.42:0/2981871797 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd440ae660 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd37e00700 1 -- 172.21.15.42:0/2981871797 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fcd4401f060 con 0x55b5fc1934d0 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd6b370b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd6b370b80 10 client.25816 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd6b370b80 20 client.25816 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc2183f0 0x55b5fc21a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd6b370b80 4 client.25816 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-27T10:22:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.023+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55b5fc0bc700 con 0x55b5fc2183f0 2024-05-27T10:22:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.024+0000 7fcd59e00700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc2183f0 0x55b5fc21a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.025+0000 7fcd59e00700 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc2183f0 0x55b5fc21a810 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5291336e2b4b8316 server_cookie=d50af85eb247d37b in_seq=0 out_seq=0 2024-05-27T10:22:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:45.025+0000 7fcd37e00700 10 client.25816 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:45.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:44 smithi104 ceph-mon[34812]: pgmap v123: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 706 KiB/s wr, 56 op/s 2024-05-27T10:22:45.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:44 smithi143 ceph-mon[35545]: pgmap v123: 97 pgs: 97 active+clean; 29 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 706 KiB/s wr, 56 op/s 2024-05-27T10:22:46.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 -- 172.21.15.42:0/1651959313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d1010e8d0 msgr2=0x7f6d10110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 --2- 172.21.15.42:0/1651959313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d1010e8d0 0x7f6d10110db0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b0a0 tx=0x7f6d0800b3b0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 -- 172.21.15.42:0/1651959313 shutdown_connections 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 --2- 172.21.15.42:0/1651959313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d1010e8d0 0x7f6d10110db0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 --2- 172.21.15.42:0/1651959313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d101054f0 0x7f6d10105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 --2- 172.21.15.42:0/1651959313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 0x7f6d1010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 -- 172.21.15.42:0/1651959313 >> 172.21.15.42:0/1651959313 conn(0x7f6d100fedb0 msgr2=0x7f6d101011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 -- 172.21.15.42:0/1651959313 shutdown_connections 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.210+0000 7f6d15000700 1 -- 172.21.15.42:0/1651959313 wait complete. 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.211+0000 7f6d15000700 1 Processor -- start 2024-05-27T10:22:46.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 -- start start 2024-05-27T10:22:46.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 0x7f6d1019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 0x7f6d1019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1010e8d0 0x7f6d101a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10114330 con 0x7f6d101054f0 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101141b0 con 0x7f6d10105e40 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.212+0000 7f6d15000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101144b0 con 0x7f6d1010e8d0 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 0x7f6d1019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 0x7f6d1019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55864/0 (socket says 172.21.15.42:55864) 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 -- 172.21.15.42:0/1327941417 learned_addr learned my addr 172.21.15.42:0/1327941417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0d400700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 0x7f6d1019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0e800700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1010e8d0 0x7f6d101a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 -- 172.21.15.42:0/1327941417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1010e8d0 msgr2=0x7f6d101a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1010e8d0 0x7f6d101a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 -- 172.21.15.42:0/1327941417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 msgr2=0x7f6d1019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 0x7f6d1019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0de00700 1 -- 172.21.15.42:0/1327941417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d101a1db0 con 0x7f6d101054f0 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.213+0000 7f6d0d400700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 0x7f6d1019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.214+0000 7f6d0e800700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1010e8d0 0x7f6d101a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.214+0000 7f6d0de00700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 0x7f6d1019cee0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000d950 tx=0x7f6d0000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.214+0000 7f6d06a00700 1 -- 172.21.15.42:0/1327941417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00016070 con 0x7f6d101054f0 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.214+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101a2040 con 0x7f6d101054f0 2024-05-27T10:22:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.214+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101a99c0 con 0x7f6d101054f0 2024-05-27T10:22:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.215+0000 7f6d06a00700 1 -- 172.21.15.42:0/1327941417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d00010460 con 0x7f6d101054f0 2024-05-27T10:22:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.215+0000 7f6d06a00700 1 -- 172.21.15.42:0/1327941417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000f620 con 0x7f6d101054f0 2024-05-27T10:22:46.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.221+0000 7f6d06a00700 1 -- 172.21.15.42:0/1327941417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6d00019070 con 0x7f6d101054f0 2024-05-27T10:22:46.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.221+0000 7f6d23c35b80 1 Processor -- start 2024-05-27T10:22:46.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 -- start start 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e4f63e40 0x5654e50e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e50e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e2db0 0x5654e50e31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5654e5010e40 con 0x5654e50e8d30 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654e50009e0 con 0x5654e4f63e40 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.222+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654e5004a20 con 0x5654e50e2db0 2024-05-27T10:22:46.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e50e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e50e2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55866/0 (socket says 172.21.15.42:55866) 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 -- 172.21.15.42:0/2314000483 learned_addr learned my addr 172.21.15.42:0/2314000483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0e800700 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e4f63e40 0x5654e50e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0d400700 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e2db0 0x5654e50e31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 -- 172.21.15.42:0/2314000483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e2db0 msgr2=0x5654e50e31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e2db0 0x5654e50e31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 -- 172.21.15.42:0/2314000483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e4f63e40 msgr2=0x5654e50e87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e4f63e40 0x5654e50e87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.223+0000 7f6d0de00700 1 -- 172.21.15.42:0/2314000483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e4fc9420 con 0x5654e50e8d30 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.224+0000 7f6d0de00700 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e50e2870 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6d000df700 tx=0x7f6d000ea540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe2cbace5c91b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:46.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.224+0000 7f6d06a00700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6cf40b2a70 0x7f6cf40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.224+0000 7f6d06a00700 1 -- 172.21.15.42:0/1327941417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6d000e2080 con 0x7f6d101054f0 2024-05-27T10:22:46.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.224+0000 7f6cfb400700 1 -- 172.21.15.42:0/2314000483 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00117070 con 0x5654e50e8d30 2024-05-27T10:22:46.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.224+0000 7f6cfb400700 1 -- 172.21.15.42:0/2314000483 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d000a8e60 con 0x5654e50e8d30 2024-05-27T10:22:46.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.224+0000 7f6cfb400700 1 -- 172.21.15.42:0/2314000483 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d000a9120 con 0x5654e50e8d30 2024-05-27T10:22:46.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.225+0000 7f6d0e800700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6cf40b2a70 0x7f6cf40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.226+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2314000483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 msgr2=0x5654e50e2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.226+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e50e2870 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6d000df700 tx=0x7f6d000ea540 comp rx=0 tx=0).stop 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d0e800700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6cf40b2a70 0x7f6cf40b4f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009900 tx=0x7f6d08009ca0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2314000483 shutdown_connections 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e2db0 0x5654e50e31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e4f63e40 0x5654e50e87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2314000483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e50e2870 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2314000483 >> 172.21.15.42:0/2314000483 conn(0x5654e4fc3fc0 msgr2=0x5654e50dd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:46.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2314000483 shutdown_connections 2024-05-27T10:22:46.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.227+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2314000483 wait complete. 2024-05-27T10:22:46.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.228+0000 7f6d23c35b80 1 -- start start 2024-05-27T10:22:46.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.228+0000 7f6d23c35b80 1 Processor -- start 2024-05-27T10:22:46.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 0x5654e50eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e8d30 0x5654e50ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5654e4f06fa0 con 0x5654e50e2db0 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654e4fa1760 con 0x5654e50e8d30 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654e50ef740 con 0x5654e4f63e40 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d23c35b80 10 client.0 _ll_register_callbacks cb 0x5654e4faac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d0d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 0x5654e50eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d0d400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 0x5654e50eca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55874/0 (socket says 172.21.15.42:55874) 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.229+0000 7f6d0d400700 1 -- 172.21.15.42:0/2014775609 learned_addr learned my addr 172.21.15.42:0/2014775609 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0e800700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e8d30 0x5654e50ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0de00700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0d400700 1 -- 172.21.15.42:0/2014775609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 msgr2=0x5654e50ec4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0d400700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50ec4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0d400700 1 -- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e8d30 msgr2=0x5654e50ecf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0d400700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e8d30 0x5654e50ecf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.230+0000 7f6d0d400700 1 -- 172.21.15.42:0/2014775609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e4f508d0 con 0x5654e50e2db0 2024-05-27T10:22:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.231+0000 7f6d0d400700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 0x5654e50eca30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc003c90 tx=0x7f6cfc0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bf6c8ecbe5f4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:46.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.231+0000 7f6cefe00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:46.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.231+0000 7f6cefe00700 1 -- 172.21.15.42:0/2014775609 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc011cb0 con 0x5654e50e2db0 2024-05-27T10:22:46.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.232+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d000a8e60 con 0x5654e50e2db0 2024-05-27T10:22:46.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.232+0000 7f6cefe00700 1 -- 172.21.15.42:0/2014775609 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6cfc012450 con 0x5654e50e2db0 2024-05-27T10:22:46.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.232+0000 7f6cefe00700 1 -- 172.21.15.42:0/2014775609 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc011e50 con 0x5654e50e2db0 2024-05-27T10:22:46.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.233+0000 7f6d23c35b80 10 client.16215 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:46.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.233+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654e4fd1060 con 0x5654e50e2db0 2024-05-27T10:22:46.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.233+0000 7f6cefe00700 1 -- 172.21.15.42:0/2014775609 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d000a8e60 con 0x5654e50e2db0 2024-05-27T10:22:46.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.234+0000 7f6cefe00700 1 -- 172.21.15.42:0/2014775609 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6cfc01b870 con 0x5654e50e2db0 2024-05-27T10:22:46.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d23c35b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:46.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d23c35b80 10 client.16215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:22:46.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d23c35b80 20 client.16215 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:46.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e50fc720 0x5654e50feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d23c35b80 4 client.16215 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:46.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654e5012d90 con 0x5654e50fc720 2024-05-27T10:22:46.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.235+0000 7f6d0de00700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e50fc720 0x5654e50feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.236+0000 7f6d0de00700 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e50fc720 0x5654e50feb40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5a85e242adaab94 server_cookie=2365be3acbc19404 in_seq=0 out_seq=0 2024-05-27T10:22:46.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.236+0000 7f6cefe00700 10 client.16215 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:22:46.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.246+0000 7f6cefe00700 1 -- 172.21.15.42:0/2014775609 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654e5012d90 con 0x5654e50fc720 2024-05-27T10:22:46.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.246+0000 7f6cefe00700 10 client.16215 handle_command_reply: tid=0 2024-05-27T10:22:46.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 client.16215 shutdown 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 msgr2=0x5654e50eca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 0x5654e50eca30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc003c90 tx=0x7f6cfc0113e0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 shutdown_connections 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e50fc720 0x5654e50feb40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e8d30 0x5654e50ecf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/2014775609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e2db0 0x5654e50eca30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 >> 172.21.15.42:0/2014775609 conn(0x5654e4fc3fc0 msgr2=0x5654e50dd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6cefe00700 0 client.16215 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:22:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 shutdown_connections 2024-05-27T10:22:46.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.247+0000 7f6d23c35b80 1 -- 172.21.15.42:0/2014775609 wait complete. 2024-05-27T10:22:46.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.248+0000 7f6d23c35b80 20 client.16215 trim_cache size 0 max 16384 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.269+0000 7f6d23c35b80 1 Processor -- start 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.269+0000 7f6d23c35b80 1 -- start start 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 0x5654e50e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e8d30 0x5654e50e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf000fd00 con 0x5654e4f63e40 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf000d590 con 0x5654e50e2db0 2024-05-27T10:22:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf000b3e0 con 0x5654e50e8d30 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 0x5654e50e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 0x5654e50e2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55876/0 (socket says 172.21.15.42:55876) 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d0de00700 1 -- 172.21.15.42:0/3388822219 learned_addr learned my addr 172.21.15.42:0/3388822219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d0d400700 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.270+0000 7f6d0e800700 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e8d30 0x5654e50e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.271+0000 7f6d0de00700 1 -- 172.21.15.42:0/3388822219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e8d30 msgr2=0x5654e50e9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.271+0000 7f6d0de00700 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e8d30 0x5654e50e9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.271+0000 7f6d0de00700 1 -- 172.21.15.42:0/3388822219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 msgr2=0x5654e50e6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.271+0000 7f6d0de00700 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50e6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.271+0000 7f6d0de00700 1 -- 172.21.15.42:0/3388822219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e50fa150 con 0x5654e4f63e40 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.271+0000 7f6d0de00700 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 0x5654e50e2870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d000a53d0 tx=0x7f6d000ea940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a418611a1ed0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:46.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.272+0000 7f6cfaa00700 1 -- 172.21.15.42:0/3388822219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00020070 con 0x5654e4f63e40 2024-05-27T10:22:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.272+0000 7f6cfaa00700 1 -- 172.21.15.42:0/3388822219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d000a8e60 con 0x5654e4f63e40 2024-05-27T10:22:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.272+0000 7f6cfaa00700 1 -- 172.21.15.42:0/3388822219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d000a8ac0 con 0x5654e4f63e40 2024-05-27T10:22:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3388822219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 msgr2=0x5654e50e2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 0x5654e50e2870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d000a53d0 tx=0x7f6d000ea940 comp rx=0 tx=0).stop 2024-05-27T10:22:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3388822219 shutdown_connections 2024-05-27T10:22:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e50e8d30 0x5654e50e9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e4f63e40 0x5654e50e2870 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3388822219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50e6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3388822219 >> 172.21.15.42:0/3388822219 conn(0x5654e4fc3fc0 msgr2=0x5654e50e7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3388822219 shutdown_connections 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.273+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3388822219 wait complete. 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.274+0000 7f6d23c35b80 1 -- start start 2024-05-27T10:22:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.274+0000 7f6d23c35b80 1 Processor -- start 2024-05-27T10:22:46.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e51045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf000ba50 con 0x5654e50e8d30 2024-05-27T10:22:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf000cec0 con 0x5654e50e2db0 2024-05-27T10:22:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf000c120 con 0x5654e4f63e40 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d23c35b80 10 client.0 _ll_register_callbacks cb 0x5654e4faac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d0de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:46696/0 (socket says 172.21.15.42:46696) 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.275+0000 7f6d0de00700 1 -- 172.21.15.42:0/3493443001 learned_addr learned my addr 172.21.15.42:0/3493443001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0e800700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e51045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0de00700 1 -- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 msgr2=0x5654e50f5160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0de00700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50f5160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0de00700 1 -- 172.21.15.42:0/3493443001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 msgr2=0x5654e51045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0de00700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e51045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0de00700 1 -- 172.21.15.42:0/3493443001 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e50ed2e0 con 0x5654e4f63e40 2024-05-27T10:22:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.276+0000 7f6d0e800700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e51045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:46.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.277+0000 7f6d0de00700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50f4c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d000ea1b0 tx=0x7f6d000df8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e14b01de81af53d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:46.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.277+0000 7f6cefe00700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:22:46.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.277+0000 7f6cefe00700 1 -- 172.21.15.42:0/3493443001 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00020070 con 0x5654e4f63e40 2024-05-27T10:22:46.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.277+0000 7f6cefe00700 1 -- 172.21.15.42:0/3493443001 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6d000a55e0 con 0x5654e4f63e40 2024-05-27T10:22:46.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.277+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d000a8e60 con 0x5654e4f63e40 2024-05-27T10:22:46.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.278+0000 7f6cefe00700 1 -- 172.21.15.42:0/3493443001 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d000a8ca0 con 0x5654e4f63e40 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.278+0000 7f6d23c35b80 10 client.25831 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.278+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654e5105e20 con 0x5654e4f63e40 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.278+0000 7f6cefe00700 1 -- 172.21.15.42:0/3493443001 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d000a8e60 con 0x5654e4f63e40 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6cefe00700 1 -- 172.21.15.42:0/3493443001 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6d000dfa40 con 0x5654e4f63e40 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6d23c35b80 10 client.25831 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6d23c35b80 10 client.25831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6d23c35b80 20 client.25831 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e51674d0 0x5654e51698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6d23c35b80 4 client.25831 mds_command: new command op to 34466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:22:46.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.279+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 --> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654e500c570 con 0x5654e51674d0 2024-05-27T10:22:46.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.280+0000 7f6d0d400700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e51674d0 0x5654e51698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:46.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.281+0000 7f6d0d400700 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e51674d0 0x5654e51698f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=812fe921dc27d83d server_cookie=8455d45aac7c04f2 in_seq=0 out_seq=0 2024-05-27T10:22:46.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.281+0000 7f6cefe00700 10 client.25831 ms_handle_connect on v2:172.21.15.104:6836/219118322 2024-05-27T10:22:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.292+0000 7f6cefe00700 1 -- 172.21.15.42:0/3493443001 <== mds.1 v2:172.21.15.104:6836/219118322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6318 (crc 0 0 0) 0x5654e500c570 con 0x5654e51674d0 2024-05-27T10:22:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.292+0000 7f6cefe00700 10 client.25831 handle_command_reply: tid=0 2024-05-27T10:22:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.292+0000 7f6d23c35b80 1 client.25831 shutdown 2024-05-27T10:22:46.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.292+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 msgr2=0x5654e50f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.292+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50f4c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d000ea1b0 tx=0x7f6d000df8c0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 shutdown_connections 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:6836/219118322,v1:172.21.15.104:6837/219118322] conn(0x5654e51674d0 0x5654e51698f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654e50e8d30 0x5654e51045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654e4f63e40 0x5654e50f4c20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 --2- 172.21.15.42:0/3493443001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654e50e2db0 0x5654e50f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 >> 172.21.15.42:0/3493443001 conn(0x5654e4fc3fc0 msgr2=0x5654e50e7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6cefe00700 0 client.25831 ms_handle_reset on v2:172.21.15.104:6836/219118322 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 shutdown_connections 2024-05-27T10:22:46.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 1 -- 172.21.15.42:0/3493443001 wait complete. 2024-05-27T10:22:46.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.293+0000 7f6d23c35b80 20 client.25831 trim_cache size 0 max 16384 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 133349416, 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628316, 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/compile" 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 247677524, 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628301, 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.c" 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 464415667, 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628293, 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/blogbench.8" 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 495209833, 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628314, 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/helpers.c" 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 674972773, 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628305, 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h" 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 815003069, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628292, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man" 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 834628144, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628298, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.h" 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1107507586, 2024-05-27T10:22:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628306, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.am" 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1231616604, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628317, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/depcomp" 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1381291436, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628299, 2024-05-27T10:22:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.c" 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1490116186, 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628303, 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench_p.h" 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1501016016, 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628315, 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/NEWS" 2024-05-27T10:22:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1652420151, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628302, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.h" 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1728253692, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628313, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/writer.c" 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 1897883745, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628304, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c" 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2365897189, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628312, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/reader.c" 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2486120528, 2024-05-27T10:22:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628296, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src" 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2773772391, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628297, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.c" 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2925647389, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628318, 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/aclocal.m4" 2024-05-27T10:22:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2980271907, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628310, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/commenter.c" 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2996849560, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628295, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.in" 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3015229518, 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628309, 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/globals.h" 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3174560903, 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628308, 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/process.c" 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3755556396, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628300, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.h" 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3829600169, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628294, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.am" 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3889680779, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628307, 2024-05-27T10:22:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.in" 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3951018840, 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628311, 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/rewriter.c" 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3959197956, 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628290, 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4" 2024-05-27T10:22:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "damage_type": "backtrace", 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4031416416, 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 1099511628291, 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/acx_pthread.m4" 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6cf40b2a70 msgr2=0x7f6cf40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6cf40b2a70 0x7f6cf40b4f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009900 tx=0x7f6d08009ca0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 msgr2=0x7f6d1019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:46.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 0x7f6d1019cee0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000d950 tx=0x7f6d0000dd10 comp rx=0 tx=0).stop 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 shutdown_connections 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6cf40b2a70 0x7f6cf40b4f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1010e8d0 0x7f6d101a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d101054f0 0x7f6d1019cee0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 --2- 172.21.15.42:0/1327941417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d10105e40 0x7f6d1019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 >> 172.21.15.42:0/1327941417 conn(0x7f6d100fedb0 msgr2=0x7f6d101031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.294+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 shutdown_connections 2024-05-27T10:22:46.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:46.295+0000 7f6d15000700 1 -- 172.21.15.42:0/1327941417 wait complete. 2024-05-27T10:22:47.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:46 smithi042 ceph-mon[31661]: pgmap v124: 97 pgs: 97 active+clean; 38 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-27T10:22:47.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:46 smithi104 ceph-mon[34812]: pgmap v124: 97 pgs: 97 active+clean; 38 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-27T10:22:47.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:46 smithi143 ceph-mon[35545]: pgmap v124: 97 pgs: 97 active+clean; 38 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-05-27T10:22:47.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T10:22:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:22:48.718 INFO:tasks.workunit.client.0.smithi042.stdout: 5 4857 117 3144 170 1440 173 2024-05-27T10:22:48.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.953+0000 7fcd37e00700 1 -- 172.21.15.42:0/2981871797 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b5fc0bc700 con 0x55b5fc2183f0 2024-05-27T10:22:48.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.953+0000 7fcd37e00700 10 client.25816 handle_command_reply: tid=0 2024-05-27T10:22:48.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.953+0000 7fcd6b370b80 1 client.25816 shutdown 2024-05-27T10:22:48.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 msgr2=0x55b5fc1a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:48.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1a1ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd440aa240 tx=0x7fcd44009c00 comp rx=0 tx=0).stop 2024-05-27T10:22:48.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 shutdown_connections 2024-05-27T10:22:48.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5fc2183f0 0x55b5fc21a810 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5fc199380 0x55b5fc1a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5fc013fd0 0x55b5fc1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 --2- 172.21.15.42:0/2981871797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5fc1934d0 0x55b5fc1a1ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 >> 172.21.15.42:0/2981871797 conn(0x55b5fc074150 msgr2=0x55b5fc197a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd37e00700 0 client.25816 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 shutdown_connections 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 1 -- 172.21.15.42:0/2981871797 wait complete. 2024-05-27T10:22:48.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd6b370b80 20 client.25816 trim_cache size 0 max 16384 2024-05-27T10:22:48.959 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:22:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 29, 2024-05-27T10:22:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:22:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 10, 2024-05-27T10:22:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "d78bfd53": { 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 10, 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:22:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 150.0, 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:22:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:22:48.962 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:22:48.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcd3c0b2b30 msgr2=0x7fcd3c0b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:48.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcd3c0b2b30 0x7fcd3c0b5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55b5fc194180 tx=0x7fcd500099e0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd54078630 msgr2=0x7fcd5406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:48.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd54078630 0x7fcd5406e4c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4400b790 comp rx=0 tx=0).stop 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 shutdown_connections 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcd3c0b2b30 0x7fcd3c0b5040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd541151e0 0x7fcd5406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd54077ce0 0x7fcd54075390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 --2- 172.21.15.42:0/2677554569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd54078630 0x7fcd5406e4c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 >> 172.21.15.42:0/2677554569 conn(0x7fcd54102080 msgr2=0x7fcd5407ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 shutdown_connections 2024-05-27T10:22:48.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:48.955+0000 7fcd5c800700 1 -- 172.21.15.42:0/2677554569 wait complete. 2024-05-27T10:22:49.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:48 smithi104 ceph-mon[34812]: pgmap v125: 97 pgs: 97 active+clean; 38 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-05-27T10:22:49.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:48 smithi143 ceph-mon[35545]: pgmap v125: 97 pgs: 97 active+clean; 38 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-05-27T10:22:49.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:48 smithi042 ceph-mon[31661]: pgmap v125: 97 pgs: 97 active+clean; 38 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-05-27T10:22:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:50 smithi104 ceph-mon[34812]: pgmap v126: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-05-27T10:22:51.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:50 smithi143 ceph-mon[35545]: pgmap v126: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-05-27T10:22:51.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:50 smithi042 ceph-mon[31661]: pgmap v126: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-05-27T10:22:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 -- 172.21.15.42:0/4213659854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 msgr2=0x7f091010f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 --2- 172.21.15.42:0/4213659854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f091010f2b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f08f8007350 tx=0x7f08f8029f50 comp rx=0 tx=0).stop 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 -- 172.21.15.42:0/4213659854 shutdown_connections 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 --2- 172.21.15.42:0/4213659854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091010f940 0x7f09101170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 --2- 172.21.15.42:0/4213659854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f091010f2b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 --2- 172.21.15.42:0/4213659854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f091010a8f0 0x7f091010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 -- 172.21.15.42:0/4213659854 >> 172.21.15.42:0/4213659854 conn(0x7f091006eda0 msgr2=0x7f091006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 -- 172.21.15.42:0/4213659854 shutdown_connections 2024-05-27T10:22:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.702+0000 7f0914e00640 1 -- 172.21.15.42:0/4213659854 wait complete. 2024-05-27T10:22:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.703+0000 7f0914e00640 1 Processor -- start 2024-05-27T10:22:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 -- start start 2024-05-27T10:22:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f091010a8f0 0x7f09101a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f09101a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091010f940 0x7f09101a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f091011a260 con 0x7f091010ee80 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f091011a0e0 con 0x7f091010a8f0 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0914e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f091011a3e0 con 0x7f091010f940 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f09101a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f09101a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55910/0 (socket says 172.21.15.42:55910) 2024-05-27T10:22:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.704+0000 7f0905400640 1 -- 172.21.15.42:0/3042850007 learned_addr learned my addr 172.21.15.42:0/3042850007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f090de00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f091010a8f0 0x7f09101a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f090e800640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091010f940 0x7f09101a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f0905400640 1 -- 172.21.15.42:0/3042850007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091010f940 msgr2=0x7f09101a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f0905400640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091010f940 0x7f09101a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f0905400640 1 -- 172.21.15.42:0/3042850007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f091010a8f0 msgr2=0x7f09101a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f0905400640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f091010a8f0 0x7f09101a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f0905400640 1 -- 172.21.15.42:0/3042850007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09101aa080 con 0x7f091010ee80 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.705+0000 7f0905400640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f09101a5780 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f8009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.706+0000 7f0907400640 1 -- 172.21.15.42:0/3042850007 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8009870 con 0x7f091010ee80 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.706+0000 7f0907400640 1 -- 172.21.15.42:0/3042850007 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08f803b910 con 0x7f091010ee80 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.706+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09101aa310 con 0x7f091010ee80 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.706+0000 7f0907400640 1 -- 172.21.15.42:0/3042850007 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8004f10 con 0x7f091010ee80 2024-05-27T10:22:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.706+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101aa800 con 0x7f091010ee80 2024-05-27T10:22:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 Processor -- start 2024-05-27T10:22:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 -- start start 2024-05-27T10:22:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f0907400640 1 -- 172.21.15.42:0/3042850007 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f08f8042020 con 0x7f091010ee80 2024-05-27T10:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336851b0 0x561433a27860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143365c2f0 0x561433a27da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561433998b90 con 0x5614336644d0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561433988730 con 0x56143365c2f0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.710+0000 7f091bddf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56143398c770 con 0x5614336851b0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55924/0 (socket says 172.21.15.42:55924) 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 -- 172.21.15.42:0/3792862725 learned_addr learned my addr 172.21.15.42:0/3792862725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f090de00640 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143365c2f0 0x561433a27da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f090e800640 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336851b0 0x561433a27860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 -- 172.21.15.42:0/3792862725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336851b0 msgr2=0x561433a27860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336851b0 0x561433a27860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 -- 172.21.15.42:0/3792862725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143365c2f0 msgr2=0x561433a27da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143365c2f0 0x561433a27da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.711+0000 7f0905400640 1 -- 172.21.15.42:0/3792862725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614339218b0 con 0x5614336644d0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f0905400640 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a282e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f08f80ff730 tx=0x7f08f810a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6194c81759b767 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f08ff400640 1 -- 172.21.15.42:0/3792862725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f810ad20 con 0x5614336644d0 2024-05-27T10:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f0907400640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f08f40b2950 0x7f08f40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f08ff400640 1 -- 172.21.15.42:0/3792862725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08f810f070 con 0x5614336644d0 2024-05-27T10:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f08ff400640 1 -- 172.21.15.42:0/3792862725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f80c59d0 con 0x5614336644d0 2024-05-27T10:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f0907400640 1 -- 172.21.15.42:0/3042850007 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f08f8101fa0 con 0x7f091010ee80 2024-05-27T10:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.712+0000 7f090e800640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f08f40b2950 0x7f08f40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 -- 172.21.15.42:0/3792862725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 msgr2=0x561433a282e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a282e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f08f80ff730 tx=0x7f08f810a0e0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 -- 172.21.15.42:0/3792862725 shutdown_connections 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a282e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143365c2f0 0x561433a27da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 --2- 172.21.15.42:0/3792862725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336851b0 0x561433a27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 -- 172.21.15.42:0/3792862725 >> 172.21.15.42:0/3792862725 conn(0x561433679300 msgr2=0x561433a24010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 -- 172.21.15.42:0/3792862725 shutdown_connections 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f090e800640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f08f40b2950 0x7f08f40b4e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0908005fd0 tx=0x7f0908005cf0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.713+0000 7f091bddf740 1 -- 172.21.15.42:0/3792862725 wait complete. 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.714+0000 7f091bddf740 1 -- start start 2024-05-27T10:22:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.714+0000 7f091bddf740 1 Processor -- start 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a31a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336644d0 0x561433a31f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56143366e790 con 0x56143365c2f0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561433930bc0 con 0x5614336851b0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561433a34750 con 0x5614336644d0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f091bddf740 10 client.0 _ll_register_callbacks cb 0x56143390e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f090de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a31a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f090de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a31a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55936/0 (socket says 172.21.15.42:55936) 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f090de00640 1 -- 172.21.15.42:0/736018716 learned_addr learned my addr 172.21.15.42:0/736018716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.715+0000 7f0905400640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336644d0 0x561433a31f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090e800640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090de00640 1 -- 172.21.15.42:0/736018716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336644d0 msgr2=0x561433a31f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090de00640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336644d0 0x561433a31f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090de00640 1 -- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 msgr2=0x561433a3da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090de00640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090de00640 1 -- 172.21.15.42:0/736018716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143369d210 con 0x56143365c2f0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090e800640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f090de00640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a31a00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0900011890 tx=0x7f0900011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fe5f51c851fd08 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f08f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f08f3e00640 1 -- 172.21.15.42:0/736018716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090000fb40 con 0x56143365c2f0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f08f3e00640 1 -- 172.21.15.42:0/736018716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f090000fce0 con 0x56143365c2f0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.716+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f810f070 con 0x56143365c2f0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f08f3e00640 1 -- 172.21.15.42:0/736018716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0900012450 con 0x56143365c2f0 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561433769620 con 0x56143365c2f0 2024-05-27T10:22:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f08f3e00640 1 -- 172.21.15.42:0/736018716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08f810f070 con 0x56143365c2f0 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f08f3e00640 1 -- 172.21.15.42:0/736018716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f090000f040 con 0x56143365c2f0 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 20 client.16230 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433643d80 0x561433a43ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 4 client.16230 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.717+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143399aae0 con 0x561433643d80 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.718+0000 7f0905400640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433643d80 0x561433a43ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.719+0000 7f0905400640 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433643d80 0x561433a43ae0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bdd9dd975eb52d server_cookie=25a5ac6e2920cbb6 in_seq=0 out_seq=0 2024-05-27T10:22:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.719+0000 7f08f3e00640 10 client.16230 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f08f3e00640 1 -- 172.21.15.42:0/736018716 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56143399aae0 con 0x561433643d80 2024-05-27T10:22:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f08f3e00640 10 client.16230 handle_command_reply: tid=0 2024-05-27T10:22:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 client.16230 shutdown 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 msgr2=0x561433a31a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a31a00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0900011890 tx=0x7f0900011d50 comp rx=0 tx=0).stop 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 shutdown_connections 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433643d80 0x561433a43ae0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336644d0 0x561433a31f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 --2- 172.21.15.42:0/736018716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a31a00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f08f3e00640 0 client.16230 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.721+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 >> 172.21.15.42:0/736018716 conn(0x561433679300 msgr2=0x561433a29580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.722+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 shutdown_connections 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.722+0000 7f091bddf740 1 -- 172.21.15.42:0/736018716 wait complete. 2024-05-27T10:22:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.722+0000 7f091bddf740 20 client.16230 trim_cache size 0 max 16384 2024-05-27T10:22:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 Processor -- start 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 -- start start 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56143365c2f0 0x561433a34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a27fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09101bcc60 con 0x5614336644d0 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09101ba530 con 0x5614336851b0 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.739+0000 7f091bddf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09101b8380 con 0x56143365c2f0 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a27fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a27fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55952/0 (socket says 172.21.15.42:55952) 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 -- 172.21.15.42:0/524644773 learned_addr learned my addr 172.21.15.42:0/524644773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f090de00640 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56143365c2f0 0x561433a34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f090e800640 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 -- 172.21.15.42:0/524644773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56143365c2f0 msgr2=0x561433a34130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56143365c2f0 0x561433a34130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 -- 172.21.15.42:0/524644773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 msgr2=0x561433a3d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f0905400640 1 -- 172.21.15.42:0/524644773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561433a24c00 con 0x5614336644d0 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.740+0000 7f090e800640 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.741+0000 7f0905400640 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a27fb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f08f80c5400 tx=0x7f08f8105a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bddf8d7048b47e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.741+0000 7f08fea00640 1 -- 172.21.15.42:0/524644773 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f80c8cd0 con 0x5614336644d0 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.741+0000 7f08fea00640 1 -- 172.21.15.42:0/524644773 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08f810f070 con 0x5614336644d0 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.741+0000 7f08fea00640 1 -- 172.21.15.42:0/524644773 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f80c8ec0 con 0x5614336644d0 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 -- 172.21.15.42:0/524644773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 msgr2=0x561433a27fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a27fb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f08f80c5400 tx=0x7f08f8105a50 comp rx=0 tx=0).stop 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 -- 172.21.15.42:0/524644773 shutdown_connections 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614336644d0 0x561433a27fb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 --2- 172.21.15.42:0/524644773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56143365c2f0 0x561433a34130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 -- 172.21.15.42:0/524644773 >> 172.21.15.42:0/524644773 conn(0x561433679300 msgr2=0x561433a31cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 -- 172.21.15.42:0/524644773 shutdown_connections 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 -- 172.21.15.42:0/524644773 wait complete. 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.742+0000 7f091bddf740 1 -- start start 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 Processor -- start 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a39c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336191c0 0x561433a34fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09101b89f0 con 0x56143365c2f0 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09101b9e60 con 0x5614336851b0 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09101b90c0 con 0x5614336191c0 2024-05-27T10:22:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f091bddf740 10 client.0 _ll_register_callbacks cb 0x7f09000096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.743+0000 7f090de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a39c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a39c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55962/0 (socket says 172.21.15.42:55962) 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 -- 172.21.15.42:0/444471432 learned_addr learned my addr 172.21.15.42:0/444471432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f0905400640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 -- 172.21.15.42:0/444471432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336191c0 msgr2=0x561433a34fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090e800640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336191c0 0x561433a34fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336191c0 0x561433a34fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 -- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 msgr2=0x561433a3a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 -- 172.21.15.42:0/444471432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561433a35730 con 0x56143365c2f0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090e800640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336191c0 0x561433a34fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f0905400640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f090de00640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a39c10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0900011890 tx=0x7f090001b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdb178c72626880 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.744+0000 7f08f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f08f3e00640 1 -- 172.21.15.42:0/444471432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900028040 con 0x56143365c2f0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f08f3e00640 1 -- 172.21.15.42:0/444471432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f090000f940 con 0x56143365c2f0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f810f070 con 0x56143365c2f0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f08f3e00640 1 -- 172.21.15.42:0/444471432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f090003c050 con 0x56143365c2f0 2024-05-27T10:22:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f091bddf740 10 client.16236 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561433aabbc0 con 0x56143365c2f0 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.745+0000 7f08f3e00640 1 -- 172.21.15.42:0/444471432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08f810f070 con 0x56143365c2f0 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f08f3e00640 1 -- 172.21.15.42:0/444471432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f09000125f0 con 0x56143365c2f0 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f091bddf740 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f091bddf740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f091bddf740 20 client.16236 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f091bddf740 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433aac420 0x561433aae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f091bddf740 4 client.16236 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614339942c0 con 0x561433aac420 2024-05-27T10:22:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.746+0000 7f0905400640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433aac420 0x561433aae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.747+0000 7f0905400640 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433aac420 0x561433aae860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8cc9be79ef09a68 server_cookie=79150519c1b18bf6 in_seq=0 out_seq=0 2024-05-27T10:22:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.747+0000 7f08f3e00640 10 client.16236 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.769+0000 7f08f3e00640 1 -- 172.21.15.42:0/444471432 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5614339942c0 con 0x561433aac420 2024-05-27T10:22:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.769+0000 7f08f3e00640 10 client.16236 handle_command_reply: tid=0 2024-05-27T10:22:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.769+0000 7f091bddf740 1 client.16236 shutdown 2024-05-27T10:22:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 msgr2=0x561433a39c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a39c10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0900011890 tx=0x7f090001b430 comp rx=0 tx=0).stop 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 shutdown_connections 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561433aac420 0x561433aae860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614336191c0 0x561433a34fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f08f3e00640 0 client.16236 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614336851b0 0x561433a3a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 --2- 172.21.15.42:0/444471432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56143365c2f0 0x561433a39c10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 >> 172.21.15.42:0/444471432 conn(0x561433679300 msgr2=0x561433a31cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 shutdown_connections 2024-05-27T10:22:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.770+0000 7f091bddf740 1 -- 172.21.15.42:0/444471432 wait complete. 2024-05-27T10:22:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.771+0000 7f091bddf740 20 client.16236 trim_cache size 0 max 16384 2024-05-27T10:22:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:22:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:22:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.771+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f08f40b2950 msgr2=0x7f08f40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.771+0000 7f0914e00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f08f40b2950 0x7f08f40b4e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0908005fd0 tx=0x7f0908005cf0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 msgr2=0x7f09101a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f09101a5780 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f8009440 comp rx=0 tx=0).stop 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 shutdown_connections 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f08f40b2950 0x7f08f40b4e60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091010f940 0x7f09101a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f091010ee80 0x7f09101a5780 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 --2- 172.21.15.42:0/3042850007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f091010a8f0 0x7f09101a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 >> 172.21.15.42:0/3042850007 conn(0x7f091006eda0 msgr2=0x7f0910115a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 shutdown_connections 2024-05-27T10:22:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:22:52.772+0000 7f0914e00640 1 -- 172.21.15.42:0/3042850007 wait complete. 2024-05-27T10:22:53.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:52 smithi104 ceph-mon[34812]: pgmap v127: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-27T10:22:53.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:52 smithi143 ceph-mon[35545]: pgmap v127: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-27T10:22:53.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:52 smithi042 ceph-mon[31661]: pgmap v127: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-27T10:22:55.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:54 smithi104 ceph-mon[34812]: pgmap v128: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-27T10:22:55.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:54 smithi042 ceph-mon[31661]: pgmap v128: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-27T10:22:55.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:54 smithi143 ceph-mon[35545]: pgmap v128: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-27T10:22:56.430 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd78bfd53', quiesce took 22.4 seconds. Will release after: 7.600000000000001 2024-05-27T10:22:57.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:56 smithi042 ceph-mon[31661]: pgmap v129: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-05-27T10:22:57.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 -- 172.21.15.42:0/3449509536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093410b890 msgr2=0x7f09341131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 --2- 172.21.15.42:0/3449509536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093410b890 0x7f09341131f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f092c00b0d0 tx=0x7f092c00b3e0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 -- 172.21.15.42:0/3449509536 shutdown_connections 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 --2- 172.21.15.42:0/3449509536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093410b890 0x7f09341131f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 --2- 172.21.15.42:0/3449509536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0934106610 0x7f0934106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 --2- 172.21.15.42:0/3449509536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f093410af40 0x7f093410b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.165+0000 7f0933e00700 1 -- 172.21.15.42:0/3449509536 >> 172.21.15.42:0/3449509536 conn(0x7f0934102020 msgr2=0x7f0934104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.166+0000 7f0933e00700 1 -- 172.21.15.42:0/3449509536 shutdown_connections 2024-05-27T10:22:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.166+0000 7f0933e00700 1 -- 172.21.15.42:0/3449509536 wait complete. 2024-05-27T10:22:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.167+0000 7f0933e00700 1 Processor -- start 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.167+0000 7f0933e00700 1 -- start start 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 0x7f093419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f093410af40 0x7f093419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f093410b890 0x7f09341a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09341163c0 con 0x7f0934106610 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0934116240 con 0x7f093410af40 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0934116540 con 0x7f093410b890 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0932a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 0x7f093419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0932a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 0x7f093419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36876/0 (socket says 172.21.15.42:36876) 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0932a00700 1 -- 172.21.15.42:0/2308292206 learned_addr learned my addr 172.21.15.42:0/2308292206 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0932000700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f093410af40 0x7f093419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.168+0000 7f0933400700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f093410b890 0x7f09341a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0932a00700 1 -- 172.21.15.42:0/2308292206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f093410b890 msgr2=0x7f09341a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0932a00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f093410b890 0x7f09341a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0932a00700 1 -- 172.21.15.42:0/2308292206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f093410af40 msgr2=0x7f093419d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0932a00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f093410af40 0x7f093419d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0932a00700 1 -- 172.21.15.42:0/2308292206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09341a1c00 con 0x7f0934106610 2024-05-27T10:22:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0932a00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 0x7f093419cd30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f092400d950 tx=0x7f092400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f092b400700 1 -- 172.21.15.42:0/2308292206 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924016070 con 0x7f0934106610 2024-05-27T10:22:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f092b400700 1 -- 172.21.15.42:0/2308292206 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0924010460 con 0x7f0934106610 2024-05-27T10:22:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f092b400700 1 -- 172.21.15.42:0/2308292206 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092400f620 con 0x7f0934106610 2024-05-27T10:22:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.169+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09341a1ef0 con 0x7f0934106610 2024-05-27T10:22:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.170+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09341a9810 con 0x7f0934106610 2024-05-27T10:22:57.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.173+0000 7f092b400700 1 -- 172.21.15.42:0/2308292206 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0924019070 con 0x7f0934106610 2024-05-27T10:22:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.174+0000 7f09485e4b80 1 Processor -- start 2024-05-27T10:22:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.174+0000 7f09485e4b80 1 -- start start 2024-05-27T10:22:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f7ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d410 0x556b99f87ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 0x556b99f8a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f09485e4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b99eabe40 con 0x556b99f7d820 2024-05-27T10:22:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f09485e4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556b99e9b9e0 con 0x556b99dfee40 2024-05-27T10:22:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f09485e4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556b99e9fa20 con 0x556b99f7d410 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f092b400700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f09180b2b40 0x7f09180b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f0932000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 0x556b99f8a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f0932000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 0x556b99f8a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36880/0 (socket says 172.21.15.42:36880) 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f0932000700 1 -- 172.21.15.42:0/2635496083 learned_addr learned my addr 172.21.15.42:0/2635496083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f092b400700 1 -- 172.21.15.42:0/2308292206 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f09240e20f0 con 0x7f0934106610 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f0932a00700 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d410 0x556b99f87ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.175+0000 7f0933400700 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f7ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0933400700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f09180b2b40 0x7f09180b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0932000700 1 -- 172.21.15.42:0/2635496083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d410 msgr2=0x556b99f87ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0932000700 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d410 0x556b99f87ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0932000700 1 -- 172.21.15.42:0/2635496083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 msgr2=0x556b99f7ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0932000700 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f7ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0932000700 1 -- 172.21.15.42:0/2635496083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b99e64420 con 0x556b99f7d820 2024-05-27T10:22:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0932000700 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 0x556b99f8a020 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f092000dc80 tx=0x7f092000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2ba6f7ef2aaaa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.176+0000 7f0933400700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f09180b2b40 0x7f09180b5050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556b99f79690 tx=0x7f092c00b040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.177+0000 7f091fe00700 1 -- 172.21.15.42:0/2635496083 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092000fb70 con 0x556b99f7d820 2024-05-27T10:22:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.177+0000 7f091fe00700 1 -- 172.21.15.42:0/2635496083 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f092000fd10 con 0x556b99f7d820 2024-05-27T10:22:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.177+0000 7f091fe00700 1 -- 172.21.15.42:0/2635496083 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092001b610 con 0x556b99f7d820 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 -- 172.21.15.42:0/2635496083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 msgr2=0x556b99f8a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 0x556b99f8a020 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f092000dc80 tx=0x7f092000df90 comp rx=0 tx=0).stop 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 -- 172.21.15.42:0/2635496083 shutdown_connections 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d820 0x556b99f8a020 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f7ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 --2- 172.21.15.42:0/2635496083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d410 0x556b99f87ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 -- 172.21.15.42:0/2635496083 >> 172.21.15.42:0/2635496083 conn(0x556b99e5efc0 msgr2=0x556b99f78d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 -- 172.21.15.42:0/2635496083 shutdown_connections 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 -- 172.21.15.42:0/2635496083 wait complete. 2024-05-27T10:22:57.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.178+0000 7f09485e4b80 1 -- start start 2024-05-27T10:22:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 Processor -- start 2024-05-27T10:22:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f87560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 0x556b99f87aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d820 0x556b99f87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b99da1fa0 con 0x556b99f7d410 2024-05-27T10:22:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556b99e3c760 con 0x556b99dfee40 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556b99f8a990 con 0x556b99f7d820 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.179+0000 7f09485e4b80 10 client.0 _ll_register_callbacks cb 0x556b99e45c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 0x556b99f87aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 0x556b99f87aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36884/0 (socket says 172.21.15.42:36884) 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 -- 172.21.15.42:0/3017925570 learned_addr learned my addr 172.21.15.42:0/3017925570 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0933400700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d820 0x556b99f87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932a00700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f87560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 -- 172.21.15.42:0/3017925570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d820 msgr2=0x556b99f87fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d820 0x556b99f87fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 -- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 msgr2=0x556b99f87560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f87560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 -- 172.21.15.42:0/3017925570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b99deb8d0 con 0x556b99f7d410 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0933400700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d820 0x556b99f87fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932a00700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f87560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f0932000700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 0x556b99f87aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f0920011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526a2a93a67b2445 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.180+0000 7f091cc00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f091cc00700 1 -- 172.21.15.42:0/3017925570 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092002a060 con 0x556b99f7d410 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092000fd10 con 0x556b99f7d410 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f091cc00700 1 -- 172.21.15.42:0/3017925570 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f0920004040 con 0x556b99f7d410 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f091cc00700 1 -- 172.21.15.42:0/3017925570 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0920004240 con 0x556b99f7d410 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f09485e4b80 10 client.16248 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b99e6c060 con 0x556b99f7d410 2024-05-27T10:22:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.181+0000 7f091cc00700 1 -- 172.21.15.42:0/3017925570 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f092000fd10 con 0x556b99f7d410 2024-05-27T10:22:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f091cc00700 1 -- 172.21.15.42:0/3017925570 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f09200051e0 con 0x556b99f7d410 2024-05-27T10:22:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f09485e4b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f09485e4b80 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f09485e4b80 20 client.16248 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f97850 0x556b99f99c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f09485e4b80 4 client.16248 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.182+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b99eadd90 con 0x556b99f97850 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.183+0000 7f0932a00700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f97850 0x556b99f99c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.184+0000 7f0932a00700 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f97850 0x556b99f99c70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac8424087ca5821b server_cookie=b51d7e87f248f248 in_seq=0 out_seq=0 2024-05-27T10:22:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.184+0000 7f091cc00700 10 client.16248 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:57.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.185+0000 7f091cc00700 1 -- 172.21.15.42:0/3017925570 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b99eadd90 con 0x556b99f97850 2024-05-27T10:22:57.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.185+0000 7f091cc00700 10 client.16248 handle_command_reply: tid=0 2024-05-27T10:22:57.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 client.16248 shutdown 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 msgr2=0x556b99f87aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 0x556b99f87aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f0920011ca0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 shutdown_connections 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f97850 0x556b99f99c70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99f7d820 0x556b99f87fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99dfee40 0x556b99f87560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3017925570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99f7d410 0x556b99f87aa0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 >> 172.21.15.42:0/3017925570 conn(0x556b99e5efc0 msgr2=0x556b99f77ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f091cc00700 0 client.16248 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 shutdown_connections 2024-05-27T10:22:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 1 -- 172.21.15.42:0/3017925570 wait complete. 2024-05-27T10:22:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.186+0000 7f09485e4b80 20 client.16248 trim_cache size 0 max 16384 2024-05-27T10:22:57.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:56 smithi104 ceph-mon[34812]: pgmap v129: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-05-27T10:22:57.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.203+0000 7f09485e4b80 1 Processor -- start 2024-05-27T10:22:57.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.203+0000 7f09485e4b80 1 -- start start 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f82dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f89a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f93c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f09485e4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b3520 con 0x556b99dfee40 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f09485e4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b0db0 con 0x556b99f7d410 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f09485e4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09341aec00 con 0x556b99e5efc0 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f0932000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f89a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f0932000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f89a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36898/0 (socket says 172.21.15.42:36898) 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f0932000700 1 -- 172.21.15.42:0/3416204434 learned_addr learned my addr 172.21.15.42:0/3416204434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.204+0000 7f0932a00700 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f82dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0933400700 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f93c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932000700 1 -- 172.21.15.42:0/3416204434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 msgr2=0x556b99f82dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932000700 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f82dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932000700 1 -- 172.21.15.42:0/3416204434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 msgr2=0x556b99f93c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932000700 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f93c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932000700 1 -- 172.21.15.42:0/3416204434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b99f731d0 con 0x556b99dfee40 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932a00700 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f82dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f0932000700 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f89a00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0920005810 tx=0x7f092000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c136f2b2f8f6b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.205+0000 7f091f400700 1 -- 172.21.15.42:0/3416204434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092002a040 con 0x556b99dfee40 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.206+0000 7f091f400700 1 -- 172.21.15.42:0/3416204434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f092000fcc0 con 0x556b99dfee40 2024-05-27T10:22:57.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.206+0000 7f091f400700 1 -- 172.21.15.42:0/3416204434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920004d80 con 0x556b99dfee40 2024-05-27T10:22:57.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.206+0000 7f09485e4b80 1 -- 172.21.15.42:0/3416204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 msgr2=0x556b99f89a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.206+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f89a00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0920005810 tx=0x7f092000f640 comp rx=0 tx=0).stop 2024-05-27T10:22:57.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 -- 172.21.15.42:0/3416204434 shutdown_connections 2024-05-27T10:22:57.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f93c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f82dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 --2- 172.21.15.42:0/3416204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f89a00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 -- 172.21.15.42:0/3416204434 >> 172.21.15.42:0/3416204434 conn(0x556b99f71f60 msgr2=0x556b99f85e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 -- 172.21.15.42:0/3416204434 shutdown_connections 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 -- 172.21.15.42:0/3416204434 wait complete. 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.207+0000 7f09485e4b80 1 -- start start 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 Processor -- start 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f9cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f9a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f9a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09341af270 con 0x556b99dfee40 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b06e0 con 0x556b99f7d410 2024-05-27T10:22:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09341af940 con 0x556b99e5efc0 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.208+0000 7f09485e4b80 10 client.0 _ll_register_callbacks cb 0x556b99e45c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f9cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f9cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36900/0 (socket says 172.21.15.42:36900) 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 -- 172.21.15.42:0/1508803309 learned_addr learned my addr 172.21.15.42:0/1508803309 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932000700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f9a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0933400700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f9a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 -- 172.21.15.42:0/1508803309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 msgr2=0x556b99f9a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f9a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 -- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 msgr2=0x556b99f9a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f9a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 -- 172.21.15.42:0/1508803309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b99f88350 con 0x556b99dfee40 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932000700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f9a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:22:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0933400700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f9a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f0932a00700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f9cd00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f09240dfe90 tx=0x7f09240e1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fff3b21db34305 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.209+0000 7f091cc00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f091cc00700 1 -- 172.21.15.42:0/1508803309 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240c2020 con 0x556b99dfee40 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f091cc00700 1 -- 172.21.15.42:0/1508803309 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f09240a9680 con 0x556b99dfee40 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092000fcc0 con 0x556b99dfee40 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f091cc00700 1 -- 172.21.15.42:0/1508803309 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0924020040 con 0x556b99dfee40 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f09485e4b80 10 client.16254 fetch_fsmap learned FSMap version 29 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b99f7e420 con 0x556b99dfee40 2024-05-27T10:22:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.210+0000 7f091cc00700 1 -- 172.21.15.42:0/1508803309 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f092000fcc0 con 0x556b99dfee40 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f091cc00700 1 -- 172.21.15.42:0/1508803309 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f09240ca310 con 0x556b99dfee40 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f09485e4b80 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f09485e4b80 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f09485e4b80 20 client.16254 populate_metadata read hostname 'smithi042' 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f09485e4b80 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f7d820 0x556b9a004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f09485e4b80 4 client.16254 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b99ea7570 con 0x556b99f7d820 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.211+0000 7f0932000700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f7d820 0x556b9a004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:22:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.212+0000 7f0932000700 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f7d820 0x556b9a004160 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7f0036d9d2e5d7 server_cookie=55f45850e1d4592e in_seq=0 out_seq=0 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.212+0000 7f091cc00700 10 client.16254 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f091cc00700 1 -- 172.21.15.42:0/1508803309 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b99ea7570 con 0x556b99f7d820 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f091cc00700 10 client.16254 handle_command_reply: tid=0 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 client.16254 shutdown 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 msgr2=0x556b99f9cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f9cd00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f09240dfe90 tx=0x7f09240e1040 comp rx=0 tx=0).stop 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 shutdown_connections 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x556b99f7d820 0x556b9a004160 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b99f7d410 0x556b99f9a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b99dfee40 0x556b99f9cd00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 --2- 172.21.15.42:0/1508803309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556b99e5efc0 0x556b99f9a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 >> 172.21.15.42:0/1508803309 conn(0x556b99f71f60 msgr2=0x556b99f82910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f091cc00700 0 client.16254 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 shutdown_connections 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.213+0000 7f09485e4b80 1 -- 172.21.15.42:0/1508803309 wait complete. 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f09485e4b80 20 client.16254 trim_cache size 0 max 16384 2024-05-27T10:22:57.217 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f09180b2b40 msgr2=0x7f09180b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f09180b2b40 0x7f09180b5050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556b99f79690 tx=0x7f092c00b040 comp rx=0 tx=0).stop 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 msgr2=0x7f093419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 0x7f093419cd30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f092400d950 tx=0x7f092400dd10 comp rx=0 tx=0).stop 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 shutdown_connections 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f09180b2b40 0x7f09180b5050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f093410b890 0x7f09341a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0934106610 0x7f093419cd30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 --2- 172.21.15.42:0/2308292206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f093410af40 0x7f093419d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:22:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.214+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 >> 172.21.15.42:0/2308292206 conn(0x7f0934102020 msgr2=0x7f093410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:22:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.215+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 shutdown_connections 2024-05-27T10:22:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:22:57.215+0000 7f0933e00700 1 -- 172.21.15.42:0/2308292206 wait complete. 2024-05-27T10:22:57.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:56 smithi143 ceph-mon[35545]: pgmap v129: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-05-27T10:22:58.349 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 133349416, 'ino': 1099511628316, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/compile'}, {'damage_type': 'backtrace', 'id': 247677524, 'ino': 1099511628301, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.c'}, {'damage_type': 'backtrace', 'id': 464415667, 'ino': 1099511628293, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/blogbench.8'}, {'damage_type': 'backtrace', 'id': 495209833, 'ino': 1099511628314, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/helpers.c'}, {'damage_type': 'backtrace', 'id': 674972773, 'ino': 1099511628305, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h'}, {'damage_type': 'backtrace', 'id': 815003069, 'ino': 1099511628292, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man'}, {'damage_type': 'backtrace', 'id': 834628144, 'ino': 1099511628298, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.h'}, {'damage_type': 'backtrace', 'id': 1107507586, 'ino': 1099511628306, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.am'}, {'damage_type': 'backtrace', 'id': 1231616604, 'ino': 1099511628317, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/depcomp'}, {'damage_type': 'backtrace', 'id': 1381291436, 'ino': 1099511628299, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.c'}, {'damage_type': 'backtrace', 'id': 1490116186, 'ino': 1099511628303, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench_p.h'}, {'damage_type': 'backtrace', 'id': 1501016016, 'ino': 1099511628315, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/NEWS'}, {'damage_type': 'backtrace', 'id': 1652420151, 'ino': 1099511628302, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/fakesnprintf.h'}, {'damage_type': 'backtrace', 'id': 1728253692, 'ino': 1099511628313, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/writer.c'}, {'damage_type': 'backtrace', 'id': 1897883745, 'ino': 1099511628304, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c'}, {'damage_type': 'backtrace', 'id': 2365897189, 'ino': 1099511628312, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/reader.c'}, {'damage_type': 'backtrace', 'id': 2486120528, 'ino': 1099511628296, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src'}, {'damage_type': 'backtrace', 'id': 2773772391, 'ino': 1099511628297, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/mysnprintf.c'}, {'damage_type': 'backtrace', 'id': 2925647389, 'ino': 1099511628318, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/aclocal.m4'}, {'damage_type': 'backtrace', 'id': 2980271907, 'ino': 1099511628310, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/commenter.c'}, {'damage_type': 'backtrace', 'id': 2996849560, 'ino': 1099511628295, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.in'}, {'damage_type': 'backtrace', 'id': 3015229518, 'ino': 1099511628309, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/globals.h'}, {'damage_type': 'backtrace', 'id': 3174560903, 'ino': 1099511628308, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/process.c'}, {'damage_type': 'backtrace', 'id': 3755556396, 'ino': 1099511628300, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/blogbench.h'}, {'damage_type': 'backtrace', 'id': 3829600169, 'ino': 1099511628294, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man/Makefile.am'}, {'damage_type': 'backtrace', 'id': 3889680779, 'ino': 1099511628307, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/Makefile.in'}, {'damage_type': 'backtrace', 'id': 3951018840, 'ino': 1099511628311, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/src/rewriter.c'}, {'damage_type': 'backtrace', 'id': 3959197956, 'ino': 1099511628290, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4'}, {'damage_type': 'backtrace', 'id': 4031416416, 'ino': 1099511628291, 'path': '/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/m4/acx_pthread.m4'}] 2024-05-27T10:22:58.349 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-05-27T10:22:58.715 INFO:tasks.workunit.client.0.smithi042.stdout: 5 672 0 673 0 11 0 2024-05-27T10:22:59.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:22:58 smithi104 ceph-mon[34812]: pgmap v130: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 419 KiB/s wr, 117 op/s 2024-05-27T10:22:59.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:22:58 smithi143 ceph-mon[35545]: pgmap v130: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 419 KiB/s wr, 117 op/s 2024-05-27T10:22:59.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:22:58 smithi042 ceph-mon[31661]: pgmap v130: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 419 KiB/s wr, 117 op/s 2024-05-27T10:23:01.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:00 smithi104 ceph-mon[34812]: pgmap v131: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 421 KiB/s wr, 121 op/s 2024-05-27T10:23:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:00 smithi143 ceph-mon[35545]: pgmap v131: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 421 KiB/s wr, 121 op/s 2024-05-27T10:23:01.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:00 smithi042 ceph-mon[31661]: pgmap v131: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 421 KiB/s wr, 121 op/s 2024-05-27T10:23:03.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:02 smithi104 ceph-mon[34812]: pgmap v132: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-05-27T10:23:03.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:02 smithi143 ceph-mon[35545]: pgmap v132: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-05-27T10:23:03.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:02 smithi042 ceph-mon[31661]: pgmap v132: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-05-27T10:23:04.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:23:04.516 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:23:04.955 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:04 smithi042 ceph-mon[31661]: pgmap v133: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-05-27T10:23:05.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:04 smithi104 ceph-mon[34812]: pgmap v133: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-05-27T10:23:05.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:04 smithi143 ceph-mon[35545]: pgmap v133: 97 pgs: 97 active+clean; 42 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 -- 172.21.15.42:0/1216024101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb301102a0 msgr2=0x7fdb30112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 --2- 172.21.15.42:0/1216024101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb301102a0 0x7fdb30112780 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009ca0 tx=0x7fdb2802a8b0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 -- 172.21.15.42:0/1216024101 shutdown_connections 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 --2- 172.21.15.42:0/1216024101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb301102a0 0x7fdb30112780 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 --2- 172.21.15.42:0/1216024101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb3010d940 0x7fdb3010fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 --2- 172.21.15.42:0/1216024101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb30076770 0x7fdb30076b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 -- 172.21.15.42:0/1216024101 >> 172.21.15.42:0/1216024101 conn(0x7fdb300ff170 msgr2=0x7fdb301015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 -- 172.21.15.42:0/1216024101 shutdown_connections 2024-05-27T10:23:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.007+0000 7fdb36e00640 1 -- 172.21.15.42:0/1216024101 wait complete. 2024-05-27T10:23:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.008+0000 7fdb36e00640 1 Processor -- start 2024-05-27T10:23:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 -- start start 2024-05-27T10:23:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 0x7fdb3019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb3010d940 0x7fdb3019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb301102a0 0x7fdb301a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30115d90 con 0x7fdb30076770 2024-05-27T10:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30115c10 con 0x7fdb3010d940 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.009+0000 7fdb36e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30115f10 con 0x7fdb301102a0 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 0x7fdb3019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 0x7fdb3019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58072/0 (socket says 172.21.15.42:58072) 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 -- 172.21.15.42:0/2264642016 learned_addr learned my addr 172.21.15.42:0/2264642016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb36400640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb301102a0 0x7fdb301a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35000640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb3010d940 0x7fdb3019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 -- 172.21.15.42:0/2264642016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb301102a0 msgr2=0x7fdb301a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb301102a0 0x7fdb301a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 -- 172.21.15.42:0/2264642016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb3010d940 msgr2=0x7fdb3019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb3010d940 0x7fdb3019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.010+0000 7fdb35a00640 1 -- 172.21.15.42:0/2264642016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb301a1550 con 0x7fdb30076770 2024-05-27T10:23:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.011+0000 7fdb35a00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 0x7fdb3019c770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c950 tx=0x7fdb2400ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.011+0000 7fdb2e000640 1 -- 172.21.15.42:0/2264642016 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24007c70 con 0x7fdb30076770 2024-05-27T10:23:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.011+0000 7fdb2e000640 1 -- 172.21.15.42:0/2264642016 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb24007e10 con 0x7fdb30076770 2024-05-27T10:23:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.011+0000 7fdb2e000640 1 -- 172.21.15.42:0/2264642016 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24018760 con 0x7fdb30076770 2024-05-27T10:23:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.011+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb301a1840 con 0x7fdb30076770 2024-05-27T10:23:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.011+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301a1d90 con 0x7fdb30076770 2024-05-27T10:23:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.015+0000 7fdb3f588740 1 Processor -- start 2024-05-27T10:23:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.015+0000 7fdb3f588740 1 -- start start 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.015+0000 7fdb2e000640 1 -- 172.21.15.42:0/2264642016 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdb2400f450 con 0x7fdb30076770 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c4ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb3f588740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559048bbcb90 con 0x5590488802f0 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb3f588740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559048bac730 con 0x5590488a91b0 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb3f588740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559048bb0770 con 0x5590488884d0 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c4bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58076/0 (socket says 172.21.15.42:58076) 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb35a00640 1 -- 172.21.15.42:0/1978001356 learned_addr learned my addr 172.21.15.42:0/1978001356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb35000640 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.016+0000 7fdb36400640 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c4ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb35a00640 1 -- 172.21.15.42:0/1978001356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 msgr2=0x559048c4c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c4c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb35a00640 1 -- 172.21.15.42:0/1978001356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 msgr2=0x559048c4ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c4ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb35a00640 1 -- 172.21.15.42:0/1978001356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559048b458b0 con 0x5590488802f0 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb36400640 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c4bfa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdb240dfea0 tx=0x7fdb240e8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b977f758006639b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.017+0000 7fdb2e000640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdb180b2950 0x7fdb180b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.018+0000 7fdb1ea00640 1 -- 172.21.15.42:0/1978001356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240f1070 con 0x5590488802f0 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.018+0000 7fdb36400640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdb180b2950 0x7fdb180b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.018+0000 7fdb2e000640 1 -- 172.21.15.42:0/2264642016 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fdb240e2820 con 0x7fdb30076770 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.018+0000 7fdb1ea00640 1 -- 172.21.15.42:0/1978001356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb240a9600 con 0x5590488802f0 2024-05-27T10:23:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.018+0000 7fdb1ea00640 1 -- 172.21.15.42:0/1978001356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240a9440 con 0x5590488802f0 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 -- 172.21.15.42:0/1978001356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 msgr2=0x559048c4bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c4bfa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdb240dfea0 tx=0x7fdb240e8cb0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb36400640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdb180b2950 0x7fdb180b4e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009a60 tx=0x7fdb28009450 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 -- 172.21.15.42:0/1978001356 shutdown_connections 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c4c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c4bfa0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1978001356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 -- 172.21.15.42:0/1978001356 >> 172.21.15.42:0/1978001356 conn(0x55904889d300 msgr2=0x559048c48210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 -- 172.21.15.42:0/1978001356 shutdown_connections 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.019+0000 7fdb3f588740 1 -- 172.21.15.42:0/1978001356 wait complete. 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.020+0000 7fdb3f588740 1 -- start start 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.020+0000 7fdb3f588740 1 Processor -- start 2024-05-27T10:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c55bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c61ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559048892790 con 0x5590488802f0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559048b54bc0 con 0x5590488a91b0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559048c58b90 con 0x5590488884d0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb3f588740 10 client.0 _ll_register_callbacks cb 0x559048b32400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c55bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c55bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58082/0 (socket says 172.21.15.42:58082) 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 -- 172.21.15.42:0/1220871915 learned_addr learned my addr 172.21.15.42:0/1220871915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35000640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb36400640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c61ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 -- 172.21.15.42:0/1220871915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 msgr2=0x559048c56130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c56130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 -- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 msgr2=0x559048c61ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c61ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb35a00640 1 -- 172.21.15.42:0/1220871915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590488c1210 con 0x5590488802f0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.021+0000 7fdb36400640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c61ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c55bf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdb240a5b70 tx=0x7fdb240a9930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd0196ea72ea976 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb13400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb13400640 1 -- 172.21.15.42:0/1220871915 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240f1030 con 0x5590488802f0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb13400640 1 -- 172.21.15.42:0/1220871915 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fdb240a60e0 con 0x5590488802f0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb240a9600 con 0x5590488802f0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb13400640 1 -- 172.21.15.42:0/1220871915 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb240a9260 con 0x5590488802f0 2024-05-27T10:23:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb3f588740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55904898d620 con 0x5590488802f0 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.022+0000 7fdb13400640 1 -- 172.21.15.42:0/1220871915 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb240a9600 con 0x5590488802f0 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb13400640 1 -- 172.21.15.42:0/1220871915 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdb2400f740 con 0x5590488802f0 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb3f588740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb3f588740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb3f588740 20 client.16266 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559048867d80 0x559048c67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb3f588740 4 client.16266 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559048bbeae0 con 0x559048867d80 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.023+0000 7fdb35000640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559048867d80 0x559048c67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.024+0000 7fdb35000640 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559048867d80 0x559048c67cc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f9449b342c7f50 server_cookie=4bbe3bffbdb26aa5 in_seq=0 out_seq=0 2024-05-27T10:23:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.024+0000 7fdb13400640 10 client.16266 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.026+0000 7fdb13400640 1 -- 172.21.15.42:0/1220871915 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559048bbeae0 con 0x559048867d80 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.026+0000 7fdb13400640 10 client.16266 handle_command_reply: tid=0 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.026+0000 7fdb3f588740 1 client.16266 shutdown 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 msgr2=0x559048c55bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c55bf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdb240a5b70 tx=0x7fdb240a9930 comp rx=0 tx=0).stop 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 shutdown_connections 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c61ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559048867d80 0x559048c67cc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488884d0 0x559048c56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1220871915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c55bf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb13400640 0 client.16266 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 >> 172.21.15.42:0/1220871915 conn(0x55904889d300 msgr2=0x559048c4ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 shutdown_connections 2024-05-27T10:23:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 1 -- 172.21.15.42:0/1220871915 wait complete. 2024-05-27T10:23:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.027+0000 7fdb3f588740 20 client.16266 trim_cache size 0 max 16384 2024-05-27T10:23:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.044+0000 7fdb3f588740 1 Processor -- start 2024-05-27T10:23:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.044+0000 7fdb3f588740 1 -- start start 2024-05-27T10:23:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488884d0 0x559048c57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488a91b0 0x559048c61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb3f588740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b4180 con 0x5590488802f0 2024-05-27T10:23:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb3f588740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b1a50 con 0x5590488884d0 2024-05-27T10:23:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb3f588740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301af8a0 con 0x5590488a91b0 2024-05-27T10:23:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58084/0 (socket says 172.21.15.42:58084) 2024-05-27T10:23:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 -- 172.21.15.42:0/389191853 learned_addr learned my addr 172.21.15.42:0/389191853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 -- 172.21.15.42:0/389191853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488a91b0 msgr2=0x559048c61c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488a91b0 0x559048c61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 -- 172.21.15.42:0/389191853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488884d0 msgr2=0x559048c57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488884d0 0x559048c57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.045+0000 7fdb35a00640 1 -- 172.21.15.42:0/389191853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559048c54140 con 0x5590488802f0 2024-05-27T10:23:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.046+0000 7fdb35a00640 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c513a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdb240e2be0 tx=0x7fdb240e8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9ad0fa9db45cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.046+0000 7fdb1e000640 1 -- 172.21.15.42:0/389191853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240c2070 con 0x5590488802f0 2024-05-27T10:23:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.046+0000 7fdb1e000640 1 -- 172.21.15.42:0/389191853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb240ea670 con 0x5590488802f0 2024-05-27T10:23:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.046+0000 7fdb1e000640 1 -- 172.21.15.42:0/389191853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240f1070 con 0x5590488802f0 2024-05-27T10:23:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 -- 172.21.15.42:0/389191853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 msgr2=0x559048c513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c513a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdb240e2be0 tx=0x7fdb240e8cb0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 -- 172.21.15.42:0/389191853 shutdown_connections 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488a91b0 0x559048c61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488884d0 0x559048c57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 --2- 172.21.15.42:0/389191853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488802f0 0x559048c513a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 -- 172.21.15.42:0/389191853 >> 172.21.15.42:0/389191853 conn(0x55904889d300 msgr2=0x559048c55eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 -- 172.21.15.42:0/389191853 shutdown_connections 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 -- 172.21.15.42:0/389191853 wait complete. 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.047+0000 7fdb3f588740 1 -- start start 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 Processor -- start 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488802f0 0x559048c5e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 0x559048c5e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301aff10 con 0x5590488884d0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b1380 con 0x5590488a91b0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b05e0 con 0x5590488802f0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.048+0000 7fdb3f588740 10 client.0 _ll_register_callbacks cb 0x7fdb2000c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 0x559048c5e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 0x559048c5e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58096/0 (socket says 172.21.15.42:58096) 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 -- 172.21.15.42:0/1463078711 learned_addr learned my addr 172.21.15.42:0/1463078711 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488802f0 0x559048c5e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb36400640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 -- 172.21.15.42:0/1463078711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488802f0 msgr2=0x559048c5e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488802f0 0x559048c5e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 -- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 msgr2=0x559048c594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 -- 172.21.15.42:0/1463078711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559048c59c60 con 0x5590488884d0 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488802f0 0x559048c5e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb36400640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb35000640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 0x559048c5e640 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdb30111fe0 tx=0x7fdb2000cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e124065583ab4e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.049+0000 7fdb13400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb13400640 1 -- 172.21.15.42:0/1463078711 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2001c470 con 0x5590488884d0 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb13400640 1 -- 172.21.15.42:0/1463078711 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fdb2001c610 con 0x5590488884d0 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb240ea670 con 0x5590488884d0 2024-05-27T10:23:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb13400640 1 -- 172.21.15.42:0/1463078711 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb2002b050 con 0x5590488884d0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb3f588740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559048ccfc50 con 0x5590488884d0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.050+0000 7fdb13400640 1 -- 172.21.15.42:0/1463078711 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb240ea670 con 0x5590488884d0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb13400640 1 -- 172.21.15.42:0/1463078711 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdb20010b60 con 0x5590488884d0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb3f588740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb3f588740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb3f588740 20 client.16272 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55904883d1c0 0x559048cd2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb3f588740 4 client.16272 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559048bb82c0 con 0x55904883d1c0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.051+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55904883d1c0 0x559048cd2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.052+0000 7fdb35a00640 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55904883d1c0 0x559048cd2520 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6eca93d3b309df server_cookie=f2dfbc81b2268d32 in_seq=0 out_seq=0 2024-05-27T10:23:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.052+0000 7fdb13400640 10 client.16272 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.075+0000 7fdb13400640 1 -- 172.21.15.42:0/1463078711 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559048bb82c0 con 0x55904883d1c0 2024-05-27T10:23:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.075+0000 7fdb13400640 10 client.16272 handle_command_reply: tid=0 2024-05-27T10:23:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.075+0000 7fdb3f588740 1 client.16272 shutdown 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 msgr2=0x559048c5e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 0x559048c5e640 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdb30111fe0 tx=0x7fdb2000cdf0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 shutdown_connections 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5590488a91b0 0x559048c594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590488884d0 0x559048c5e640 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55904883d1c0 0x559048cd2520 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 --2- 172.21.15.42:0/1463078711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590488802f0 0x559048c5e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb13400640 0 client.16272 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 >> 172.21.15.42:0/1463078711 conn(0x55904889d300 msgr2=0x559048c55be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 shutdown_connections 2024-05-27T10:23:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 1 -- 172.21.15.42:0/1463078711 wait complete. 2024-05-27T10:23:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.076+0000 7fdb3f588740 20 client.16272 trim_cache size 0 max 16384 2024-05-27T10:23:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:23:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:23:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdb180b2950 msgr2=0x7fdb180b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdb180b2950 0x7fdb180b4e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009a60 tx=0x7fdb28009450 comp rx=0 tx=0).stop 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 msgr2=0x7fdb3019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 0x7fdb3019c770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c950 tx=0x7fdb2400ce10 comp rx=0 tx=0).stop 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 shutdown_connections 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fdb180b2950 0x7fdb180b4e60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb301102a0 0x7fdb301a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb3010d940 0x7fdb3019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 --2- 172.21.15.42:0/2264642016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb30076770 0x7fdb3019c770 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 >> 172.21.15.42:0/2264642016 conn(0x7fdb300ff170 msgr2=0x7fdb301106b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 shutdown_connections 2024-05-27T10:23:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:06.077+0000 7fdb36e00640 1 -- 172.21.15.42:0/2264642016 wait complete. 2024-05-27T10:23:07.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:06 smithi042 ceph-mon[31661]: pgmap v134: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-05-27T10:23:07.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:06 smithi104 ceph-mon[34812]: pgmap v134: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 -- 172.21.15.42:0/2261623759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc10a980 msgr2=0x7f94fc10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 --2- 172.21.15.42:0/2261623759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc10a980 0x7f94fc10ad90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009990 tx=0x7f94e8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 -- 172.21.15.42:0/2261623759 shutdown_connections 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 --2- 172.21.15.42:0/2261623759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc10b2d0 0x7f94fc117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 --2- 172.21.15.42:0/2261623759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc110960 0x7f94fc110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 --2- 172.21.15.42:0/2261623759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc10a980 0x7f94fc10ad90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 -- 172.21.15.42:0/2261623759 >> 172.21.15.42:0/2261623759 conn(0x7f94fc077c70 msgr2=0x7f94fc0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 -- 172.21.15.42:0/2261623759 shutdown_connections 2024-05-27T10:23:07.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.245+0000 7f9503000700 1 -- 172.21.15.42:0/2261623759 wait complete. 2024-05-27T10:23:07.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.247+0000 7f9503000700 1 Processor -- start 2024-05-27T10:23:07.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.247+0000 7f9503000700 1 -- start start 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.247+0000 7f9503000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc10a980 0x7f94fc1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.247+0000 7f9503000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc10b2d0 0x7f94fc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9503000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 0x7f94fc1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9503000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc11a980 con 0x7f94fc110960 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9503000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc11a800 con 0x7f94fc10a980 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9503000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc11ab00 con 0x7f94fc10b2d0 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9501000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 0x7f94fc1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9501000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 0x7f94fc1a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58114/0 (socket says 172.21.15.42:58114) 2024-05-27T10:23:07.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9501000700 1 -- 172.21.15.42:0/3847171077 learned_addr learned my addr 172.21.15.42:0/3847171077 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f94fbe00700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc10a980 0x7f94fc1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f94fb400700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc10b2d0 0x7f94fc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.248+0000 7f9501000700 1 -- 172.21.15.42:0/3847171077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc10b2d0 msgr2=0x7f94fc1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f9501000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc10b2d0 0x7f94fc1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f9501000700 1 -- 172.21.15.42:0/3847171077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc10a980 msgr2=0x7f94fc1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f9501000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc10a980 0x7f94fc1a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f9501000700 1 -- 172.21.15.42:0/3847171077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc1a6280 con 0x7f94fc110960 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f94fb400700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc10b2d0 0x7f94fc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f9501000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 0x7f94fc1a5be0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f94f4007ee0 tx=0x7f94f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f400dbd0 con 0x7f94fc110960 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94f400dd70 con 0x7f94fc110960 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc1a6570 con 0x7f94fc110960 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.249+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40058d0 con 0x7f94fc110960 2024-05-27T10:23:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.250+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc1ade90 con 0x7f94fc110960 2024-05-27T10:23:07.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.251+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc10c070 con 0x7f94fc110960 2024-05-27T10:23:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.253+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f94f401a020 con 0x7f94fc110960 2024-05-27T10:23:07.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.255+0000 7f94f8c00700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f94e00b2a70 0x7f94e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:07.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.255+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f94f40e2470 con 0x7f94fc110960 2024-05-27T10:23:07.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.255+0000 7f94fbe00700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f94e00b2a70 0x7f94e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:07.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.256+0000 7f94fbe00700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f94e00b2a70 0x7f94e00b4f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005950 tx=0x7f94ec0058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:07.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.258+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94f4010070 con 0x7f94fc110960 2024-05-27T10:23:07.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:06 smithi143 ceph-mon[35545]: pgmap v134: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-05-27T10:23:07.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.551+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94fc02e5e0 con 0x7f94fc110960 2024-05-27T10:23:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.553+0000 7f94f8c00700 1 -- 172.21.15.42:0/3847171077 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11881 (secure 0 0 0) 0x7f94f40a9e40 con 0x7f94fc110960 2024-05-27T10:23:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:23:07.554 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:21:52.648444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:23:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f94e00b2a70 msgr2=0x7f94e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f94e00b2a70 0x7f94e00b4f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005950 tx=0x7f94ec0058e0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 msgr2=0x7f94fc1a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 0x7f94fc1a5be0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f94f4007ee0 tx=0x7f94f400d3b0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 shutdown_connections 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f94e00b2a70 0x7f94e00b4f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94fc110960 0x7f94fc1a5be0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc10a980 0x7f94fc1a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 --2- 172.21.15.42:0/3847171077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94fc10b2d0 0x7f94fc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 >> 172.21.15.42:0/3847171077 conn(0x7f94fc077c70 msgr2=0x7f94fc07b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 shutdown_connections 2024-05-27T10:23:07.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:07.557+0000 7f9503000700 1 -- 172.21.15.42:0/3847171077 wait complete. 2024-05-27T10:23:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-05-27T10:23:08.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:07 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3847171077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:08.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:07 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3847171077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:08.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:07 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3847171077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:08.612 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id d78bfd53 --release --await' 2024-05-27T10:23:08.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id d78bfd53 --release --await 2024-05-27T10:23:08.715 INFO:tasks.workunit.client.0.smithi042.stdout: 5 0 0 0 0 0 0 2024-05-27T10:23:09.094 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:23:09.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:08 smithi104 ceph-mon[34812]: pgmap v135: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-27T10:23:09.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:08 smithi042 ceph-mon[31661]: pgmap v135: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-27T10:23:09.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:08 smithi143 ceph-mon[35545]: pgmap v135: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-27T10:23:10.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:09 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:23:10.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:09 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:23:10.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:09 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:23:11.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:10 smithi104 ceph-mon[34812]: pgmap v136: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 26 op/s 2024-05-27T10:23:11.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:10 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:23:11.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:10 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:23:11.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:10 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:23:11.300 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:10 smithi042 ceph-mon[31661]: pgmap v136: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 26 op/s 2024-05-27T10:23:11.301 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:10 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:23:11.301 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:10 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:23:11.301 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:10 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:23:11.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:10 smithi143 ceph-mon[35545]: pgmap v136: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 26 op/s 2024-05-27T10:23:11.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:10 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:23:11.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:10 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:23:11.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:10 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 -- 172.21.15.42:0/911027625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 msgr2=0x7f6c54106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 --2- 172.21.15.42:0/911027625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c54106970 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009990 tx=0x7f6c40009ca0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 -- 172.21.15.42:0/911027625 shutdown_connections 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 --2- 172.21.15.42:0/911027625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c54106eb0 0x7f6c54113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 --2- 172.21.15.42:0/911027625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 0x7f6c5410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 --2- 172.21.15.42:0/911027625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c54106970 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 -- 172.21.15.42:0/911027625 >> 172.21.15.42:0/911027625 conn(0x7f6c54102040 msgr2=0x7f6c54104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 -- 172.21.15.42:0/911027625 shutdown_connections 2024-05-27T10:23:11.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.736+0000 7f6c5a000700 1 -- 172.21.15.42:0/911027625 wait complete. 2024-05-27T10:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.737+0000 7f6c5a000700 1 Processor -- start 2024-05-27T10:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.737+0000 7f6c5a000700 1 -- start start 2024-05-27T10:23:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c5a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c5419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c5a000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c54106eb0 0x7f6c5419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c5a000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 0x7f6c541a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c5a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54116580 con 0x7f6c54106560 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c5a000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54116400 con 0x7f6c54106eb0 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c5a000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54116700 con 0x7f6c5410c5a0 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c53400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c5419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c53400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c5419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58132/0 (socket says 172.21.15.42:58132) 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c53400700 1 -- 172.21.15.42:0/1300280449 learned_addr learned my addr 172.21.15.42:0/1300280449 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c52a00700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c54106eb0 0x7f6c5419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.738+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 0x7f6c541a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53400700 1 -- 172.21.15.42:0/1300280449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 msgr2=0x7f6c541a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53400700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 0x7f6c541a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53400700 1 -- 172.21.15.42:0/1300280449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c54106eb0 msgr2=0x7f6c5419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53400700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c54106eb0 0x7f6c5419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53400700 1 -- 172.21.15.42:0/1300280449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c541a1e00 con 0x7f6c54106560 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 0x7f6c541a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c53400700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c5419cf30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d980 tx=0x7f6c4400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c4be00700 1 -- 172.21.15.42:0/1300280449 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44017070 con 0x7f6c54106560 2024-05-27T10:23:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c4be00700 1 -- 172.21.15.42:0/1300280449 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c44010460 con 0x7f6c54106560 2024-05-27T10:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c541a20f0 con 0x7f6c54106560 2024-05-27T10:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c4be00700 1 -- 172.21.15.42:0/1300280449 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400de60 con 0x7f6c54106560 2024-05-27T10:23:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.739+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541a9a10 con 0x7f6c54106560 2024-05-27T10:23:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.745+0000 7f6c4be00700 1 -- 172.21.15.42:0/1300280449 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c44010610 con 0x7f6c54106560 2024-05-27T10:23:11.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.746+0000 7f6c68bf2b80 1 Processor -- start 2024-05-27T10:23:11.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.746+0000 7f6c68bf2b80 1 -- start start 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcbd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcbda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f83dbe4f40 con 0x55f83dc23af0 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f83dbd4ae0 con 0x55f83dcb75e0 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f83dbd8b20 con 0x55f83db37f40 2024-05-27T10:23:11.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c4be00700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6c380b29f0 0x7f6c380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c4be00700 1 -- 172.21.15.42:0/1300280449 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6c440e20f0 con 0x7f6c54106560 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c53e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcbd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c53e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcbd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58142/0 (socket says 172.21.15.42:58142) 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c53e00700 1 -- 172.21.15.42:0/1754540991 learned_addr learned my addr 172.21.15.42:0/1754540991 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c53400700 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcbda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6c380b29f0 0x7f6c380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.747+0000 7f6c52a00700 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.748+0000 7f6c53e00700 1 -- 172.21.15.42:0/1754540991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 msgr2=0x55f83dcbda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.748+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcbda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.748+0000 7f6c53e00700 1 -- 172.21.15.42:0/1754540991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 msgr2=0x55f83dcb79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.748+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcb79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.748+0000 7f6c53e00700 1 -- 172.21.15.42:0/1754540991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83db9d520 con 0x55f83dc23af0 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.748+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcbd530 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007cf0 tx=0x7f6c4c0172f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9635e3cb02a347 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.749+0000 7f6c53e00700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6c380b29f0 0x7f6c380b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c017640 tx=0x7f6c4c00b000 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.749+0000 7f6c48c00700 1 -- 172.21.15.42:0/1754540991 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c01e070 con 0x55f83dc23af0 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.749+0000 7f6c48c00700 1 -- 172.21.15.42:0/1754540991 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c4c007500 con 0x55f83dc23af0 2024-05-27T10:23:11.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.749+0000 7f6c48c00700 1 -- 172.21.15.42:0/1754540991 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c01a470 con 0x55f83dc23af0 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/1754540991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 msgr2=0x55f83dcbd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcbd530 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007cf0 tx=0x7f6c4c0172f0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/1754540991 shutdown_connections 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcb79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcbd530 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/1754540991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcbda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/1754540991 >> 172.21.15.42:0/1754540991 conn(0x55f83dbd1490 msgr2=0x55f83dcb2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/1754540991 shutdown_connections 2024-05-27T10:23:11.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.750+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/1754540991 wait complete. 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.751+0000 7f6c68bf2b80 1 -- start start 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.751+0000 7f6c68bf2b80 1 Processor -- start 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcc6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcc74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcc1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f83dadb0a0 con 0x55f83dc23af0 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f83db75860 con 0x55f83dcb75e0 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f83dcb8310 con 0x55f83db37f40 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c68bf2b80 10 client.0 _ll_register_callbacks cb 0x55f83db7ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcc74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcc74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58150/0 (socket says 172.21.15.42:58150) 2024-05-27T10:23:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 -- 172.21.15.42:0/494937830 learned_addr learned my addr 172.21.15.42:0/494937830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c53400700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcc6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c53e00700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcc1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 -- 172.21.15.42:0/494937830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 msgr2=0x55f83dcc6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcc6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 -- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 msgr2=0x55f83dcc1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcc1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.752+0000 7f6c52a00700 1 -- 172.21.15.42:0/494937830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83db249d0 con 0x55f83dc23af0 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.753+0000 7f6c52a00700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcc74d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6c54075b70 tx=0x7f6c40009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6696b5fa46b139bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.753+0000 7f6c3d600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.753+0000 7f6c3d600700 1 -- 172.21.15.42:0/494937830 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003bd0 con 0x55f83dc23af0 2024-05-27T10:23:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.753+0000 7f6c3d600700 1 -- 172.21.15.42:0/494937830 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6c40003d70 con 0x55f83dc23af0 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.753+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4c007500 con 0x55f83dc23af0 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.753+0000 7f6c3d600700 1 -- 172.21.15.42:0/494937830 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c40030050 con 0x55f83dc23af0 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 10 client.16290 fetch_fsmap learned FSMap version 29 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83dba5160 con 0x55f83dc23af0 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c3d600700 1 -- 172.21.15.42:0/494937830 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c4c007500 con 0x55f83dc23af0 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c3d600700 1 -- 172.21.15.42:0/494937830 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6c40022560 con 0x55f83dc23af0 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 10 client.16290 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 10 client.16290 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 20 client.16290 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dcd1070 0x55f83dcd34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 4 client.16290 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.754+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83dbe6e90 con 0x55f83dcd1070 2024-05-27T10:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.755+0000 7f6c53400700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dcd1070 0x55f83dcd34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.756+0000 7f6c53400700 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dcd1070 0x55f83dcd34c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84b59b954b5acf2 server_cookie=b89e6f89f0e2a674 in_seq=0 out_seq=0 2024-05-27T10:23:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.756+0000 7f6c3d600700 10 client.16290 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.758+0000 7f6c3d600700 1 -- 172.21.15.42:0/494937830 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f83dbe6e90 con 0x55f83dcd1070 2024-05-27T10:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.758+0000 7f6c3d600700 10 client.16290 handle_command_reply: tid=0 2024-05-27T10:23:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.758+0000 7f6c68bf2b80 1 client.16290 shutdown 2024-05-27T10:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.758+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 msgr2=0x55f83dcc74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.758+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcc74d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6c54075b70 tx=0x7f6c40009ee0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 shutdown_connections 2024-05-27T10:23:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dcd1070 0x55f83dcd34c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dcc1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dcc6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/494937830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dcc74d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 >> 172.21.15.42:0/494937830 conn(0x55f83dbd1490 msgr2=0x55f83dcb65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c3d600700 0 client.16290 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 shutdown_connections 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/494937830 wait complete. 2024-05-27T10:23:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.759+0000 7f6c68bf2b80 20 client.16290 trim_cache size 0 max 16384 2024-05-27T10:23:11.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.782+0000 7f6c68bf2b80 1 Processor -- start 2024-05-27T10:23:11.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.782+0000 7f6c68bf2b80 1 -- start start 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 0x55f83dcb7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dc23af0 0x55f83dcb1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83dcb75e0 0x55f83dcb24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b3730 con 0x55f83db37f40 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b0fc0 con 0x55f83dc23af0 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541aee10 con 0x55f83dcb75e0 2024-05-27T10:23:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 0x55f83dcb7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 0x55f83dcb7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58158/0 (socket says 172.21.15.42:58158) 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 -- 172.21.15.42:0/3949177405 learned_addr learned my addr 172.21.15.42:0/3949177405 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c52a00700 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dc23af0 0x55f83dcb1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 -- 172.21.15.42:0/3949177405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83dcb75e0 msgr2=0x55f83dcb24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53e00700 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83dcb75e0 0x55f83dcb24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83dcb75e0 0x55f83dcb24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 -- 172.21.15.42:0/3949177405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dc23af0 msgr2=0x55f83dcb1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.783+0000 7f6c53400700 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dc23af0 0x55f83dcb1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.784+0000 7f6c53400700 1 -- 172.21.15.42:0/3949177405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83dcbf900 con 0x55f83db37f40 2024-05-27T10:23:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.784+0000 7f6c53400700 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 0x55f83dcb7000 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44091050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a228ac1baffcbba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.784+0000 7f6c3fe00700 1 -- 172.21.15.42:0/3949177405 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44020040 con 0x55f83db37f40 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.784+0000 7f6c3fe00700 1 -- 172.21.15.42:0/3949177405 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c440b22c0 con 0x55f83db37f40 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.784+0000 7f6c3fe00700 1 -- 172.21.15.42:0/3949177405 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c440b1e30 con 0x55f83db37f40 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/3949177405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 msgr2=0x55f83dcb7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 0x55f83dcb7000 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44091050 comp rx=0 tx=0).stop 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/3949177405 shutdown_connections 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83dcb75e0 0x55f83dcb24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83db37f40 0x55f83dcb7000 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/3949177405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dc23af0 0x55f83dcb1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/3949177405 >> 172.21.15.42:0/3949177405 conn(0x55f83dbd1490 msgr2=0x55f83dcc02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/3949177405 shutdown_connections 2024-05-27T10:23:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.785+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/3949177405 wait complete. 2024-05-27T10:23:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.786+0000 7f6c68bf2b80 1 -- start start 2024-05-27T10:23:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.786+0000 7f6c68bf2b80 1 Processor -- start 2024-05-27T10:23:11.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.786+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dccd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dccdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c68bf2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dccba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541af480 con 0x55f83dc23af0 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b08f0 con 0x55f83dcb75e0 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c68bf2b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541afb50 con 0x55f83db37f40 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c68bf2b80 10 client.0 _ll_register_callbacks cb 0x55f83db7ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dccdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dccdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58162/0 (socket says 172.21.15.42:58162) 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 -- 172.21.15.42:0/2622363083 learned_addr learned my addr 172.21.15.42:0/2622363083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c53400700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dccd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c53e00700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dccba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 -- 172.21.15.42:0/2622363083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 msgr2=0x55f83dccd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dccd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 -- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 msgr2=0x55f83dccba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dccba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c52a00700 1 -- 172.21.15.42:0/2622363083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83dccc1a0 con 0x55f83dc23af0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.787+0000 7f6c53e00700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dccba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c52a00700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dccdac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55f83dcacc40 tx=0x7f6c40005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f518a8757758a002 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c3d600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c3d600700 1 -- 172.21.15.42:0/2622363083 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400039d0 con 0x55f83dc23af0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c440b22c0 con 0x55f83dc23af0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c3d600700 1 -- 172.21.15.42:0/2622363083 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f6c40003b70 con 0x55f83dc23af0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c3d600700 1 -- 172.21.15.42:0/2622363083 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c40003d70 con 0x55f83dc23af0 2024-05-27T10:23:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c68bf2b80 10 client.16296 fetch_fsmap learned FSMap version 29 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.788+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83dd3c690 con 0x55f83dc23af0 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c3d600700 1 -- 172.21.15.42:0/2622363083 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c440b22c0 con 0x55f83dc23af0 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c3d600700 1 -- 172.21.15.42:0/2622363083 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6c40034550 con 0x55f83dc23af0 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c68bf2b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c68bf2b80 10 client.16296 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c68bf2b80 20 client.16296 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dd3ccd0 0x55f83dd3f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c68bf2b80 4 client.16296 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d78bfd53", "release": true, "await": true}] 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.789+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d78bfd53", "release": true, "await": true}) -- 0x55f83dcaa810 con 0x55f83dd3ccd0 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.790+0000 7f6c53400700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dd3ccd0 0x55f83dd3f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.790+0000 7f6c53400700 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dd3ccd0 0x55f83dd3f0f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9429e8ffc65abd86 server_cookie=fb41b6d34190e780 in_seq=0 out_seq=0 2024-05-27T10:23:11.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.791+0000 7f6c3d600700 10 client.16296 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c3d600700 1 -- 172.21.15.42:0/2622363083 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f83dcaa810 con 0x55f83dd3ccd0 2024-05-27T10:23:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c3d600700 10 client.16296 handle_command_reply: tid=0 2024-05-27T10:23:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 client.16296 shutdown 2024-05-27T10:23:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 msgr2=0x55f83dccdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dccdac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55f83dcacc40 tx=0x7f6c40005a00 comp rx=0 tx=0).stop 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 shutdown_connections 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f83dd3ccd0 0x55f83dd3f0f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83dcb75e0 0x55f83dccba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c3d600700 0 client.16296 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f83db37f40 0x55f83dccd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 --2- 172.21.15.42:0/2622363083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f83dc23af0 0x55f83dccdac0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 >> 172.21.15.42:0/2622363083 conn(0x55f83dbd1490 msgr2=0x55f83dcbafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 shutdown_connections 2024-05-27T10:23:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.801+0000 7f6c68bf2b80 1 -- 172.21.15.42:0/2622363083 wait complete. 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.802+0000 7f6c68bf2b80 20 client.16296 trim_cache size 0 max 16384 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 29, 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 12, 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:23:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "d78bfd53": { 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 12, 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 150.0, 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:23:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:23:11.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.802+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6c380b29f0 msgr2=0x7f6c380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.802+0000 7f6c5a000700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6c380b29f0 0x7f6c380b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c017640 tx=0x7f6c4c00b000 comp rx=0 tx=0).stop 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.802+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 msgr2=0x7f6c5419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.802+0000 7f6c5a000700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c5419cf30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d980 tx=0x7f6c4400dd40 comp rx=0 tx=0).stop 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 shutdown_connections 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6c380b29f0 0x7f6c380b4f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c5410c5a0 0x7f6c541a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c54106560 0x7f6c5419cf30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 --2- 172.21.15.42:0/1300280449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c54106eb0 0x7f6c5419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 >> 172.21.15.42:0/1300280449 conn(0x7f6c54102040 msgr2=0x7f6c541038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 shutdown_connections 2024-05-27T10:23:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:11.803+0000 7f6c5a000700 1 -- 172.21.15.42:0/1300280449 wait complete. 2024-05-27T10:23:12.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:11 smithi143 ceph-mon[35545]: pgmap v137: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-05-27T10:23:12.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:11 smithi042 ceph-mon[31661]: pgmap v137: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-05-27T10:23:12.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:11 smithi104 ceph-mon[34812]: pgmap v137: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-05-27T10:23:12.912 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd78bfd53', total seconds elapsed: 38.8 2024-05-27T10:23:12.912 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-27T10:23:12.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:23:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:23:14.162 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:14 smithi042 ceph-mon[31661]: pgmap v138: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-05-27T10:23:14.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:14 smithi104 ceph-mon[34812]: pgmap v138: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-05-27T10:23:14.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:14 smithi143 ceph-mon[35545]: pgmap v138: 97 pgs: 97 active+clean; 42 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-05-27T10:23:16.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:16 smithi042 ceph-mon[31661]: pgmap v139: 97 pgs: 97 active+clean; 45 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 321 KiB/s wr, 42 op/s 2024-05-27T10:23:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 -- 172.21.15.42:0/3265641951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e00105e60 msgr2=0x7f5e0010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 --2- 172.21.15.42:0/3265641951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e00105e60 0x7f5e0010e3b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0098f0 tx=0x7f5dec009c00 comp rx=0 tx=0).stop 2024-05-27T10:23:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 -- 172.21.15.42:0/3265641951 shutdown_connections 2024-05-27T10:23:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 --2- 172.21.15.42:0/3265641951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0010e8f0 0x7f5e00110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 --2- 172.21.15.42:0/3265641951 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e00105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 --2- 172.21.15.42:0/3265641951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e00105e60 0x7f5e0010e3b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.353+0000 7f5e07600700 1 -- 172.21.15.42:0/3265641951 >> 172.21.15.42:0/3265641951 conn(0x7f5e000fedb0 msgr2=0x7f5e001011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.354+0000 7f5e07600700 1 -- 172.21.15.42:0/3265641951 shutdown_connections 2024-05-27T10:23:16.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.354+0000 7f5e07600700 1 -- 172.21.15.42:0/3265641951 wait complete. 2024-05-27T10:23:16.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.356+0000 7f5e07600700 1 Processor -- start 2024-05-27T10:23:16.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.357+0000 7f5e07600700 1 -- start start 2024-05-27T10:23:16.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.357+0000 7f5e07600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e0019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:16.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.357+0000 7f5e07600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e00105e60 0x7f5e0019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:16.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e07600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e0010e8f0 0x7f5e001a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:16.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e07600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00114330 con 0x7f5e00105e60 2024-05-27T10:23:16.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e07600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001141b0 con 0x7f5e00105510 2024-05-27T10:23:16.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e07600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001144b0 con 0x7f5e0010e8f0 2024-05-27T10:23:16.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e04c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e0019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:16.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e04c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e0019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:59950/0 (socket says 172.21.15.42:59950) 2024-05-27T10:23:16.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.358+0000 7f5e04c00700 1 -- 172.21.15.42:0/531101420 learned_addr learned my addr 172.21.15.42:0/531101420 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:16.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5e05600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e0010e8f0 0x7f5e001a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:16.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5dffe00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e00105e60 0x7f5e0019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:16.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5e04c00700 1 -- 172.21.15.42:0/531101420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e0010e8f0 msgr2=0x7f5e001a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:16.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5e04c00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e0010e8f0 0x7f5e001a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5e04c00700 1 -- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e00105e60 msgr2=0x7f5e0019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:16.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5e04c00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e00105e60 0x7f5e0019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5e04c00700 1 -- 172.21.15.42:0/531101420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001a1dc0 con 0x7f5e00105510 2024-05-27T10:23:16.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.359+0000 7f5dffe00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e00105e60 0x7f5e0019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:16.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.360+0000 7f5e04c00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e0019cfa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c9b0 tx=0x7f5df400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.360+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4007b00 con 0x7f5e00105510 2024-05-27T10:23:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.360+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df4007ca0 con 0x7f5e00105510 2024-05-27T10:23:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.360+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e001a20b0 con 0x7f5e00105510 2024-05-27T10:23:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.360+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4018660 con 0x7f5e00105510 2024-05-27T10:23:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.361+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001a9a80 con 0x7f5e00105510 2024-05-27T10:23:16.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.362+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e000512b0 con 0x7f5e00105510 2024-05-27T10:23:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.371+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5df400f450 con 0x7f5e00105510 2024-05-27T10:23:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.373+0000 7f5dfd600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5de80b2af0 0x7f5de80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.374+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5df40e2900 con 0x7f5e00105510 2024-05-27T10:23:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.374+0000 7f5dffe00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5de80b2af0 0x7f5de80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:16.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.375+0000 7f5dffe00700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5de80b2af0 0x7f5de80b5000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005f40 tx=0x7f5dec009f50 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.379+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5df4014030 con 0x7f5e00105510 2024-05-27T10:23:16.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:16 smithi104 ceph-mon[34812]: pgmap v139: 97 pgs: 97 active+clean; 45 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 321 KiB/s wr, 42 op/s 2024-05-27T10:23:16.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:16 smithi143 ceph-mon[35545]: pgmap v139: 97 pgs: 97 active+clean; 45 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 321 KiB/s wr, 42 op/s 2024-05-27T10:23:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.876+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0019e300 con 0x7f5e00105510 2024-05-27T10:23:16.879 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:23:16.879 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:21:52.648444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34442,34466,34499]},"id":1}]} 2024-05-27T10:23:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.879+0000 7f5dfd600700 1 -- 172.21.15.42:0/531101420 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11881 (secure 0 0 0) 0x7f5df40aa3e0 con 0x7f5e00105510 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.881+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5de80b2af0 msgr2=0x7f5de80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.881+0000 7f5e07600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5de80b2af0 0x7f5de80b5000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005f40 tx=0x7f5dec009f50 comp rx=0 tx=0).stop 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.881+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 msgr2=0x7f5e0019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.881+0000 7f5e07600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e0019cfa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c9b0 tx=0x7f5df400cd70 comp rx=0 tx=0).stop 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.881+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 shutdown_connections 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.881+0000 7f5e07600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5de80b2af0 0x7f5de80b5000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.882+0000 7f5e07600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e0010e8f0 0x7f5e001a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.882+0000 7f5e07600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e00105510 0x7f5e0019cfa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.882+0000 7f5e07600700 1 --2- 172.21.15.42:0/531101420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e00105e60 0x7f5e0019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.882+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 >> 172.21.15.42:0/531101420 conn(0x7f5e000fedb0 msgr2=0x7f5e00103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.882+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 shutdown_connections 2024-05-27T10:23:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:16.882+0000 7f5e07600700 1 -- 172.21.15.42:0/531101420 wait complete. 2024-05-27T10:23:16.885 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-05-27T10:23:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:17 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/531101420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:17.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:17 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/531101420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:17.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:17 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/531101420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:17.953 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-27T10:23:17.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-27T10:23:18.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:18 smithi042 ceph-mon[31661]: pgmap v140: 97 pgs: 97 active+clean; 45 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 298 KiB/s wr, 25 op/s 2024-05-27T10:23:18.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:18 smithi104 ceph-mon[34812]: pgmap v140: 97 pgs: 97 active+clean; 45 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 298 KiB/s wr, 25 op/s 2024-05-27T10:23:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:23:18.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:18 smithi143 ceph-mon[35545]: pgmap v140: 97 pgs: 97 active+clean; 45 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 298 KiB/s wr, 25 op/s 2024-05-27T10:23:18.715 INFO:tasks.workunit.client.0.smithi042.stdout: 8 4002 242 2160 140 1103 87 2024-05-27T10:23:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 -- 172.21.15.42:0/1523434181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10be90 msgr2=0x7fe78c105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 --2- 172.21.15.42:0/1523434181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10be90 0x7fe78c105a30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe778009900 tx=0x7fe77802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 -- 172.21.15.42:0/1523434181 shutdown_connections 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 --2- 172.21.15.42:0/1523434181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe78c1065e0 0x7fe78c1129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 --2- 172.21.15.42:0/1523434181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c10c330 0x7fe78c105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 --2- 172.21.15.42:0/1523434181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10be90 0x7fe78c105a30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 -- 172.21.15.42:0/1523434181 >> 172.21.15.42:0/1523434181 conn(0x7fe78c077120 msgr2=0x7fe78c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 -- 172.21.15.42:0/1523434181 shutdown_connections 2024-05-27T10:23:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.389+0000 7fe792a00640 1 -- 172.21.15.42:0/1523434181 wait complete. 2024-05-27T10:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 Processor -- start 2024-05-27T10:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 -- start start 2024-05-27T10:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 0x7fe78c1a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10c330 0x7fe78c1a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe78c19c8d0 0x7fe78c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c115f90 con 0x7fe78c10c330 2024-05-27T10:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c115e10 con 0x7fe78c1065e0 2024-05-27T10:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.391+0000 7fe792a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c116110 con 0x7fe78c19c8d0 2024-05-27T10:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.392+0000 7fe78be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 0x7fe78c1a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.392+0000 7fe78be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 0x7fe78c1a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:59966/0 (socket says 172.21.15.42:59966) 2024-05-27T10:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.392+0000 7fe78be00640 1 -- 172.21.15.42:0/4006717058 learned_addr learned my addr 172.21.15.42:0/4006717058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.392+0000 7fe78b400640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10c330 0x7fe78c1a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.392+0000 7fe790a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe78c19c8d0 0x7fe78c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.393+0000 7fe78be00640 1 -- 172.21.15.42:0/4006717058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe78c19c8d0 msgr2=0x7fe78c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.393+0000 7fe78be00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe78c19c8d0 0x7fe78c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.393+0000 7fe78be00640 1 -- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10c330 msgr2=0x7fe78c1a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.393+0000 7fe78be00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10c330 0x7fe78c1a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.393+0000 7fe78be00640 1 -- 172.21.15.42:0/4006717058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c19d460 con 0x7fe78c1065e0 2024-05-27T10:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.393+0000 7fe78be00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 0x7fe78c1a2780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe778006010 tx=0x7fe778003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.394+0000 7fe788c00640 1 -- 172.21.15.42:0/4006717058 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77803f020 con 0x7fe78c1065e0 2024-05-27T10:23:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.394+0000 7fe788c00640 1 -- 172.21.15.42:0/4006717058 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe77803bb00 con 0x7fe78c1065e0 2024-05-27T10:23:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.394+0000 7fe788c00640 1 -- 172.21.15.42:0/4006717058 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778035070 con 0x7fe78c1065e0 2024-05-27T10:23:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.394+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c19d6f0 con 0x7fe78c1065e0 2024-05-27T10:23:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.394+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1a97e0 con 0x7fe78c1065e0 2024-05-27T10:23:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.398+0000 7fe799a2c740 1 Processor -- start 2024-05-27T10:23:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.398+0000 7fe799a2c740 1 -- start start 2024-05-27T10:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.398+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aee91b0 0x56369b28b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.398+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec02f0 0x56369b28bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.398+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 0x56369b28c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe799a2c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56369b1fcb90 con 0x56369aec02f0 2024-05-27T10:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe799a2c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56369b1ec730 con 0x56369aec84d0 2024-05-27T10:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe799a2c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56369b1f0770 con 0x56369aee91b0 2024-05-27T10:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe78b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 0x56369b28c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe790a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aee91b0 0x56369b28b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe78b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 0x56369b28c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:59968/0 (socket says 172.21.15.42:59968) 2024-05-27T10:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.399+0000 7fe78b400640 1 -- 172.21.15.42:0/2457828482 learned_addr learned my addr 172.21.15.42:0/2457828482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe78b400640 1 -- 172.21.15.42:0/2457828482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aee91b0 msgr2=0x56369b28b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe78b400640 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aee91b0 0x56369b28b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe78b400640 1 -- 172.21.15.42:0/2457828482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec02f0 msgr2=0x56369b28bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe78b400640 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec02f0 0x56369b28bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe78b400640 1 -- 172.21.15.42:0/2457828482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369b1858b0 con 0x56369aec84d0 2024-05-27T10:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe78b400640 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 0x56369b28c2e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00bb70 tx=0x7fe77c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301f9a78952f04fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.400+0000 7fe788c00640 1 -- 172.21.15.42:0/4006717058 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe778042020 con 0x7fe78c1065e0 2024-05-27T10:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.402+0000 7fe781200640 1 -- 172.21.15.42:0/2457828482 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c004030 con 0x56369aec84d0 2024-05-27T10:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.402+0000 7fe781200640 1 -- 172.21.15.42:0/2457828482 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe77c005020 con 0x56369aec84d0 2024-05-27T10:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.402+0000 7fe781200640 1 -- 172.21.15.42:0/2457828482 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c004b40 con 0x56369aec84d0 2024-05-27T10:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.402+0000 7fe788c00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe7700b29d0 0x7fe7700b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe788c00640 1 -- 172.21.15.42:0/4006717058 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe778102de0 con 0x7fe78c1065e0 2024-05-27T10:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457828482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 msgr2=0x56369b28c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 0x56369b28c2e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00bb70 tx=0x7fe77c0107a0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe790a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe7700b29d0 0x7fe7700b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457828482 shutdown_connections 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec84d0 0x56369b28c2e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec02f0 0x56369b28bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457828482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aee91b0 0x56369b28b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.403+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457828482 >> 172.21.15.42:0/2457828482 conn(0x56369aedd300 msgr2=0x56369b288010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.404+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457828482 shutdown_connections 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.404+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457828482 wait complete. 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.404+0000 7fe790a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe7700b29d0 0x7fe7700b4ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x56369b2889d0 tx=0x7fe784009500 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.405+0000 7fe799a2c740 1 -- start start 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.405+0000 7fe799a2c740 1 Processor -- start 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aec02f0 0x56369b295a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 0x56369b295f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aee91b0 0x56369b2aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56369aed2790 con 0x56369aec84d0 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56369b194bc0 con 0x56369aee91b0 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56369b298990 con 0x56369aec02f0 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe799a2c740 10 client.0 _ll_register_callbacks cb 0x56369b172400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe78b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 0x56369b295f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe790a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aee91b0 0x56369b2aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe78b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 0x56369b295f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49396/0 (socket says 172.21.15.42:49396) 2024-05-27T10:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe78b400640 1 -- 172.21.15.42:0/1567836755 learned_addr learned my addr 172.21.15.42:0/1567836755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe790a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aee91b0 0x56369b2aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:59976/0 (socket says 172.21.15.42:59976) 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.406+0000 7fe78be00640 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aec02f0 0x56369b295a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.407+0000 7fe78b400640 1 -- 172.21.15.42:0/1567836755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aec02f0 msgr2=0x56369b295a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.407+0000 7fe78b400640 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aec02f0 0x56369b295a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.407+0000 7fe78b400640 1 -- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aee91b0 msgr2=0x56369b2aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.407+0000 7fe78b400640 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aee91b0 0x56369b2aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.407+0000 7fe78b400640 1 -- 172.21.15.42:0/1567836755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369af01210 con 0x56369aec84d0 2024-05-27T10:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.407+0000 7fe78b400640 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 0x56369b295f50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe77c011010 tx=0x7fe77c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f839e773649ad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe775600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe775600640 1 -- 172.21.15.42:0/1567836755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0053e0 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77c005020 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe775600640 1 -- 172.21.15.42:0/1567836755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe77c022070 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe775600640 1 -- 172.21.15.42:0/1567836755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe77c0053e0 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe799a2c740 10 client.16308 fetch_fsmap learned FSMap version 29 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.408+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56369afcd620 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.409+0000 7fe775600640 1 -- 172.21.15.42:0/1567836755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe77c005020 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.409+0000 7fe775600640 1 -- 172.21.15.42:0/1567836755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fe77c0035f0 con 0x56369aec84d0 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe799a2c740 10 client.16308 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe799a2c740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe799a2c740 20 client.16308 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe799a2c740 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369aea7d80 0x56369b29e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe799a2c740 4 client.16308 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56369b1feae0 con 0x56369aea7d80 2024-05-27T10:23:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.410+0000 7fe78be00640 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369aea7d80 0x56369b29e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.411+0000 7fe78be00640 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369aea7d80 0x56369b29e650 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd29e3d89db6b6f4 server_cookie=fa7979ae1065095b in_seq=0 out_seq=0 2024-05-27T10:23:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.411+0000 7fe775600640 10 client.16308 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.412+0000 7fe775600640 1 -- 172.21.15.42:0/1567836755 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56369b1feae0 con 0x56369aea7d80 2024-05-27T10:23:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.412+0000 7fe775600640 10 client.16308 handle_command_reply: tid=0 2024-05-27T10:23:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 client.16308 shutdown 2024-05-27T10:23:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 msgr2=0x56369b295f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 0x56369b295f50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe77c011010 tx=0x7fe77c00b880 comp rx=0 tx=0).stop 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 shutdown_connections 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aee91b0 0x56369b2aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aec84d0 0x56369b295f50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369aea7d80 0x56369b29e650 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 --2- 172.21.15.42:0/1567836755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369aec02f0 0x56369b295a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 >> 172.21.15.42:0/1567836755 conn(0x56369aedd300 msgr2=0x56369b28d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.413+0000 7fe775600640 0 client.16308 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.414+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 shutdown_connections 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.414+0000 7fe799a2c740 1 -- 172.21.15.42:0/1567836755 wait complete. 2024-05-27T10:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.414+0000 7fe799a2c740 20 client.16308 trim_cache size 0 max 16384 2024-05-27T10:23:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.437+0000 7fe799a2c740 1 Processor -- start 2024-05-27T10:23:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.437+0000 7fe799a2c740 1 -- start start 2024-05-27T10:23:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.438+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b297c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.438+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b28c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.438+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe799a2c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b0560 con 0x56369aee91b0 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe799a2c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1ade30 con 0x56369aec02f0 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe799a2c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1abcc0 con 0x56369ae7d1c0 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b297c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b297c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:59978/0 (socket says 172.21.15.42:59978) 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 -- 172.21.15.42:0/2457095836 learned_addr learned my addr 172.21.15.42:0/2457095836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78b400640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b28c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe790a00640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 -- 172.21.15.42:0/2457095836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 msgr2=0x56369b28f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 -- 172.21.15.42:0/2457095836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 msgr2=0x56369b28c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b28c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe78be00640 1 -- 172.21.15.42:0/2457095836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369b282050 con 0x56369aec02f0 2024-05-27T10:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.439+0000 7fe790a00640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.440+0000 7fe78b400640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b28c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.440+0000 7fe78be00640 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b297c00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe778100580 tx=0x7fe77802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bae7b14096b904c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.440+0000 7fe777e00640 1 -- 172.21.15.42:0/2457095836 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780e3070 con 0x56369aec02f0 2024-05-27T10:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.441+0000 7fe777e00640 1 -- 172.21.15.42:0/2457095836 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7780ce460 con 0x56369aec02f0 2024-05-27T10:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.441+0000 7fe777e00640 1 -- 172.21.15.42:0/2457095836 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780c6340 con 0x56369aec02f0 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.441+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457095836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 msgr2=0x56369b297c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.441+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b297c00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe778100580 tx=0x7fe77802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457095836 shutdown_connections 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b28c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 --2- 172.21.15.42:0/2457095836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b297c00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457095836 >> 172.21.15.42:0/2457095836 conn(0x56369aedd300 msgr2=0x56369b295cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457095836 shutdown_connections 2024-05-27T10:23:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 -- 172.21.15.42:0/2457095836 wait complete. 2024-05-27T10:23:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.442+0000 7fe799a2c740 1 -- start start 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.443+0000 7fe799a2c740 1 Processor -- start 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.443+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.443+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b28fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.443+0000 7fe799a2c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b290090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe799a2c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1ac310 con 0x56369aee91b0 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe799a2c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1ad760 con 0x56369aec02f0 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe799a2c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1ac9c0 con 0x56369ae7d1c0 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe799a2c740 10 client.0 _ll_register_callbacks cb 0x7fe7840099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe78be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe78be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:55094/0 (socket says 172.21.15.42:55094) 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe78be00640 1 -- 172.21.15.42:0/3249563625 learned_addr learned my addr 172.21.15.42:0/3249563625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe790a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b290090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.444+0000 7fe78b400640 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b28fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe790a00640 1 -- 172.21.15.42:0/3249563625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 msgr2=0x56369b28f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe790a00640 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe790a00640 1 -- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 msgr2=0x56369b28fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe790a00640 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b28fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe790a00640 1 -- 172.21.15.42:0/3249563625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369b31f980 con 0x56369aee91b0 2024-05-27T10:23:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe790a00640 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b290090 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe7840049d0 tx=0x7fe784004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7aedbf4f327724 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.445+0000 7fe775600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.446+0000 7fe775600640 1 -- 172.21.15.42:0/3249563625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78401db20 con 0x56369aee91b0 2024-05-27T10:23:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.446+0000 7fe775600640 1 -- 172.21.15.42:0/3249563625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe78401dcc0 con 0x56369aee91b0 2024-05-27T10:23:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.446+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7780ce460 con 0x56369aee91b0 2024-05-27T10:23:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.446+0000 7fe775600640 1 -- 172.21.15.42:0/3249563625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe784036050 con 0x56369aee91b0 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.446+0000 7fe799a2c740 10 client.16311 fetch_fsmap learned FSMap version 29 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.446+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56369b320960 con 0x56369aee91b0 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe775600640 1 -- 172.21.15.42:0/3249563625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7780ce460 con 0x56369aee91b0 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe775600640 1 -- 172.21.15.42:0/3249563625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fe7840283f0 con 0x56369aee91b0 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe799a2c740 10 client.16311 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe799a2c740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe799a2c740 20 client.16311 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe799a2c740 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369b3211c0 0x56369b3235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe799a2c740 4 client.16311 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:23:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.447+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56369b1f82c0 con 0x56369b3211c0 2024-05-27T10:23:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.448+0000 7fe78be00640 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369b3211c0 0x56369b3235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.449+0000 7fe78be00640 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369b3211c0 0x56369b3235e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30af678354acebc0 server_cookie=fb4fca4ff99b3f0b in_seq=0 out_seq=0 2024-05-27T10:23:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.449+0000 7fe775600640 10 client.16311 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.466+0000 7fe775600640 1 -- 172.21.15.42:0/3249563625 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56369b1f82c0 con 0x56369b3211c0 2024-05-27T10:23:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.466+0000 7fe775600640 10 client.16311 handle_command_reply: tid=0 2024-05-27T10:23:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.466+0000 7fe799a2c740 1 client.16311 shutdown 2024-05-27T10:23:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 msgr2=0x56369b290090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b290090 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe7840049d0 tx=0x7fe784004ab0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 shutdown_connections 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56369aee91b0 0x56369b290090 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56369aec02f0 0x56369b28fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56369b3211c0 0x56369b3235e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 --2- 172.21.15.42:0/3249563625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56369ae7d1c0 0x56369b28f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 >> 172.21.15.42:0/3249563625 conn(0x56369aedd300 msgr2=0x56369b295a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe775600640 0 client.16311 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.467+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 shutdown_connections 2024-05-27T10:23:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.468+0000 7fe799a2c740 1 -- 172.21.15.42:0/3249563625 wait complete. 2024-05-27T10:23:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.468+0000 7fe799a2c740 20 client.16311 trim_cache size 0 max 16384 2024-05-27T10:23:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:23:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:23:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:23:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.469+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe7700b29d0 msgr2=0x7fe7700b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.469+0000 7fe792a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe7700b29d0 0x7fe7700b4ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x56369b2889d0 tx=0x7fe784009500 comp rx=0 tx=0).stop 2024-05-27T10:23:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.469+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 msgr2=0x7fe78c1a2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.469+0000 7fe792a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 0x7fe78c1a2780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe778006010 tx=0x7fe778003c10 comp rx=0 tx=0).stop 2024-05-27T10:23:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 shutdown_connections 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe7700b29d0 0x7fe7700b4ee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe78c19c8d0 0x7fe78c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe78c10c330 0x7fe78c1a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 --2- 172.21.15.42:0/4006717058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe78c1065e0 0x7fe78c1a2780 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 >> 172.21.15.42:0/4006717058 conn(0x7fe78c077120 msgr2=0x7fe78c102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 shutdown_connections 2024-05-27T10:23:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:19.470+0000 7fe792a00640 1 -- 172.21.15.42:0/4006717058 wait complete. 2024-05-27T10:23:20.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:20 smithi042 ceph-mon[31661]: pgmap v141: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-27T10:23:20.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:20 smithi104 ceph-mon[34812]: pgmap v141: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-27T10:23:20.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:20 smithi143 ceph-mon[35545]: pgmap v141: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.228+0000 7f147da00700 1 -- 172.21.15.42:0/442393848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 msgr2=0x7f1478109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.228+0000 7f147da00700 1 --2- 172.21.15.42:0/442393848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f1478109130 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f14640099c0 tx=0x7f1464009cd0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 -- 172.21.15.42:0/442393848 shutdown_connections 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 --2- 172.21.15.42:0/442393848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 0x7f1478110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 --2- 172.21.15.42:0/442393848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14781043f0 0x7f1478104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 --2- 172.21.15.42:0/442393848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f1478109130 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 -- 172.21.15.42:0/442393848 >> 172.21.15.42:0/442393848 conn(0x7f1478079d00 msgr2=0x7f147807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:21.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 -- 172.21.15.42:0/442393848 shutdown_connections 2024-05-27T10:23:21.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.229+0000 7f147da00700 1 -- 172.21.15.42:0/442393848 wait complete. 2024-05-27T10:23:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.230+0000 7f147da00700 1 Processor -- start 2024-05-27T10:23:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.230+0000 7f147da00700 1 -- start start 2024-05-27T10:23:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147da00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14781043f0 0x7f14781a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147da00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f14781a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147da00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 0x7f147819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147da00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1478114180 con 0x7f1478108d20 2024-05-27T10:23:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147da00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1478114000 con 0x7f14781043f0 2024-05-27T10:23:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147da00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1478114300 con 0x7f1478109670 2024-05-27T10:23:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f1477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f14781a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f1477e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14781043f0 0x7f14781a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f1477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f14781a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49426/0 (socket says 172.21.15.42:49426) 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f1477400700 1 -- 172.21.15.42:0/3219265546 learned_addr learned my addr 172.21.15.42:0/3219265546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.231+0000 7f147d000700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 0x7f147819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1477400700 1 -- 172.21.15.42:0/3219265546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 msgr2=0x7f147819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1477400700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 0x7f147819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1477400700 1 -- 172.21.15.42:0/3219265546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14781043f0 msgr2=0x7f14781a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1477400700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14781043f0 0x7f14781a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1477400700 1 -- 172.21.15.42:0/3219265546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147819b070 con 0x7f1478108d20 2024-05-27T10:23:21.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f147d000700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 0x7f147819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1477400700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f14781a0ce0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f14640098e0 tx=0x7f14640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.232+0000 7f1474c00700 1 -- 172.21.15.42:0/3219265546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464027070 con 0x7f1478108d20 2024-05-27T10:23:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.233+0000 7f1474c00700 1 -- 172.21.15.42:0/3219265546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f146400fd30 con 0x7f1478108d20 2024-05-27T10:23:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.233+0000 7f1474c00700 1 -- 172.21.15.42:0/3219265546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464022dc0 con 0x7f1478108d20 2024-05-27T10:23:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.233+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147819b300 con 0x7f1478108d20 2024-05-27T10:23:21.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.233+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147819b790 con 0x7f1478108d20 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.239+0000 7f1474c00700 1 -- 172.21.15.42:0/3219265546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1464022590 con 0x7f1478108d20 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.240+0000 7f148dbf0b80 1 Processor -- start 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.240+0000 7f148dbf0b80 1 -- start start 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 0x56260a4c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4e5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56260a3f5fd0 con 0x56260a348fd0 2024-05-27T10:23:21.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56260a3e5b70 con 0x56260a4c8d20 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56260a3e9bb0 con 0x56260a4c8910 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f147d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 0x56260a4c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f147d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 0x56260a4c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49428/0 (socket says 172.21.15.42:49428) 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f147d000700 1 -- 172.21.15.42:0/3142281693 learned_addr learned my addr 172.21.15.42:0/3142281693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f1477400700 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.241+0000 7f1474c00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f145c0b2a70 0x7f145c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f1474c00700 1 -- 172.21.15.42:0/3219265546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f14640e3500 con 0x7f1478108d20 2024-05-27T10:23:21.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f1477e00700 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f147d000700 1 -- 172.21.15.42:0/3142281693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 msgr2=0x56260a4e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f147d000700 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4e30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f147d000700 1 -- 172.21.15.42:0/3142281693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 msgr2=0x56260a4e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f147d000700 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4e5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.242+0000 7f147d000700 1 -- 172.21.15.42:0/3142281693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260a3ae5b0 con 0x56260a348fd0 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.243+0000 7f147d000700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f145c0b2a70 0x7f145c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.243+0000 7f147d000700 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 0x56260a4c83d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1470009ed0 tx=0x7f14700077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3426b645e2773b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.243+0000 7f147d000700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f145c0b2a70 0x7f145c0b4f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1470039000 tx=0x7f147003b000 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.243+0000 7f146d600700 1 -- 172.21.15.42:0/3142281693 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470033050 con 0x56260a348fd0 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.244+0000 7f146d600700 1 -- 172.21.15.42:0/3142281693 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1470034040 con 0x56260a348fd0 2024-05-27T10:23:21.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.244+0000 7f146d600700 1 -- 172.21.15.42:0/3142281693 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700397a0 con 0x56260a348fd0 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 -- 172.21.15.42:0/3142281693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 msgr2=0x56260a4c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 0x56260a4c83d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1470009ed0 tx=0x7f14700077e0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 -- 172.21.15.42:0/3142281693 shutdown_connections 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a348fd0 0x56260a4c83d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/3142281693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 -- 172.21.15.42:0/3142281693 >> 172.21.15.42:0/3142281693 conn(0x56260a3a9150 msgr2=0x56260a4c3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 -- 172.21.15.42:0/3142281693 shutdown_connections 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.245+0000 7f148dbf0b80 1 -- 172.21.15.42:0/3142281693 wait complete. 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.246+0000 7f148dbf0b80 1 -- start start 2024-05-27T10:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.246+0000 7f148dbf0b80 1 Processor -- start 2024-05-27T10:23:21.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.247+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.247+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 0x56260a4d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.247+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.247+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56260a2ec130 con 0x56260a4c8910 2024-05-27T10:23:21.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.247+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56260a3868f0 con 0x56260a4c8d20 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.247+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56260a4ca2c0 con 0x56260a348fd0 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f148dbf0b80 10 client.0 _ll_register_callbacks cb 0x56260a38fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 0x56260a4d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 0x56260a4d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49444/0 (socket says 172.21.15.42:49444) 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 -- 172.21.15.42:0/1048712861 learned_addr learned my addr 172.21.15.42:0/1048712861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f147d000700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 -- 172.21.15.42:0/1048712861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 msgr2=0x56260a4d6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4d6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 -- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 msgr2=0x56260a4d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f1477400700 1 -- 172.21.15.42:0/1048712861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260a335a60 con 0x56260a4c8910 2024-05-27T10:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.248+0000 7f147d000700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.249+0000 7f1477400700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 0x56260a4d7310 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1464009990 tx=0x7f14640aada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a4f0a5f0d53d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.249+0000 7f1462000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.249+0000 7f1462000700 1 -- 172.21.15.42:0/1048712861 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640a6720 con 0x56260a4c8910 2024-05-27T10:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.249+0000 7f1462000700 1 -- 172.21.15.42:0/1048712861 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f14640a6e50 con 0x56260a4c8910 2024-05-27T10:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.249+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1470034040 con 0x56260a4c8910 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f1462000700 1 -- 172.21.15.42:0/1048712861 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14640a9fd0 con 0x56260a4c8910 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 10 client.16323 fetch_fsmap learned FSMap version 30 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56260a3b61f0 con 0x56260a4c8910 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f1462000700 1 -- 172.21.15.42:0/1048712861 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1470034040 con 0x56260a4c8910 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f1462000700 1 -- 172.21.15.42:0/1048712861 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f14640a9ac0 con 0x56260a4c8910 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T10:23:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 10 client.16323 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:23:21.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 20 client.16323 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:21.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a4e1f60 0x56260a4e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 4 client.16323 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:21.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.250+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260a3f7f20 con 0x56260a4e1f60 2024-05-27T10:23:21.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.251+0000 7f1477e00700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a4e1f60 0x56260a4e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.252+0000 7f1477e00700 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a4e1f60 0x56260a4e43a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389e708271600a99 server_cookie=1396ebaae438f62d in_seq=0 out_seq=0 2024-05-27T10:23:21.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.252+0000 7f1462000700 10 client.16323 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:21.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f1462000700 1 -- 172.21.15.42:0/1048712861 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56260a3f7f20 con 0x56260a4e1f60 2024-05-27T10:23:21.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f1462000700 10 client.16323 handle_command_reply: tid=0 2024-05-27T10:23:21.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 client.16323 shutdown 2024-05-27T10:23:21.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 msgr2=0x56260a4d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:21.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 0x56260a4d7310 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1464009990 tx=0x7f14640aada0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 shutdown_connections 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a4e1f60 0x56260a4e43a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8d20 0x56260a4d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1048712861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8910 0x56260a4d7310 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 >> 172.21.15.42:0/1048712861 conn(0x56260a3a9150 msgr2=0x56260a4c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f1462000700 0 client.16323 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 shutdown_connections 2024-05-27T10:23:21.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.255+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1048712861 wait complete. 2024-05-27T10:23:21.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.256+0000 7f148dbf0b80 20 client.16323 trim_cache size 0 max 16384 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.297+0000 7f148dbf0b80 1 Processor -- start 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 -- start start 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8910 0x56260a4cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a4eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1478074b30 con 0x56260a4c8d20 2024-05-27T10:23:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1478072390 con 0x56260a4c8910 2024-05-27T10:23:21.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.298+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1478070220 con 0x56260a348fd0 2024-05-27T10:23:21.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f147d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a4eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:55138/0 (socket says 172.21.15.42:55138) 2024-05-27T10:23:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477e00700 1 -- 172.21.15.42:0/1390507669 learned_addr learned my addr 172.21.15.42:0/1390507669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477400700 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8910 0x56260a4cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477e00700 1 -- 172.21.15.42:0/1390507669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8910 msgr2=0x56260a4cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477e00700 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8910 0x56260a4cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.299+0000 7f1477e00700 1 -- 172.21.15.42:0/1390507669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 msgr2=0x56260a4eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.300+0000 7f1477e00700 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a4eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.300+0000 7f1477e00700 1 -- 172.21.15.42:0/1390507669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260a4c4600 con 0x56260a348fd0 2024-05-27T10:23:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.300+0000 7f147d000700 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a4eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:23:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.301+0000 7f1477e00700 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4c6f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f147819bb00 tx=0x7f1468004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79129289005c3b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.302+0000 7f146cc00700 1 -- 172.21.15.42:0/1390507669 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468003cc0 con 0x56260a348fd0 2024-05-27T10:23:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.302+0000 7f146cc00700 1 -- 172.21.15.42:0/1390507669 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1468003e60 con 0x56260a348fd0 2024-05-27T10:23:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.302+0000 7f146cc00700 1 -- 172.21.15.42:0/1390507669 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004e60 con 0x56260a348fd0 2024-05-27T10:23:21.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1390507669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 msgr2=0x56260a4c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:21.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4c6f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f147819bb00 tx=0x7f1468004750 comp rx=0 tx=0).stop 2024-05-27T10:23:21.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1390507669 shutdown_connections 2024-05-27T10:23:21.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a4eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a348fd0 0x56260a4c6f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1390507669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a4c8910 0x56260a4cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1390507669 >> 172.21.15.42:0/1390507669 conn(0x56260a3a9150 msgr2=0x56260a4c30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1390507669 shutdown_connections 2024-05-27T10:23:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.303+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1390507669 wait complete. 2024-05-27T10:23:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.304+0000 7f148dbf0b80 1 -- start start 2024-05-27T10:23:21.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.304+0000 7f148dbf0b80 1 Processor -- start 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 0x56260a4d9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a54e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1478070870 con 0x56260a4c8d20 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1478071cc0 con 0x56260a348fd0 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1478070f20 con 0x56260a4c8910 2024-05-27T10:23:21.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.305+0000 7f148dbf0b80 10 client.0 _ll_register_callbacks cb 0x56260a38fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:21.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.306+0000 7f1477e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 0x56260a4d9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.306+0000 7f1477e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 0x56260a4d9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:60036/0 (socket says 172.21.15.42:60036) 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.306+0000 7f1477e00700 1 -- 172.21.15.42:0/1440822169 learned_addr learned my addr 172.21.15.42:0/1440822169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.306+0000 7f1477400700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.306+0000 7f147d000700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a54e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1477e00700 1 -- 172.21.15.42:0/1440822169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 msgr2=0x56260a4d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1477e00700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4d9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1477e00700 1 -- 172.21.15.42:0/1440822169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 msgr2=0x56260a54e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1477e00700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a54e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1477e00700 1 -- 172.21.15.42:0/1440822169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260a54eaa0 con 0x56260a348fd0 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f147d000700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a54e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1477e00700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 0x56260a4d9540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f146800f260 tx=0x7f146800f290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8eed088e9e3ab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:21.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.307+0000 7f1462000700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:23:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.308+0000 7f1462000700 1 -- 172.21.15.42:0/1440822169 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680049d0 con 0x56260a348fd0 2024-05-27T10:23:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.308+0000 7f1462000700 1 -- 172.21.15.42:0/1440822169 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1468004b70 con 0x56260a348fd0 2024-05-27T10:23:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.308+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1468003e60 con 0x56260a348fd0 2024-05-27T10:23:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.309+0000 7f1462000700 1 -- 172.21.15.42:0/1440822169 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1468004db0 con 0x56260a348fd0 2024-05-27T10:23:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.309+0000 7f148dbf0b80 10 client.35867 fetch_fsmap learned FSMap version 30 2024-05-27T10:23:21.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.309+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56260a546430 con 0x56260a348fd0 2024-05-27T10:23:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.309+0000 7f1462000700 1 -- 172.21.15.42:0/1440822169 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1468003e60 con 0x56260a348fd0 2024-05-27T10:23:21.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f1462000700 1 -- 172.21.15.42:0/1440822169 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f1468024520 con 0x56260a348fd0 2024-05-27T10:23:21.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f148dbf0b80 10 client.35867 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T10:23:21.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f148dbf0b80 10 client.35867 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:23:21.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f148dbf0b80 20 client.35867 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:21.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a546b90 0x56260a548fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:21.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f148dbf0b80 4 client.35867 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-27T10:23:21.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.310+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x56260a3f1700 con 0x56260a546b90 2024-05-27T10:23:21.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.311+0000 7f1477400700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a546b90 0x56260a548fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:21.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.312+0000 7f1477400700 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a546b90 0x56260a548fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27eeba16b473a00 server_cookie=97aacb543d6d6b0b in_seq=0 out_seq=0 2024-05-27T10:23:21.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:21.312+0000 7f1462000700 10 client.35867 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:22.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:22 smithi104 ceph-mon[34812]: pgmap v142: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-05-27T10:23:22.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:22 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:22.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:22 smithi143 ceph-mon[35545]: pgmap v142: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-05-27T10:23:22.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:22 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:22.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:22 smithi042 ceph-mon[31661]: pgmap v142: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-05-27T10:23:22.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:22 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:23.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.073+0000 7f1462000700 1 -- 172.21.15.42:0/1440822169 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56260a3f1700 con 0x56260a546b90 2024-05-27T10:23:23.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.073+0000 7f1462000700 10 client.35867 handle_command_reply: tid=0 2024-05-27T10:23:23.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.073+0000 7f148dbf0b80 1 client.35867 shutdown 2024-05-27T10:23:23.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 msgr2=0x56260a4d9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 0x56260a4d9540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f146800f260 tx=0x7f146800f290 comp rx=0 tx=0).stop 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 shutdown_connections 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56260a546b90 0x56260a548fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260a4c8d20 0x56260a54e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260a348fd0 0x56260a4d9540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 --2- 172.21.15.42:0/1440822169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56260a4c8910 0x56260a4d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f1462000700 0 client.35867 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:23.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.074+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 >> 172.21.15.42:0/1440822169 conn(0x56260a3a9150 msgr2=0x56260a4d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.075+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 shutdown_connections 2024-05-27T10:23:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.075+0000 7f148dbf0b80 1 -- 172.21.15.42:0/1440822169 wait complete. 2024-05-27T10:23:23.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.075+0000 7f148dbf0b80 20 client.35867 trim_cache size 0 max 16384 2024-05-27T10:23:23.148 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:23:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 30, 2024-05-27T10:23:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 14, 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "e15a8e9": { 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 14, 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:23:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 150.0, 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f145c0b2a70 msgr2=0x7f145c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f145c0b2a70 0x7f145c0b4f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1470039000 tx=0x7f147003b000 comp rx=0 tx=0).stop 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 msgr2=0x7f14781a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f14781a0ce0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f14640098e0 tx=0x7f14640047b0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 shutdown_connections 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f145c0b2a70 0x7f145c0b4f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1478109670 0x7f147819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14781043f0 0x7f14781a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 --2- 172.21.15.42:0/3219265546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1478108d20 0x7f14781a0ce0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.077+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 >> 172.21.15.42:0/3219265546 conn(0x7f1478079d00 msgr2=0x7f1478101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.078+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 shutdown_connections 2024-05-27T10:23:23.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:23.078+0000 7f147da00700 1 -- 172.21.15.42:0/3219265546 wait complete. 2024-05-27T10:23:24.178 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e15a8e9', quiesce took 11.3 seconds. Will release after: 18.7 2024-05-27T10:23:24.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:24 smithi104 ceph-mon[34812]: pgmap v143: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-05-27T10:23:24.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:24 smithi143 ceph-mon[35545]: pgmap v143: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-05-27T10:23:24.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:24 smithi042 ceph-mon[31661]: pgmap v143: 97 pgs: 97 active+clean; 57 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-05-27T10:23:26.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:26 smithi042 ceph-mon[31661]: pgmap v144: 97 pgs: 97 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.9 MiB/s wr, 144 op/s 2024-05-27T10:23:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:26 smithi104 ceph-mon[34812]: pgmap v144: 97 pgs: 97 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.9 MiB/s wr, 144 op/s 2024-05-27T10:23:26.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:26 smithi143 ceph-mon[35545]: pgmap v144: 97 pgs: 97 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.9 MiB/s wr, 144 op/s 2024-05-27T10:23:28.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:28 smithi104 ceph-mon[34812]: pgmap v145: 97 pgs: 97 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-05-27T10:23:28.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:28 smithi143 ceph-mon[35545]: pgmap v145: 97 pgs: 97 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-05-27T10:23:28.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:28 smithi042 ceph-mon[31661]: pgmap v145: 97 pgs: 97 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-05-27T10:23:28.715 INFO:tasks.workunit.client.0.smithi042.stdout: 9 4509 30 2427 97 672 43 2024-05-27T10:23:30.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:30 smithi104 ceph-mon[34812]: pgmap v146: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-05-27T10:23:30.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:30 smithi143 ceph-mon[35545]: pgmap v146: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-05-27T10:23:30.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:30 smithi042 ceph-mon[31661]: pgmap v146: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-05-27T10:23:31.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:31 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:31.458 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:31 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:31.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:31 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:32.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:32 smithi104 ceph-mon[34812]: pgmap v147: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 655 KiB/s wr, 79 op/s 2024-05-27T10:23:32.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:32 smithi143 ceph-mon[35545]: pgmap v147: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 655 KiB/s wr, 79 op/s 2024-05-27T10:23:32.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:32 smithi042 ceph-mon[31661]: pgmap v147: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 655 KiB/s wr, 79 op/s 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 -- 172.21.15.42:0/1608119252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328110c80 msgr2=0x7fe3281110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 --2- 172.21.15.42:0/1608119252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328110c80 0x7fe3281110b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe314009900 tx=0x7fe31402a3f0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 -- 172.21.15.42:0/1608119252 shutdown_connections 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 --2- 172.21.15.42:0/1608119252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe328111740 0x7fe328118ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 --2- 172.21.15.42:0/1608119252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328110c80 0x7fe3281110b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 --2- 172.21.15.42:0/1608119252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32806c020 0x7fe32806c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 -- 172.21.15.42:0/1608119252 >> 172.21.15.42:0/1608119252 conn(0x7fe32806eb10 msgr2=0x7fe328070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 -- 172.21.15.42:0/1608119252 shutdown_connections 2024-05-27T10:23:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.709+0000 7fe32ca00640 1 -- 172.21.15.42:0/1608119252 wait complete. 2024-05-27T10:23:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.710+0000 7fe32ca00640 1 Processor -- start 2024-05-27T10:23:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 -- start start 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32806c020 0x7fe3281a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe328110c80 0x7fe3281a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 0x7fe3281b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe32811c540 con 0x7fe328111740 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe32811c3c0 con 0x7fe328110c80 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe32ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe32811c6c0 con 0x7fe32806c020 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.711+0000 7fe326800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 0x7fe3281b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 0x7fe3281b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56360/0 (socket says 172.21.15.42:56360) 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 -- 172.21.15.42:0/2580021218 learned_addr learned my addr 172.21.15.42:0/2580021218 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe325e00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32806c020 0x7fe3281a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe325400640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe328110c80 0x7fe3281a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 -- 172.21.15.42:0/2580021218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32806c020 msgr2=0x7fe3281a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32806c020 0x7fe3281a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 -- 172.21.15.42:0/2580021218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe328110c80 msgr2=0x7fe3281a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe328110c80 0x7fe3281a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 -- 172.21.15.42:0/2580021218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281b13e0 con 0x7fe328111740 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.712+0000 7fe326800640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 0x7fe3281b0ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe32000cad0 tx=0x7fe32000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.713+0000 7fe31ea00640 1 -- 172.21.15.42:0/2580021218 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000ae20 con 0x7fe328111740 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.713+0000 7fe31ea00640 1 -- 172.21.15.42:0/2580021218 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe320005020 con 0x7fe328111740 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.713+0000 7fe31ea00640 1 -- 172.21.15.42:0/2580021218 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000f6b0 con 0x7fe328111740 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.713+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281b1670 con 0x7fe328111740 2024-05-27T10:23:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.713+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281b1b90 con 0x7fe328111740 2024-05-27T10:23:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.717+0000 7fe3339d7740 1 Processor -- start 2024-05-27T10:23:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.718+0000 7fe31ea00640 1 -- 172.21.15.42:0/2580021218 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe3200051d0 con 0x7fe328111740 2024-05-27T10:23:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.718+0000 7fe3339d7740 1 -- start start 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 0x55969132e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f632f0 0x55969132eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969132f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe3339d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55969129fb90 con 0x559690f8c1b0 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe3339d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55969128f730 con 0x559690f632f0 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe3339d7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559691293770 con 0x559690f6b4d0 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe325400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 0x55969132e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe325400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 0x55969132e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56368/0 (socket says 172.21.15.42:56368) 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe325400640 1 -- 172.21.15.42:0/2660593112 learned_addr learned my addr 172.21.15.42:0/2660593112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.719+0000 7fe325e00640 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f632f0 0x55969132eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969132f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 -- 172.21.15.42:0/2660593112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 msgr2=0x55969132f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969132f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 -- 172.21.15.42:0/2660593112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f632f0 msgr2=0x55969132eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f632f0 0x55969132eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 -- 172.21.15.42:0/2660593112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596912288b0 con 0x559690f8c1b0 2024-05-27T10:23:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe31ea00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe30c0b29d0 0x7fe30c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe325400640 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 0x55969132e860 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe3140096c0 tx=0x7fe314005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9c5027b635bfbe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.720+0000 7fe31ea00640 1 -- 172.21.15.42:0/2580021218 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe3200e28c0 con 0x7fe328111740 2024-05-27T10:23:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.721+0000 7fe325e00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe30c0b29d0 0x7fe30c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.721+0000 7fe313000640 1 -- 172.21.15.42:0/2660593112 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314043020 con 0x559690f8c1b0 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.721+0000 7fe313000640 1 -- 172.21.15.42:0/2660593112 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3140053e0 con 0x559690f8c1b0 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.721+0000 7fe313000640 1 -- 172.21.15.42:0/2660593112 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314035070 con 0x559690f8c1b0 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 -- 172.21.15.42:0/2660593112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 msgr2=0x55969132e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 0x55969132e860 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe3140096c0 tx=0x7fe314005d50 comp rx=0 tx=0).stop 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe325e00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe30c0b29d0 0x7fe30c0b4ee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55969132fc10 tx=0x7fe31800b390 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 -- 172.21.15.42:0/2660593112 shutdown_connections 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969132f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f632f0 0x55969132eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 --2- 172.21.15.42:0/2660593112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f8c1b0 0x55969132e860 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 -- 172.21.15.42:0/2660593112 >> 172.21.15.42:0/2660593112 conn(0x559690f80300 msgr2=0x55969132b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 -- 172.21.15.42:0/2660593112 shutdown_connections 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.722+0000 7fe3339d7740 1 -- 172.21.15.42:0/2660593112 wait complete. 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 -- start start 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 Processor -- start 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x559691338ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x5596913391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691344c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559690f75790 con 0x559690f632f0 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559691237bc0 con 0x559690f8c1b0 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.723+0000 7fe3339d7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55969133b930 con 0x559690f6b4d0 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe3339d7740 10 client.0 _ll_register_callbacks cb 0x559691215400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x559691338ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x559691338ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56380/0 (socket says 172.21.15.42:56380) 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 -- 172.21.15.42:0/764139459 learned_addr learned my addr 172.21.15.42:0/764139459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325400640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x5596913391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe326800640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691344c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 -- 172.21.15.42:0/764139459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 msgr2=0x5596913391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x5596913391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 -- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 msgr2=0x559691344c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691344c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.724+0000 7fe325e00640 1 -- 172.21.15.42:0/764139459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559690fa4210 con 0x559690f632f0 2024-05-27T10:23:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe325e00640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x559691338ca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe318003d00 tx=0x7fe3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5a57e763199f99 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe307400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe307400640 1 -- 172.21.15.42:0/764139459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318027400 con 0x559690f632f0 2024-05-27T10:23:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe307400640 1 -- 172.21.15.42:0/764139459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe318017030 con 0x559690f632f0 2024-05-27T10:23:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3140053e0 con 0x559690f632f0 2024-05-27T10:23:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe307400640 1 -- 172.21.15.42:0/764139459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe3180275a0 con 0x559690f632f0 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe3339d7740 10 client.16338 fetch_fsmap learned FSMap version 31 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.725+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559691070620 con 0x559690f632f0 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe307400640 1 -- 172.21.15.42:0/764139459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3140053e0 con 0x559690f632f0 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe307400640 1 -- 172.21.15.42:0/764139459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe3180116e0 con 0x559690f632f0 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe3339d7740 10 client.16338 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe3339d7740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe3339d7740 20 client.16338 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe3339d7740 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f4ad80 0x55969134ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe3339d7740 4 client.16338 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.726+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596912a1ae0 con 0x559690f4ad80 2024-05-27T10:23:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.727+0000 7fe325400640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f4ad80 0x55969134ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.728+0000 7fe325400640 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f4ad80 0x55969134ae80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2dbb89c42fe8975 server_cookie=1ee73917410de8a5 in_seq=0 out_seq=0 2024-05-27T10:23:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.728+0000 7fe307400640 10 client.16338 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.729+0000 7fe307400640 1 -- 172.21.15.42:0/764139459 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5596912a1ae0 con 0x559690f4ad80 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.729+0000 7fe307400640 10 client.16338 handle_command_reply: tid=0 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.729+0000 7fe3339d7740 1 client.16338 shutdown 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 msgr2=0x559691338ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x559691338ca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe318003d00 tx=0x7fe3180046c0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 shutdown_connections 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691344c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f4ad80 0x55969134ae80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x5596913391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 --2- 172.21.15.42:0/764139459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x559691338ca0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe307400640 0 client.16338 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 >> 172.21.15.42:0/764139459 conn(0x559690f80300 msgr2=0x55969132e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 shutdown_connections 2024-05-27T10:23:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 1 -- 172.21.15.42:0/764139459 wait complete. 2024-05-27T10:23:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.730+0000 7fe3339d7740 20 client.16338 trim_cache size 0 max 16384 2024-05-27T10:23:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 Processor -- start 2024-05-27T10:23:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 -- start start 2024-05-27T10:23:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f632f0 0x5596913449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 0x559691344f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691350680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281bcac0 con 0x559690f6b4d0 2024-05-27T10:23:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281ba360 con 0x559690f8c1b0 2024-05-27T10:23:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.747+0000 7fe3339d7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b81b0 con 0x559690f632f0 2024-05-27T10:23:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 0x559691344f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 0x559691344f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56392/0 (socket says 172.21.15.42:56392) 2024-05-27T10:23:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 -- 172.21.15.42:0/637296259 learned_addr learned my addr 172.21.15.42:0/637296259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325e00640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f632f0 0x5596913449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe326800640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691350680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 -- 172.21.15.42:0/637296259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f632f0 msgr2=0x5596913449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f632f0 0x5596913449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 -- 172.21.15.42:0/637296259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 msgr2=0x559691350680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691350680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325400640 1 -- 172.21.15.42:0/637296259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55969132fc40 con 0x559690f6b4d0 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe325e00640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f632f0 0x5596913449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.748+0000 7fe326800640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691350680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.749+0000 7fe325400640 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 0x559691344f10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe314005f70 tx=0x7fe314005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3f021dc64f6c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.749+0000 7fe312600640 1 -- 172.21.15.42:0/637296259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314043070 con 0x559690f6b4d0 2024-05-27T10:23:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.749+0000 7fe312600640 1 -- 172.21.15.42:0/637296259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe314055070 con 0x559690f6b4d0 2024-05-27T10:23:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.749+0000 7fe312600640 1 -- 172.21.15.42:0/637296259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314035040 con 0x559690f6b4d0 2024-05-27T10:23:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 -- 172.21.15.42:0/637296259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 msgr2=0x559691344f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 0x559691344f10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe314005f70 tx=0x7fe314005d50 comp rx=0 tx=0).stop 2024-05-27T10:23:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 -- 172.21.15.42:0/637296259 shutdown_connections 2024-05-27T10:23:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x559691350680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f6b4d0 0x559691344f10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 --2- 172.21.15.42:0/637296259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f632f0 0x5596913449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 -- 172.21.15.42:0/637296259 >> 172.21.15.42:0/637296259 conn(0x559690f80300 msgr2=0x55969133abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 -- 172.21.15.42:0/637296259 shutdown_connections 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.750+0000 7fe3339d7740 1 -- 172.21.15.42:0/637296259 wait complete. 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 -- start start 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 Processor -- start 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x55969133dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969133e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x5596913b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b8820 con 0x559690f632f0 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b9c90 con 0x559690f8c1b0 2024-05-27T10:23:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.751+0000 7fe3339d7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b8ef0 con 0x559690f6b4d0 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe3339d7740 10 client.0 _ll_register_callbacks cb 0x7fe318004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x55969133dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x55969133dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56396/0 (socket says 172.21.15.42:56396) 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 -- 172.21.15.42:0/724470225 learned_addr learned my addr 172.21.15.42:0/724470225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe326800640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x5596913b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325400640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969133e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 -- 172.21.15.42:0/724470225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 msgr2=0x55969133e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969133e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 -- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 msgr2=0x5596913b9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x5596913b9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.752+0000 7fe325e00640 1 -- 172.21.15.42:0/724470225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55969133eb50 con 0x559690f632f0 2024-05-27T10:23:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe325e00640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x55969133dd30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe318018040 tx=0x7fe3180119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e29b86dd1a77f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe307400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe307400640 1 -- 172.21.15.42:0/724470225 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318027400 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe307400640 1 -- 172.21.15.42:0/724470225 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe318005450 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe314055070 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe307400640 1 -- 172.21.15.42:0/724470225 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe318040050 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe3339d7740 10 client.16344 fetch_fsmap learned FSMap version 31 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5596913bb220 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.753+0000 7fe307400640 1 -- 172.21.15.42:0/724470225 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe314055070 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe307400640 1 -- 172.21.15.42:0/724470225 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe318038020 con 0x559690f632f0 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe3339d7740 10 client.16344 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe3339d7740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe3339d7740 20 client.16344 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe3339d7740 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f201c0 0x5596913bda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe3339d7740 4 client.16344 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:23:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.754+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55969129b2c0 con 0x559690f201c0 2024-05-27T10:23:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.755+0000 7fe325400640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f201c0 0x5596913bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.755+0000 7fe325400640 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f201c0 0x5596913bda90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218ceed1d0f2e2c5 server_cookie=1949fd55ac069874 in_seq=0 out_seq=0 2024-05-27T10:23:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.755+0000 7fe307400640 10 client.16344 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe307400640 1 -- 172.21.15.42:0/724470225 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55969129b2c0 con 0x559690f201c0 2024-05-27T10:23:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe307400640 10 client.16344 handle_command_reply: tid=0 2024-05-27T10:23:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 client.16344 shutdown 2024-05-27T10:23:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 msgr2=0x55969133dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x55969133dd30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe318018040 tx=0x7fe3180119f0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 shutdown_connections 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559690f8c1b0 0x5596913b9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559690f201c0 0x5596913bda90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559690f6b4d0 0x55969133e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 --2- 172.21.15.42:0/724470225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559690f632f0 0x55969133dd30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe307400640 0 client.16344 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.777+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 >> 172.21.15.42:0/724470225 conn(0x559690f80300 msgr2=0x559691333220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.778+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 shutdown_connections 2024-05-27T10:23:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.778+0000 7fe3339d7740 1 -- 172.21.15.42:0/724470225 wait complete. 2024-05-27T10:23:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.778+0000 7fe3339d7740 20 client.16344 trim_cache size 0 max 16384 2024-05-27T10:23:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:23:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:23:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:23:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe30c0b29d0 msgr2=0x7fe30c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe30c0b29d0 0x7fe30c0b4ee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55969132fc10 tx=0x7fe31800b390 comp rx=0 tx=0).stop 2024-05-27T10:23:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 msgr2=0x7fe3281b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 0x7fe3281b0ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe32000cad0 tx=0x7fe32000cf90 comp rx=0 tx=0).stop 2024-05-27T10:23:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 shutdown_connections 2024-05-27T10:23:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe30c0b29d0 0x7fe30c0b4ee0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe328111740 0x7fe3281b0ea0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe328110c80 0x7fe3281a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 --2- 172.21.15.42:0/2580021218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32806c020 0x7fe3281a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 >> 172.21.15.42:0/2580021218 conn(0x7fe32806eb10 msgr2=0x7fe32810b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 shutdown_connections 2024-05-27T10:23:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:32.779+0000 7fe32ca00640 1 -- 172.21.15.42:0/2580021218 wait complete. 2024-05-27T10:23:34.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:34 smithi104 ceph-mon[34812]: pgmap v148: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 655 KiB/s wr, 79 op/s 2024-05-27T10:23:34.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:34 smithi143 ceph-mon[35545]: pgmap v148: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 655 KiB/s wr, 79 op/s 2024-05-27T10:23:34.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:34 smithi042 ceph-mon[31661]: pgmap v148: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 655 KiB/s wr, 79 op/s 2024-05-27T10:23:36.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:36 smithi104 ceph-mon[34812]: pgmap v149: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 659 KiB/s wr, 85 op/s 2024-05-27T10:23:36.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:36 smithi143 ceph-mon[35545]: pgmap v149: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 659 KiB/s wr, 85 op/s 2024-05-27T10:23:36.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:36 smithi042 ceph-mon[31661]: pgmap v149: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 659 KiB/s wr, 85 op/s 2024-05-27T10:23:38.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:38 smithi104 ceph-mon[34812]: pgmap v150: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-27T10:23:38.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:38 smithi143 ceph-mon[35545]: pgmap v150: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-27T10:23:38.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:38 smithi042 ceph-mon[31661]: pgmap v150: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-27T10:23:38.715 INFO:tasks.workunit.client.0.smithi042.stdout: 9 0 0 0 0 0 0 2024-05-27T10:23:40.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:40 smithi104 ceph-mon[34812]: pgmap v151: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 56 KiB/s wr, 13 op/s 2024-05-27T10:23:40.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:40 smithi143 ceph-mon[35545]: pgmap v151: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 56 KiB/s wr, 13 op/s 2024-05-27T10:23:40.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:40 smithi042 ceph-mon[31661]: pgmap v151: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 56 KiB/s wr, 13 op/s 2024-05-27T10:23:42.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:42 smithi104 ceph-mon[34812]: pgmap v152: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:23:42.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:42 smithi143 ceph-mon[35545]: pgmap v152: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:23:42.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:42 smithi042 ceph-mon[31661]: pgmap v152: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:23:42.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:23:43.340 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:23:44.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:44 smithi104 ceph-mon[34812]: pgmap v153: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:23:44.454 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:44 smithi042 ceph-mon[31661]: pgmap v153: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:23:44.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:44 smithi143 ceph-mon[35545]: pgmap v153: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 -- 172.21.15.42:0/503668430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 msgr2=0x7f34cc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 --2- 172.21.15.42:0/503668430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc109320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f34b80099c0 tx=0x7f34b8009cd0 comp rx=0 tx=0).stop 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 -- 172.21.15.42:0/503668430 shutdown_connections 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 --2- 172.21.15.42:0/503668430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc109860 0x7f34cc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 --2- 172.21.15.42:0/503668430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34cc1045e0 0x7f34cc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 --2- 172.21.15.42:0/503668430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc109320 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 -- 172.21.15.42:0/503668430 >> 172.21.15.42:0/503668430 conn(0x7f34cc100030 msgr2=0x7f34cc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 -- 172.21.15.42:0/503668430 shutdown_connections 2024-05-27T10:23:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.934+0000 7f34d4200700 1 -- 172.21.15.42:0/503668430 wait complete. 2024-05-27T10:23:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.935+0000 7f34d4200700 1 Processor -- start 2024-05-27T10:23:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 -- start start 2024-05-27T10:23:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34cc1045e0 0x7f34cc19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc109860 0x7f34cc1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc114320 con 0x7f34cc108f10 2024-05-27T10:23:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1141a0 con 0x7f34cc109860 2024-05-27T10:23:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.936+0000 7f34d4200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1144a0 con 0x7f34cc1045e0 2024-05-27T10:23:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54440/0 (socket says 172.21.15.42:54440) 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 -- 172.21.15.42:0/3235715307 learned_addr learned my addr 172.21.15.42:0/3235715307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d2200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc109860 0x7f34cc1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d1800700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34cc1045e0 0x7f34cc19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 -- 172.21.15.42:0/3235715307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34cc1045e0 msgr2=0x7f34cc19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34cc1045e0 0x7f34cc19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 -- 172.21.15.42:0/3235715307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc109860 msgr2=0x7f34cc1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc109860 0x7f34cc1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.937+0000 7f34d0e00700 1 -- 172.21.15.42:0/3235715307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc1a1e00 con 0x7f34cc108f10 2024-05-27T10:23:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.938+0000 7f34d0e00700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc19d4d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f34b800b580 tx=0x7f34b800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.938+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8026070 con 0x7f34cc108f10 2024-05-27T10:23:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.938+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc1a2090 con 0x7f34cc108f10 2024-05-27T10:23:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.938+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34b8005030 con 0x7f34cc108f10 2024-05-27T10:23:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.938+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b80049f0 con 0x7f34cc108f10 2024-05-27T10:23:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.938+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1a9a10 con 0x7f34cc108f10 2024-05-27T10:23:45.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.940+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc0512b0 con 0x7f34cc108f10 2024-05-27T10:23:45.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.943+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f34b8029070 con 0x7f34cc108f10 2024-05-27T10:23:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.945+0000 7f34ca000700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f34b40b2a70 0x7f34b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.945+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f34b80e2fe0 con 0x7f34cc108f10 2024-05-27T10:23:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.945+0000 7f34d1800700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f34b40b2a70 0x7f34b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:45.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.946+0000 7f34d1800700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f34b40b2a70 0x7f34b40b4f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005950 tx=0x7f34c00058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:45.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:45.955+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34b8013070 con 0x7f34cc108f10 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 -- 172.21.15.42:0/888725892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 msgr2=0x7f2f70112700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 --2- 172.21.15.42:0/888725892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70112700 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009ca0 tx=0x7f2f6802a950 comp rx=0 tx=0).stop 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 -- 172.21.15.42:0/888725892 shutdown_connections 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 --2- 172.21.15.42:0/888725892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70112700 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 --2- 172.21.15.42:0/888725892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 0x7f2f7010fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 --2- 172.21.15.42:0/888725892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70075660 0x7f2f70075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.015+0000 7f2f6f200640 1 -- 172.21.15.42:0/888725892 >> 172.21.15.42:0/888725892 conn(0x7f2f70079140 msgr2=0x7f2f70079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.016+0000 7f2f6f200640 1 -- 172.21.15.42:0/888725892 shutdown_connections 2024-05-27T10:23:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.016+0000 7f2f6f200640 1 -- 172.21.15.42:0/888725892 wait complete. 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 Processor -- start 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 -- start start 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70075660 0x7f2f701035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 0x7f2f70103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70115d70 con 0x7f2f701102a0 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70115bf0 con 0x7f2f70075660 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.017+0000 7f2f6f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70115ef0 con 0x7f2f7010d940 2024-05-27T10:23:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70104030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54454/0 (socket says 172.21.15.42:54454) 2024-05-27T10:23:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 -- 172.21.15.42:0/835916168 learned_addr learned my addr 172.21.15.42:0/835916168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6de00640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70075660 0x7f2f701035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6d400640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 0x7f2f70103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 -- 172.21.15.42:0/835916168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 msgr2=0x7f2f70103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 0x7f2f70103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 -- 172.21.15.42:0/835916168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70075660 msgr2=0x7f2f701035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70075660 0x7f2f701035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6e800640 1 -- 172.21.15.42:0/835916168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701acd00 con 0x7f2f701102a0 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.018+0000 7f2f6d400640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 0x7f2f70103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.019+0000 7f2f6e800640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70104030 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009dc0 tx=0x7f2f68003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.019+0000 7f2f66a00640 1 -- 172.21.15.42:0/835916168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68034040 con 0x7f2f701102a0 2024-05-27T10:23:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.019+0000 7f2f66a00640 1 -- 172.21.15.42:0/835916168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f68030850 con 0x7f2f701102a0 2024-05-27T10:23:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.019+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f701acf30 con 0x7f2f701102a0 2024-05-27T10:23:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.019+0000 7f2f66a00640 1 -- 172.21.15.42:0/835916168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680046e0 con 0x7f2f701102a0 2024-05-27T10:23:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.019+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f701ad3f0 con 0x7f2f701102a0 2024-05-27T10:23:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.023+0000 7f2f7c05d740 1 Processor -- start 2024-05-27T10:23:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.024+0000 7f2f66a00640 1 -- 172.21.15.42:0/835916168 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f68037020 con 0x7f2f701102a0 2024-05-27T10:23:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.024+0000 7f2f7c05d740 1 -- start start 2024-05-27T10:23:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.024+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.024+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 0x55d95fd17fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f9544d0 0x55d95fd184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d95fc88b90 con 0x55d95f94c2f0 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d95fc78730 con 0x55d95f9544d0 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d95fc7c770 con 0x55d95f9751b0 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 0x55d95fd17fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 0x55d95fd17fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54468/0 (socket says 172.21.15.42:54468) 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f6de00640 1 -- 172.21.15.42:0/3482135116 learned_addr learned my addr 172.21.15.42:0/3482135116 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f6d400640 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.025+0000 7f2f6d400640 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f9544d0 0x55d95fd184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6de00640 1 -- 172.21.15.42:0/3482135116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 msgr2=0x55d95fd17a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f66a00640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2f540b29d0 0x7f2f540b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6de00640 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd17a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6de00640 1 -- 172.21.15.42:0/3482135116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f9544d0 msgr2=0x55d95fd184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6de00640 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f9544d0 0x55d95fd184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6de00640 1 -- 172.21.15.42:0/3482135116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95fc118b0 con 0x55d95f94c2f0 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6d400640 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f9544d0 0x55d95fd184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f66a00640 1 -- 172.21.15.42:0/835916168 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f2f681034d0 con 0x7f2f701102a0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6e800640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2f540b29d0 0x7f2f540b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.026+0000 7f2f6de00640 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 0x55d95fd17fa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000bcd0 tx=0x7f2f600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b505640295bef718 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.027+0000 7f2f5b400640 1 -- 172.21.15.42:0/3482135116 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60005220 con 0x55d95f94c2f0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.027+0000 7f2f5b400640 1 -- 172.21.15.42:0/3482135116 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f60004c30 con 0x55d95f94c2f0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.027+0000 7f2f5b400640 1 -- 172.21.15.42:0/3482135116 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f600116a0 con 0x55d95f94c2f0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3482135116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 msgr2=0x55d95fd17fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 0x55d95fd17fa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000bcd0 tx=0x7f2f600107a0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f6e800640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2f540b29d0 0x7f2f540b4ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f68100e30 tx=0x7f2f680095c0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3482135116 shutdown_connections 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f9544d0 0x55d95fd184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f94c2f0 0x55d95fd17fa0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3482135116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3482135116 >> 172.21.15.42:0/3482135116 conn(0x55d95f969300 msgr2=0x55d95fd14210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3482135116 shutdown_connections 2024-05-27T10:23:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.028+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3482135116 wait complete. 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.029+0000 7f2f7c05d740 1 -- start start 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.029+0000 7f2f7c05d740 1 Processor -- start 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd22170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd2df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d95f95e790 con 0x55d95f9544d0 2024-05-27T10:23:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d95fc20bc0 con 0x55d95f94c2f0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d95fd24be0 con 0x55d95f9751b0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f7c05d740 10 client.0 _ll_register_callbacks cb 0x55d95fbfe400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd22170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f6d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd22170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54470/0 (socket says 172.21.15.42:54470) 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f6de00640 1 -- 172.21.15.42:0/492628697 learned_addr learned my addr 172.21.15.42:0/492628697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.030+0000 7f2f6d400640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd2df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6d400640 1 -- 172.21.15.42:0/492628697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 msgr2=0x55d95fd2df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6d400640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd2df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6d400640 1 -- 172.21.15.42:0/492628697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 msgr2=0x55d95fd22170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6d400640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd22170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6d400640 1 -- 172.21.15.42:0/492628697 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95f98d210 con 0x55d95f94c2f0 2024-05-27T10:23:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6de00640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd22170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f6d400640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd21c30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0058c0 tx=0x7f2f5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a40a28e69dea7be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.031+0000 7f2f4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.032+0000 7f2f4fe00640 1 -- 172.21.15.42:0/492628697 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c004230 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.032+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f60004c30 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.032+0000 7f2f4fe00640 1 -- 172.21.15.42:0/492628697 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f2f5c0043d0 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.032+0000 7f2f4fe00640 1 -- 172.21.15.42:0/492628697 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c0035f0 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.033+0000 7f2f7c05d740 10 client.35897 fetch_fsmap learned FSMap version 31 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.033+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d95fa59620 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.033+0000 7f2f4fe00640 1 -- 172.21.15.42:0/492628697 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f60004c30 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f4fe00640 1 -- 172.21.15.42:0/492628697 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2f5c011640 con 0x55d95f94c2f0 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f7c05d740 10 client.35897 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f7c05d740 10 client.35897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f7c05d740 20 client.35897 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f933d80 0x55d95fd34040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f7c05d740 4 client.35897 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95fc8aae0 con 0x55d95f933d80 2024-05-27T10:23:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.034+0000 7f2f6de00640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f933d80 0x55d95fd34040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.035+0000 7f2f6de00640 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f933d80 0x55d95fd34040 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c68364789e6a5f5 server_cookie=1df2441ba0e4301 in_seq=0 out_seq=0 2024-05-27T10:23:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.035+0000 7f2f4fe00640 10 client.35897 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.037+0000 7f2f4fe00640 1 -- 172.21.15.42:0/492628697 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d95fc8aae0 con 0x55d95f933d80 2024-05-27T10:23:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.037+0000 7f2f4fe00640 10 client.35897 handle_command_reply: tid=0 2024-05-27T10:23:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.037+0000 7f2f7c05d740 1 client.35897 shutdown 2024-05-27T10:23:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 msgr2=0x55d95fd21c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd21c30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0058c0 tx=0x7f2f5c0046c0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 shutdown_connections 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f933d80 0x55d95fd34040 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd2df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd22170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/492628697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd21c30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 >> 172.21.15.42:0/492628697 conn(0x55d95f969300 msgr2=0x55d95fd17890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f4fe00640 0 client.35897 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 shutdown_connections 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 1 -- 172.21.15.42:0/492628697 wait complete. 2024-05-27T10:23:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.038+0000 7f2f7c05d740 20 client.35897 trim_cache size 0 max 16384 2024-05-27T10:23:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.074+0000 7f2f7c05d740 1 Processor -- start 2024-05-27T10:23:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.074+0000 7f2f7c05d740 1 -- start start 2024-05-27T10:23:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.075+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.075+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd21bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.075+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.075+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701b85d0 con 0x55d95f9544d0 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.075+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701b5e70 con 0x55d95f94c2f0 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.075+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701b3cc0 con 0x55d95f9751b0 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd21bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd21bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54484/0 (socket says 172.21.15.42:54484) 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 -- 172.21.15.42:0/3882637036 learned_addr learned my addr 172.21.15.42:0/3882637036 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6d400640 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6d400640 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 -- 172.21.15.42:0/3882637036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 msgr2=0x55d95fd22110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd22110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 -- 172.21.15.42:0/3882637036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 msgr2=0x55d95fd1d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd1d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6de00640 1 -- 172.21.15.42:0/3882637036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95fd18810 con 0x55d95f9544d0 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.076+0000 7f2f6d400640 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd22110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.077+0000 7f2f6de00640 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd21bd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2f60010ca0 tx=0x7f2f60005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aff3c9009c5e107 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.077+0000 7f2f5aa00640 1 -- 172.21.15.42:0/3882637036 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60004220 con 0x55d95f9544d0 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.077+0000 7f2f5aa00640 1 -- 172.21.15.42:0/3882637036 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f600043c0 con 0x55d95f9544d0 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.077+0000 7f2f5aa00640 1 -- 172.21.15.42:0/3882637036 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60004d50 con 0x55d95f9544d0 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.078+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3882637036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 msgr2=0x55d95fd21bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.078+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd21bd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2f60010ca0 tx=0x7f2f60005fb0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3882637036 shutdown_connections 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9751b0 0x55d95fd22110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9544d0 0x55d95fd21bd0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/3882637036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd1d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3882637036 >> 172.21.15.42:0/3882637036 conn(0x55d95f969300 msgr2=0x55d95fd214c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3882637036 shutdown_connections 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 -- 172.21.15.42:0/3882637036 wait complete. 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.079+0000 7f2f7c05d740 1 -- start start 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 Processor -- start 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd2a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9544d0 0x55d95fd26a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 0x55d95fd270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701b4330 con 0x55d95f9751b0 2024-05-27T10:23:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701b57a0 con 0x55d95f94c2f0 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701b4a00 con 0x55d95f9544d0 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.080+0000 7f2f7c05d740 10 client.0 _ll_register_callbacks cb 0x7f2f5c00ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 0x55d95fd270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 0x55d95fd270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54500/0 (socket says 172.21.15.42:54500) 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 -- 172.21.15.42:0/2617930732 learned_addr learned my addr 172.21.15.42:0/2617930732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd2a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6de00640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9544d0 0x55d95fd26a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 -- 172.21.15.42:0/2617930732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9544d0 msgr2=0x55d95fd26a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9544d0 0x55d95fd26a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 -- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 msgr2=0x55d95fd2a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd2a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6d400640 1 -- 172.21.15.42:0/2617930732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95fd277f0 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.081+0000 7f2f6de00640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9544d0 0x55d95fd26a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f6d400640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 0x55d95fd270b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c011b80 tx=0x7f2f5c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478b7d7b8f89c82f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f4fe00640 1 -- 172.21.15.42:0/2617930732 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c035020 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f4fe00640 1 -- 172.21.15.42:0/2617930732 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f2f5c005030 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f600043c0 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f4fe00640 1 -- 172.21.15.42:0/2617930732 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c005230 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f7c05d740 10 client.16368 fetch_fsmap learned FSMap version 31 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.082+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d95fd9b6f0 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f4fe00640 1 -- 172.21.15.42:0/2617930732 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f600043c0 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f4fe00640 1 -- 172.21.15.42:0/2617930732 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2f5c004bc0 con 0x55d95f9751b0 2024-05-27T10:23:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f7c05d740 10 client.16368 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f7c05d740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f7c05d740 20 client.16368 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f9091c0 0x55d95fd9dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f7c05d740 4 client.16368 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.083+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d95fc842c0 con 0x55d95f9091c0 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.084+0000 7f2f6de00640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f9091c0 0x55d95fd9dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.085+0000 7f2f6de00640 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f9091c0 0x55d95fd9dd90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2fe34c9d28db45 server_cookie=2f1a1eaefb411f33 in_seq=0 out_seq=0 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.085+0000 7f2f4fe00640 10 client.16368 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.107+0000 7f2f4fe00640 1 -- 172.21.15.42:0/2617930732 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d95fc842c0 con 0x55d95f9091c0 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.107+0000 7f2f4fe00640 10 client.16368 handle_command_reply: tid=0 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.107+0000 7f2f7c05d740 1 client.16368 shutdown 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 msgr2=0x55d95fd270b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 0x55d95fd270b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c011b80 tx=0x7f2f5c00cb70 comp rx=0 tx=0).stop 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 shutdown_connections 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d95f9751b0 0x55d95fd270b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55d95f9091c0 0x55d95fd9dd90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d95f9544d0 0x55d95fd26a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 --2- 172.21.15.42:0/2617930732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d95f94c2f0 0x55d95fd2a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f4fe00640 0 client.16368 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 >> 172.21.15.42:0/2617930732 conn(0x55d95f969300 msgr2=0x55d95fd21ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 shutdown_connections 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 1 -- 172.21.15.42:0/2617930732 wait complete. 2024-05-27T10:23:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.108+0000 7f2f7c05d740 20 client.16368 trim_cache size 0 max 16384 2024-05-27T10:23:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:23:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:23:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:23:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2f540b29d0 msgr2=0x7f2f540b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2f540b29d0 0x7f2f540b4ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f68100e30 tx=0x7f2f680095c0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 msgr2=0x7f2f70104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70104030 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009dc0 tx=0x7f2f68003f10 comp rx=0 tx=0).stop 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 shutdown_connections 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f2f540b29d0 0x7f2f540b4ee0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f701102a0 0x7f2f70104030 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f7010d940 0x7f2f70103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 --2- 172.21.15.42:0/835916168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70075660 0x7f2f701035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 >> 172.21.15.42:0/835916168 conn(0x7f2f70079140 msgr2=0x7f2f70102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 shutdown_connections 2024-05-27T10:23:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:46.109+0000 7f2f6f200640 1 -- 172.21.15.42:0/835916168 wait complete. 2024-05-27T10:23:46.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.300+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34cc19e290 con 0x7f34cc108f10 2024-05-27T10:23:46.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.302+0000 7f34ca000700 1 -- 172.21.15.42:0/3235715307 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11880 (secure 0 0 0) 0x7f34b80aaa40 con 0x7f34cc108f10 2024-05-27T10:23:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:23:46.322 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:23:29.900931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f34b40b2a70 msgr2=0x7f34b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f34b40b2a70 0x7f34b40b4f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005950 tx=0x7f34c00058e0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 msgr2=0x7f34cc19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc19d4d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f34b800b580 tx=0x7f34b800f510 comp rx=0 tx=0).stop 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 shutdown_connections 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f34b40b2a70 0x7f34b40b4f80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34cc109860 0x7f34cc1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34cc1045e0 0x7f34cc19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 --2- 172.21.15.42:0/3235715307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34cc108f10 0x7f34cc19d4d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:46.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 >> 172.21.15.42:0/3235715307 conn(0x7f34cc100030 msgr2=0x7f34cc10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:46.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.305+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 shutdown_connections 2024-05-27T10:23:46.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:46.306+0000 7f34d4200700 1 -- 172.21.15.42:0/3235715307 wait complete. 2024-05-27T10:23:46.325 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 31 2024-05-27T10:23:46.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:46 smithi104 ceph-mon[34812]: pgmap v154: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-27T10:23:46.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:46 smithi143 ceph-mon[35545]: pgmap v154: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-27T10:23:46.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:46 smithi042 ceph-mon[31661]: pgmap v154: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-27T10:23:47.253 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id e15a8e9 --release --await' 2024-05-27T10:23:47.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id e15a8e9 --release --await 2024-05-27T10:23:47.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:47 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3235715307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:47.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:47 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3235715307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:47.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:47 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3235715307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:23:47.705 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:23:48.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:48 smithi143 ceph-mon[35545]: pgmap v155: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-27T10:23:48.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:48 smithi042 ceph-mon[31661]: pgmap v155: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-27T10:23:48.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:48 smithi104 ceph-mon[34812]: pgmap v155: 97 pgs: 97 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-27T10:23:48.715 INFO:tasks.workunit.client.0.smithi042.stdout: 9 0 0 0 0 0 0 2024-05-27T10:23:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 -- 172.21.15.42:0/2766175870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010a7b0 msgr2=0x7f8cf010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 --2- 172.21.15.42:0/2766175870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010a7b0 0x7f8cf010abc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009990 tx=0x7f8cd8009ca0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 -- 172.21.15.42:0/2766175870 shutdown_connections 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 --2- 172.21.15.42:0/2766175870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cf010b100 0x7f8cf0117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 --2- 172.21.15.42:0/2766175870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010f6b0 0x7f8cf010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 --2- 172.21.15.42:0/2766175870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010a7b0 0x7f8cf010abc0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.252+0000 7f8cefe00700 1 -- 172.21.15.42:0/2766175870 >> 172.21.15.42:0/2766175870 conn(0x7f8cf0076af0 msgr2=0x7f8cf0076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.253+0000 7f8cefe00700 1 -- 172.21.15.42:0/2766175870 shutdown_connections 2024-05-27T10:23:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.253+0000 7f8cefe00700 1 -- 172.21.15.42:0/2766175870 wait complete. 2024-05-27T10:23:50.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.254+0000 7f8cefe00700 1 Processor -- start 2024-05-27T10:23:50.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.254+0000 7f8cefe00700 1 -- start start 2024-05-27T10:23:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.254+0000 7f8cefe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010a7b0 0x7f8cf01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.254+0000 7f8cefe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cf010b100 0x7f8cf01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.254+0000 7f8cefe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 0x7f8cf01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cefe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf011a7a0 con 0x7f8cf010f6b0 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cefe00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf011a620 con 0x7f8cf010b100 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cefe00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf011a920 con 0x7f8cf010a7b0 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 0x7f8cf01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 0x7f8cf01a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54514/0 (socket says 172.21.15.42:54514) 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 -- 172.21.15.42:0/451485024 learned_addr learned my addr 172.21.15.42:0/451485024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8ce6000700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cf010b100 0x7f8cf01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8ceea00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010a7b0 0x7f8cf01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 -- 172.21.15.42:0/451485024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010a7b0 msgr2=0x7f8cf01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010a7b0 0x7f8cf01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 -- 172.21.15.42:0/451485024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cf010b100 msgr2=0x7f8cf01a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cf010b100 0x7f8cf01a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8cef400700 1 -- 172.21.15.42:0/451485024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01a6080 con 0x7f8cf010f6b0 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.255+0000 7f8ceea00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010a7b0 0x7f8cf01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.256+0000 7f8cef400700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 0x7f8cf01a59e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007ee0 tx=0x7f8ce800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.256+0000 7f8ce7e00700 1 -- 172.21.15.42:0/451485024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800dbd0 con 0x7f8cf010f6b0 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.256+0000 7f8ce7e00700 1 -- 172.21.15.42:0/451485024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ce800dd70 con 0x7f8cf010f6b0 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.256+0000 7f8ce7e00700 1 -- 172.21.15.42:0/451485024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80058d0 con 0x7f8cf010f6b0 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.256+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01a6370 con 0x7f8cf010f6b0 2024-05-27T10:23:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.256+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01adc90 con 0x7f8cf010f6b0 2024-05-27T10:23:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.262+0000 7f8ce7e00700 1 -- 172.21.15.42:0/451485024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ce801a020 con 0x7f8cf010f6b0 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.263+0000 7f8d04578b80 1 Processor -- start 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.263+0000 7f8d04578b80 1 -- start start 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.263+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549b9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8d04578b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559549ac8f40 con 0x559549b9be00 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8d04578b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559549ab8ae0 con 0x559549a1bf40 2024-05-27T10:23:50.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8d04578b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559549abcb20 con 0x559549b9b9f0 2024-05-27T10:23:50.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ce7e00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8cd40b2a70 0x7f8cd40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54524/0 (socket says 172.21.15.42:54524) 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ce6000700 1 -- 172.21.15.42:0/53626694 learned_addr learned my addr 172.21.15.42:0/53626694 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ce6000700 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549b9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ce7e00700 1 -- 172.21.15.42:0/451485024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8ce80e1e60 con 0x7f8cf010f6b0 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8ceea00700 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.264+0000 7f8cef400700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8cd40b2a70 0x7f8cd40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8ce6000700 1 -- 172.21.15.42:0/53626694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 msgr2=0x559549ba5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8ce6000700 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8ce6000700 1 -- 172.21.15.42:0/53626694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 msgr2=0x559549b9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8ce6000700 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549b9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8ce6000700 1 -- 172.21.15.42:0/53626694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559549a81520 con 0x559549b9be00 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8ce6000700 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba75a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009fd0 tx=0x7f8cd8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f9185fb97cf7b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8cef400700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8cd40b2a70 0x7f8cd40b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80a51b0 tx=0x7f8ce8004ef0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.265+0000 7f8cdfe00700 1 -- 172.21.15.42:0/53626694 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80055a0 con 0x559549b9be00 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.266+0000 7f8cdfe00700 1 -- 172.21.15.42:0/53626694 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cd8003c50 con 0x559549b9be00 2024-05-27T10:23:50.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.266+0000 7f8cdfe00700 1 -- 172.21.15.42:0/53626694 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800bd20 con 0x559549b9be00 2024-05-27T10:23:50.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 -- 172.21.15.42:0/53626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 msgr2=0x559549ba75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba75a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009fd0 tx=0x7f8cd8003680 comp rx=0 tx=0).stop 2024-05-27T10:23:50.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 -- 172.21.15.42:0/53626694 shutdown_connections 2024-05-27T10:23:50.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba75a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549b9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 --2- 172.21.15.42:0/53626694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 -- 172.21.15.42:0/53626694 >> 172.21.15.42:0/53626694 conn(0x559549a7c0c0 msgr2=0x559549b962c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:50.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 -- 172.21.15.42:0/53626694 shutdown_connections 2024-05-27T10:23:50.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.267+0000 7f8d04578b80 1 -- 172.21.15.42:0/53626694 wait complete. 2024-05-27T10:23:50.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.268+0000 7f8d04578b80 1 -- start start 2024-05-27T10:23:50.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.268+0000 7f8d04578b80 1 Processor -- start 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549ba4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5595499bf0a0 con 0x559549b9be00 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559549a59860 con 0x559549a1bf40 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559549ba7eb0 con 0x559549b9b9f0 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8d04578b80 10 client.0 _ll_register_callbacks cb 0x559549a62d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54530/0 (socket says 172.21.15.42:54530) 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8ce6000700 1 -- 172.21.15.42:0/3382834937 learned_addr learned my addr 172.21.15.42:0/3382834937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:50.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8ceea00700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.269+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549ba4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8ce6000700 1 -- 172.21.15.42:0/3382834937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 msgr2=0x559549ba4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8ce6000700 1 -- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 msgr2=0x559549ba4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549ba4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8ce6000700 1 -- 172.21.15.42:0/3382834937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559549a089d0 con 0x559549b9be00 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba5500 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8001eb0 tx=0x7f8cd801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb5d9bf0f017534 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8cdcc00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8cdcc00700 1 -- 172.21.15.42:0/3382834937 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd802b040 con 0x559549b9be00 2024-05-27T10:23:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd8003c50 con 0x559549b9be00 2024-05-27T10:23:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.270+0000 7f8cdcc00700 1 -- 172.21.15.42:0/3382834937 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8cd800b970 con 0x559549b9be00 2024-05-27T10:23:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8cdcc00700 1 -- 172.21.15.42:0/3382834937 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8cd800bb70 con 0x559549b9be00 2024-05-27T10:23:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 10 client.16380 fetch_fsmap learned FSMap version 31 2024-05-27T10:23:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559549a89160 con 0x559549b9be00 2024-05-27T10:23:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8cdcc00700 1 -- 172.21.15.42:0/3382834937 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cd8003c50 con 0x559549b9be00 2024-05-27T10:23:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8cdcc00700 1 -- 172.21.15.42:0/3382834937 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8cd8031030 con 0x559549b9be00 2024-05-27T10:23:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T10:23:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 10 client.16380 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 20 client.16380 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549bb4c90 0x559549bb70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 4 client.16380 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.271+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559549acae90 con 0x559549bb4c90 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.272+0000 7f8ceea00700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549bb4c90 0x559549bb70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.273+0000 7f8ceea00700 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549bb4c90 0x559549bb70b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0021503b15e435d server_cookie=cd29712082e1b7d4 in_seq=0 out_seq=0 2024-05-27T10:23:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.273+0000 7f8cdcc00700 10 client.16380 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8cdcc00700 1 -- 172.21.15.42:0/3382834937 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559549acae90 con 0x559549bb4c90 2024-05-27T10:23:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8cdcc00700 10 client.16380 handle_command_reply: tid=0 2024-05-27T10:23:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 client.16380 shutdown 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 msgr2=0x559549ba5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba5500 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8001eb0 tx=0x7f8cd801a040 comp rx=0 tx=0).stop 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 shutdown_connections 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549bb4c90 0x559549bb70b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549ba5500 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549a1bf40 0x559549ba4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3382834937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549ba4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 >> 172.21.15.42:0/3382834937 conn(0x559549a7c0c0 msgr2=0x559549b99e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8cdcc00700 0 client.16380 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.275+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 shutdown_connections 2024-05-27T10:23:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.276+0000 7f8d04578b80 1 -- 172.21.15.42:0/3382834937 wait complete. 2024-05-27T10:23:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.276+0000 7f8d04578b80 20 client.16380 trim_cache size 0 max 16384 2024-05-27T10:23:50.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.298+0000 7f8d04578b80 1 Processor -- start 2024-05-27T10:23:50.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.298+0000 7f8d04578b80 1 -- start start 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 0x559549ba6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549b9a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9be00 0x559549b9ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8d04578b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b7a80 con 0x559549a1bf40 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8d04578b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b52e0 con 0x559549b9be00 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8d04578b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b3150 con 0x559549b9b9f0 2024-05-27T10:23:50.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 0x559549ba6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 0x559549ba6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54540/0 (socket says 172.21.15.42:54540) 2024-05-27T10:23:50.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 -- 172.21.15.42:0/3465292702 learned_addr learned my addr 172.21.15.42:0/3465292702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:50.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ceea00700 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549b9a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 -- 172.21.15.42:0/3465292702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 msgr2=0x559549b9a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549b9a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 -- 172.21.15.42:0/3465292702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9be00 msgr2=0x559549b9ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9be00 0x559549b9ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.299+0000 7f8ce6000700 1 -- 172.21.15.42:0/3465292702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559549b9caf0 con 0x559549a1bf40 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.300+0000 7f8ce6000700 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 0x559549ba6b60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006010 tx=0x7f8cd803ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666306ad450c2646 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.300+0000 7f8cdf400700 1 -- 172.21.15.42:0/3465292702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800bd10 con 0x559549a1bf40 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.300+0000 7f8cdf400700 1 -- 172.21.15.42:0/3465292702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cd803c910 con 0x559549a1bf40 2024-05-27T10:23:50.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.300+0000 7f8cdf400700 1 -- 172.21.15.42:0/3465292702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800f5d0 con 0x559549a1bf40 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 -- 172.21.15.42:0/3465292702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 msgr2=0x559549ba6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 0x559549ba6b60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006010 tx=0x7f8cd803ced0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 -- 172.21.15.42:0/3465292702 shutdown_connections 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9be00 0x559549b9ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549a1bf40 0x559549ba6b60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 --2- 172.21.15.42:0/3465292702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549b9b9f0 0x559549b9a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 -- 172.21.15.42:0/3465292702 >> 172.21.15.42:0/3465292702 conn(0x559549a7c0c0 msgr2=0x559549ba4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 -- 172.21.15.42:0/3465292702 shutdown_connections 2024-05-27T10:23:50.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.301+0000 7f8d04578b80 1 -- 172.21.15.42:0/3465292702 wait complete. 2024-05-27T10:23:50.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.302+0000 7f8d04578b80 1 -- start start 2024-05-27T10:23:50.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.302+0000 7f8d04578b80 1 Processor -- start 2024-05-27T10:23:50.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549a1bf40 0x559549bb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9b9f0 0x559549bb7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549bb84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b37a0 con 0x559549b9be00 2024-05-27T10:23:50.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b4c10 con 0x559549b9b9f0 2024-05-27T10:23:50.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b3e70 con 0x559549a1bf40 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8d04578b80 10 client.0 _ll_register_callbacks cb 0x559549a62d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549bb84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8ce6000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549bb84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54554/0 (socket says 172.21.15.42:54554) 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8ce6000700 1 -- 172.21.15.42:0/1229189530 learned_addr learned my addr 172.21.15.42:0/1229189530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8ce6000700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549a1bf40 0x559549bb7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.303+0000 7f8ceea00700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9b9f0 0x559549bb7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ce6000700 1 -- 172.21.15.42:0/1229189530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549a1bf40 msgr2=0x559549bb7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ce6000700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549a1bf40 0x559549bb7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ce6000700 1 -- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9b9f0 msgr2=0x559549bb7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ce6000700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9b9f0 0x559549bb7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ce6000700 1 -- 172.21.15.42:0/1229189530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559549c1e500 con 0x559549b9be00 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ceea00700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9b9f0 0x559549bb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:50.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8ce6000700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549bb84d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b6c0 tx=0x7f8cd8009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4886b8f3e7ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:50.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.304+0000 7f8cdcc00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8cdcc00700 1 -- 172.21.15.42:0/1229189530 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd803c680 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd803c910 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8cdcc00700 1 -- 172.21.15.42:0/1229189530 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f8cd800f5d0 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8cdcc00700 1 -- 172.21.15.42:0/1229189530 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8cd800f7d0 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8d04578b80 10 client.16386 fetch_fsmap learned FSMap version 31 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559549c1f750 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8cdcc00700 1 -- 172.21.15.42:0/1229189530 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cd803c910 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8cdcc00700 1 -- 172.21.15.42:0/1229189530 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8cd8031050 con 0x559549b9be00 2024-05-27T10:23:50.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8d04578b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.305+0000 7f8d04578b80 10 client.16386 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.306+0000 7f8d04578b80 20 client.16386 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.306+0000 7f8d04578b80 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549c1fbc0 0x559549c21fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.306+0000 7f8d04578b80 4 client.16386 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e15a8e9", "release": true, "await": true}] 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.306+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e15a8e9", "release": true, "await": true}) -- 0x559549ac4670 con 0x559549c1fbc0 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.306+0000 7f8ceea00700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549c1fbc0 0x559549c21fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.307+0000 7f8ceea00700 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549c1fbc0 0x559549c21fe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2515d910e6abe3 server_cookie=4cfa15efb9853b49 in_seq=0 out_seq=0 2024-05-27T10:23:50.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.307+0000 7f8cdcc00700 10 client.16386 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:50.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.319+0000 7f8cdcc00700 1 -- 172.21.15.42:0/1229189530 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559549ac4670 con 0x559549c1fbc0 2024-05-27T10:23:50.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.319+0000 7f8cdcc00700 10 client.16386 handle_command_reply: tid=0 2024-05-27T10:23:50.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.319+0000 7f8d04578b80 1 client.16386 shutdown 2024-05-27T10:23:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 msgr2=0x559549bb84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549bb84d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b6c0 tx=0x7f8cd8009fd0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 shutdown_connections 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x559549c1fbc0 0x559549c21fe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559549b9be00 0x559549bb84d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559549a1bf40 0x559549bb7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 --2- 172.21.15.42:0/1229189530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559549b9b9f0 0x559549bb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8cdcc00700 0 client.16386 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 >> 172.21.15.42:0/1229189530 conn(0x559549a7c0c0 msgr2=0x559549ba5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 shutdown_connections 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 1 -- 172.21.15.42:0/1229189530 wait complete. 2024-05-27T10:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.320+0000 7f8d04578b80 20 client.16386 trim_cache size 0 max 16384 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 31, 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 16, 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "e15a8e9": { 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 16, 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:23:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 150.0, 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:23:50.323 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:23:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.321+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8cd40b2a70 msgr2=0x7f8cd40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.321+0000 7f8cefe00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8cd40b2a70 0x7f8cd40b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80a51b0 tx=0x7f8ce8004ef0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.321+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 msgr2=0x7f8cf01a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.321+0000 7f8cefe00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 0x7f8cf01a59e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007ee0 tx=0x7f8ce800d3b0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 shutdown_connections 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f8cd40b2a70 0x7f8cd40b4f80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cf010f6b0 0x7f8cf01a59e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010a7b0 0x7f8cf01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 --2- 172.21.15.42:0/451485024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cf010b100 0x7f8cf01a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 >> 172.21.15.42:0/451485024 conn(0x7f8cf0076af0 msgr2=0x7f8cf01088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 shutdown_connections 2024-05-27T10:23:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:23:50.322+0000 7f8cefe00700 1 -- 172.21.15.42:0/451485024 wait complete. 2024-05-27T10:23:50.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:50 smithi143 ceph-mon[35545]: pgmap v156: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-27T10:23:50.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:50 smithi042 ceph-mon[31661]: pgmap v156: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-27T10:23:50.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:50 smithi104 ceph-mon[34812]: pgmap v156: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-27T10:23:51.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e15a8e9', total seconds elapsed: 38.5 2024-05-27T10:23:52.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:52 smithi143 ceph-mon[35545]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-27T10:23:52.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:52 smithi042 ceph-mon[31661]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-27T10:23:52.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:52 smithi104 ceph-mon[34812]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-27T10:23:54.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:54 smithi143 ceph-mon[35545]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-27T10:23:54.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:54 smithi042 ceph-mon[31661]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-27T10:23:54.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:54 smithi104 ceph-mon[34812]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-27T10:23:56.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:56 smithi143 ceph-mon[35545]: pgmap v159: 97 pgs: 97 active+clean; 68 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 309 KiB/s wr, 22 op/s 2024-05-27T10:23:56.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:56 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:56.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:56 smithi042 ceph-mon[31661]: pgmap v159: 97 pgs: 97 active+clean; 68 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 309 KiB/s wr, 22 op/s 2024-05-27T10:23:56.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:56 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:56.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:56 smithi104 ceph-mon[34812]: pgmap v159: 97 pgs: 97 active+clean; 68 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 309 KiB/s wr, 22 op/s 2024-05-27T10:23:56.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:56 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:23:58.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:23:58 smithi143 ceph-mon[35545]: pgmap v160: 97 pgs: 97 active+clean; 68 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 308 KiB/s wr, 19 op/s 2024-05-27T10:23:58.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:23:58 smithi042 ceph-mon[31661]: pgmap v160: 97 pgs: 97 active+clean; 68 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 308 KiB/s wr, 19 op/s 2024-05-27T10:23:58.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:23:58 smithi104 ceph-mon[34812]: pgmap v160: 97 pgs: 97 active+clean; 68 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 308 KiB/s wr, 19 op/s 2024-05-27T10:23:58.715 INFO:tasks.workunit.client.0.smithi042.stdout: 11 6977 119 5099 123 894 74 2024-05-27T10:23:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 -- 172.21.15.42:0/866501692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50075420 msgr2=0x7f1a50075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 --2- 172.21.15.42:0/866501692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50075420 0x7f1a50075830 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402a480 comp rx=0 tx=0).stop 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 -- 172.21.15.42:0/866501692 shutdown_connections 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 --2- 172.21.15.42:0/866501692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a50110320 0x7f1a50112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 --2- 172.21.15.42:0/866501692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a5010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 --2- 172.21.15.42:0/866501692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50075420 0x7f1a50075830 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 -- 172.21.15.42:0/866501692 >> 172.21.15.42:0/866501692 conn(0x7f1a50079100 msgr2=0x7f1a50079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 -- 172.21.15.42:0/866501692 shutdown_connections 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.395+0000 7f1a56a00640 1 -- 172.21.15.42:0/866501692 wait complete. 2024-05-27T10:23:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.396+0000 7f1a56a00640 1 Processor -- start 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.396+0000 7f1a56a00640 1 -- start start 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a50075420 0x7f1a50103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a50103b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50110320 0x7f1a501040b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50115d90 con 0x7f1a50110320 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50115c10 con 0x7f1a5010d9c0 2024-05-27T10:23:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50115f10 con 0x7f1a50075420 2024-05-27T10:23:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.397+0000 7f1a56000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50110320 0x7f1a501040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a50103b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a50103b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:54164/0 (socket says 172.21.15.42:54164) 2024-05-27T10:23:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 -- 172.21.15.42:0/1358432690 learned_addr learned my addr 172.21.15.42:0/1358432690 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a55600640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a50075420 0x7f1a50103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 -- 172.21.15.42:0/1358432690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a50075420 msgr2=0x7f1a50103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a50075420 0x7f1a50103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 -- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50110320 msgr2=0x7f1a501040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50110320 0x7f1a501040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.398+0000 7f1a54c00640 1 -- 172.21.15.42:0/1358432690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a501ace00 con 0x7f1a5010d9c0 2024-05-27T10:23:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.399+0000 7f1a54c00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a50103b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d770 tx=0x7f1a3c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.399+0000 7f1a4e000640 1 -- 172.21.15.42:0/1358432690 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c017070 con 0x7f1a5010d9c0 2024-05-27T10:23:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.399+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a501ad090 con 0x7f1a5010d9c0 2024-05-27T10:23:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.399+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a501ad630 con 0x7f1a5010d9c0 2024-05-27T10:23:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.399+0000 7f1a4e000640 1 -- 172.21.15.42:0/1358432690 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a3c009c30 con 0x7f1a5010d9c0 2024-05-27T10:23:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.399+0000 7f1a4e000640 1 -- 172.21.15.42:0/1358432690 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00f3c0 con 0x7f1a5010d9c0 2024-05-27T10:23:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.404+0000 7f1a5efba740 1 Processor -- start 2024-05-27T10:23:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.404+0000 7f1a5efba740 1 -- start start 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a4e000640 1 -- 172.21.15.42:0/1358432690 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1a3c01a070 con 0x7f1a5010d9c0 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 0x558982299a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ece2f0 0x558982299fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a5efba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55898220ab90 con 0x558981ed64d0 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a5efba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589821fa730 con 0x558981ef71b0 2024-05-27T10:23:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.405+0000 7f1a5efba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589821fe770 con 0x558981ece2f0 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a55600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ece2f0 0x558982299fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 0x558982299a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 0x558982299a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:54180/0 (socket says 172.21.15.42:54180) 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 -- 172.21.15.42:0/1343633926 learned_addr learned my addr 172.21.15.42:0/1343633926 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 -- 172.21.15.42:0/1343633926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ece2f0 msgr2=0x558982299fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ece2f0 0x558982299fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 -- 172.21.15.42:0/1343633926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 msgr2=0x55898229a4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.406+0000 7f1a56000640 1 -- 172.21.15.42:0/1343633926 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589821938b0 con 0x558981ef71b0 2024-05-27T10:23:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.407+0000 7f1a4e000640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1a380b2a50 0x7f1a380b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.407+0000 7f1a56000640 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 0x558982299a60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800ade0 tx=0x7f1a4800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa23bf49793bebc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.407+0000 7f1a4e000640 1 -- 172.21.15.42:0/1358432690 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1a3c0e1ea0 con 0x7f1a5010d9c0 2024-05-27T10:23:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.407+0000 7f1a42600640 1 -- 172.21.15.42:0/1343633926 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004d20 con 0x558981ef71b0 2024-05-27T10:23:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.407+0000 7f1a55600640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1a380b2a50 0x7f1a380b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.408+0000 7f1a42600640 1 -- 172.21.15.42:0/1343633926 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a48003e00 con 0x558981ef71b0 2024-05-27T10:23:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.408+0000 7f1a42600640 1 -- 172.21.15.42:0/1343633926 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48010640 con 0x558981ef71b0 2024-05-27T10:23:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 -- 172.21.15.42:0/1343633926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 msgr2=0x558982299a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 0x558982299a60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800ade0 tx=0x7f1a4800f790 comp rx=0 tx=0).stop 2024-05-27T10:23:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a55600640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1a380b2a50 0x7f1a380b4f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a44005a20 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 -- 172.21.15.42:0/1343633926 shutdown_connections 2024-05-27T10:23:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ece2f0 0x558982299fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 --2- 172.21.15.42:0/1343633926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ef71b0 0x558982299a60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.409+0000 7f1a5efba740 1 -- 172.21.15.42:0/1343633926 >> 172.21.15.42:0/1343633926 conn(0x558981eeb300 msgr2=0x558982296210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.410+0000 7f1a5efba740 1 -- 172.21.15.42:0/1343633926 shutdown_connections 2024-05-27T10:23:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.410+0000 7f1a5efba740 1 -- 172.21.15.42:0/1343633926 wait complete. 2024-05-27T10:23:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.410+0000 7f1a5efba740 1 -- start start 2024-05-27T10:23:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.411+0000 7f1a5efba740 1 Processor -- start 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 0x5589822a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ef71b0 0x5589822c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558981ee0790 con 0x558981ef71b0 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589821a2bc0 con 0x558981ece2f0 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589822b7ba0 con 0x558981ed64d0 2024-05-27T10:23:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a5efba740 10 client.0 _ll_register_callbacks cb 0x558982180400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 0x5589822a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.412+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 0x5589822a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:48580/0 (socket says 172.21.15.42:48580) 2024-05-27T10:23:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.413+0000 7f1a54c00640 1 -- 172.21.15.42:0/3794516706 learned_addr learned my addr 172.21.15.42:0/3794516706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.413+0000 7f1a56000640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ef71b0 0x5589822c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.413+0000 7f1a55600640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.413+0000 7f1a54c00640 1 -- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 msgr2=0x5589822a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.413+0000 7f1a54c00640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.413+0000 7f1a54c00640 1 -- 172.21.15.42:0/3794516706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ef71b0 msgr2=0x5589822c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.414+0000 7f1a54c00640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ef71b0 0x5589822c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.414+0000 7f1a54c00640 1 -- 172.21.15.42:0/3794516706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558981f0f210 con 0x558981ed64d0 2024-05-27T10:23:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.414+0000 7f1a56000640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ef71b0 0x5589822c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:23:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.414+0000 7f1a54c00640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 0x5589822a41c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d740 tx=0x7f1a3c0a8c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba30f9a4e20ab4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.414+0000 7f1a36a00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.415+0000 7f1a36a00640 1 -- 172.21.15.42:0/3794516706 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c021070 con 0x558981ed64d0 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.415+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a48003e00 con 0x558981ed64d0 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.415+0000 7f1a36a00640 1 -- 172.21.15.42:0/3794516706 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1a3c0ad170 con 0x558981ed64d0 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.416+0000 7f1a36a00640 1 -- 172.21.15.42:0/3794516706 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a3c0dfa70 con 0x558981ed64d0 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.416+0000 7f1a5efba740 10 client.25987 fetch_fsmap learned FSMap version 32 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.416+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558981fdb620 con 0x558981ed64d0 2024-05-27T10:23:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.417+0000 7f1a36a00640 1 -- 172.21.15.42:0/3794516706 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a48003e00 con 0x558981ed64d0 2024-05-27T10:23:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.417+0000 7f1a36a00640 1 -- 172.21.15.42:0/3794516706 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7f1a3c0a4ae0 con 0x558981ed64d0 2024-05-27T10:23:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.417+0000 7f1a5efba740 10 client.25987 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T10:23:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.417+0000 7f1a5efba740 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.417+0000 7f1a5efba740 20 client.25987 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.418+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981eb5d80 0x5589822c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.418+0000 7f1a5efba740 4 client.25987 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:23:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.418+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55898220cae0 con 0x558981eb5d80 2024-05-27T10:23:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.418+0000 7f1a55600640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981eb5d80 0x5589822c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.419+0000 7f1a55600640 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981eb5d80 0x5589822c6cd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdeda0cefffe3f70 server_cookie=808761a4ee1bb66e in_seq=0 out_seq=0 2024-05-27T10:23:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.419+0000 7f1a36a00640 10 client.25987 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.421+0000 7f1a36a00640 1 -- 172.21.15.42:0/3794516706 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55898220cae0 con 0x558981eb5d80 2024-05-27T10:23:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.421+0000 7f1a36a00640 10 client.25987 handle_command_reply: tid=0 2024-05-27T10:23:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.421+0000 7f1a5efba740 1 client.25987 shutdown 2024-05-27T10:23:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 msgr2=0x5589822a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 0x5589822a41c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d740 tx=0x7f1a3c0a8c80 comp rx=0 tx=0).stop 2024-05-27T10:23:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 shutdown_connections 2024-05-27T10:23:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ef71b0 0x5589822c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981eb5d80 0x5589822c6cd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ed64d0 0x5589822a41c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3794516706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 >> 172.21.15.42:0/3794516706 conn(0x558981eeb300 msgr2=0x55898229b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a36a00640 0 client.25987 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 shutdown_connections 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.422+0000 7f1a5efba740 1 -- 172.21.15.42:0/3794516706 wait complete. 2024-05-27T10:23:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.423+0000 7f1a5efba740 20 client.25987 trim_cache size 0 max 16384 2024-05-27T10:23:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.452+0000 7f1a5efba740 1 Processor -- start 2024-05-27T10:23:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.452+0000 7f1a5efba740 1 -- start start 2024-05-27T10:23:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.452+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.452+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a5efba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30010f40 con 0x558981ed64d0 2024-05-27T10:23:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a5efba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3000e7e0 con 0x558981ece2f0 2024-05-27T10:23:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a5efba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3000c630 con 0x558981ef71b0 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a55600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a55600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:54208/0 (socket says 172.21.15.42:54208) 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33970/0 (socket says 172.21.15.42:33970) 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a55600640 1 -- 172.21.15.42:0/406185554 learned_addr learned my addr 172.21.15.42:0/406185554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.453+0000 7f1a56000640 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a54c00640 1 -- 172.21.15.42:0/406185554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 msgr2=0x5589822c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a54c00640 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a54c00640 1 -- 172.21.15.42:0/406185554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 msgr2=0x5589822a63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a54c00640 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a54c00640 1 -- 172.21.15.42:0/406185554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898229c170 con 0x558981ed64d0 2024-05-27T10:23:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a54c00640 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229f480 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0a51f0 tx=0x7f1a3c0a8bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f477c3eff59e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a41c00640 1 -- 172.21.15.42:0/406185554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c021070 con 0x558981ed64d0 2024-05-27T10:23:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a41c00640 1 -- 172.21.15.42:0/406185554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a3c0ad520 con 0x558981ed64d0 2024-05-27T10:23:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.454+0000 7f1a41c00640 1 -- 172.21.15.42:0/406185554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0a7fd0 con 0x558981ed64d0 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 -- 172.21.15.42:0/406185554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 msgr2=0x55898229f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229f480 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0a51f0 tx=0x7f1a3c0a8bb0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 -- 172.21.15.42:0/406185554 shutdown_connections 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229f480 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 --2- 172.21.15.42:0/406185554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x5589822a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 -- 172.21.15.42:0/406185554 >> 172.21.15.42:0/406185554 conn(0x558981eeb300 msgr2=0x5589822a3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 -- 172.21.15.42:0/406185554 shutdown_connections 2024-05-27T10:23:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.456+0000 7f1a5efba740 1 -- 172.21.15.42:0/406185554 wait complete. 2024-05-27T10:23:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.457+0000 7f1a5efba740 1 -- start start 2024-05-27T10:23:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.457+0000 7f1a5efba740 1 Processor -- start 2024-05-27T10:23:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x55898229dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3000cca0 con 0x558981ed64d0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3000e110 con 0x558981ece2f0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3000d370 con 0x558981ef71b0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a5efba740 10 client.0 _ll_register_callbacks cb 0x7f1a3c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33978/0 (socket says 172.21.15.42:33978) 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a54c00640 1 -- 172.21.15.42:0/3389306622 learned_addr learned my addr 172.21.15.42:0/3389306622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.458+0000 7f1a55600640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x55898229dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a56000640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a54c00640 1 -- 172.21.15.42:0/3389306622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 msgr2=0x5589822bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a54c00640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a54c00640 1 -- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 msgr2=0x55898229dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:23:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a54c00640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x55898229dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a54c00640 1 -- 172.21.15.42:0/3389306622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898229ea20 con 0x558981ed64d0 2024-05-27T10:23:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a56000640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:23:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a55600640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x55898229dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a54c00640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229e230 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0ca370 tx=0x7f1a3c0ad420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89eb51b34908914 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a36a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.459+0000 7f1a36a00640 1 -- 172.21.15.42:0/3389306622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c021070 con 0x558981ed64d0 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.460+0000 7f1a36a00640 1 -- 172.21.15.42:0/3389306622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f1a3c0a83d0 con 0x558981ed64d0 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.460+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3c0ad520 con 0x558981ed64d0 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.460+0000 7f1a36a00640 1 -- 172.21.15.42:0/3389306622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a3c0a5030 con 0x558981ed64d0 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.460+0000 7f1a5efba740 10 client.16401 fetch_fsmap learned FSMap version 32 2024-05-27T10:23:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.461+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55898229ea20 con 0x558981ed64d0 2024-05-27T10:23:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.461+0000 7f1a36a00640 1 -- 172.21.15.42:0/3389306622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a3c0ad520 con 0x558981ed64d0 2024-05-27T10:23:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a36a00640 1 -- 172.21.15.42:0/3389306622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7f1a3c0c3c40 con 0x558981ed64d0 2024-05-27T10:23:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a5efba740 10 client.16401 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T10:23:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a5efba740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:23:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a5efba740 20 client.16401 populate_metadata read hostname 'smithi042' 2024-05-27T10:23:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981e8b1c0 0x558982331350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:23:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a5efba740 4 client.16401 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:23:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589822062c0 con 0x558981e8b1c0 2024-05-27T10:23:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.462+0000 7f1a55600640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981e8b1c0 0x558982331350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:23:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.463+0000 7f1a36a00640 10 client.16401 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.463+0000 7f1a55600640 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981e8b1c0 0x558982331350 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f6fe6da998f0e0 server_cookie=98949fbf28b793e9 in_seq=0 out_seq=0 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a36a00640 1 -- 172.21.15.42:0/3389306622 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5589822062c0 con 0x558981e8b1c0 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a36a00640 10 client.16401 handle_command_reply: tid=0 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 client.16401 shutdown 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 msgr2=0x55898229e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229e230 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0ca370 tx=0x7f1a3c0ad420 comp rx=0 tx=0).stop 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 shutdown_connections 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x558981e8b1c0 0x558982331350 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558981ef71b0 0x5589822bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558981ed64d0 0x55898229e230 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 --2- 172.21.15.42:0/3389306622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558981ece2f0 0x55898229dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a36a00640 0 client.16401 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:23:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.484+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 >> 172.21.15.42:0/3389306622 conn(0x558981eeb300 msgr2=0x558982299f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.485+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 shutdown_connections 2024-05-27T10:23:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.485+0000 7f1a5efba740 1 -- 172.21.15.42:0/3389306622 wait complete. 2024-05-27T10:23:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.485+0000 7f1a5efba740 20 client.16401 trim_cache size 0 max 16384 2024-05-27T10:23:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:23:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:23:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1a380b2a50 msgr2=0x7f1a380b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1a380b2a50 0x7f1a380b4f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a44005a20 comp rx=0 tx=0).stop 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 msgr2=0x7f1a50103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a50103b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d770 tx=0x7f1a3c00dc30 comp rx=0 tx=0).stop 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 shutdown_connections 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1a380b2a50 0x7f1a380b4f60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50110320 0x7f1a501040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a5010d9c0 0x7f1a50103b70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 --2- 172.21.15.42:0/1358432690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a50075420 0x7f1a50103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 >> 172.21.15.42:0/1358432690 conn(0x7f1a50079100 msgr2=0x7f1a501022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 shutdown_connections 2024-05-27T10:23:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:23:59.486+0000 7f1a56a00640 1 -- 172.21.15.42:0/1358432690 wait complete. 2024-05-27T10:24:00.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:00 smithi143 ceph-mon[35545]: pgmap v161: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-05-27T10:24:00.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:00 smithi042 ceph-mon[31661]: pgmap v161: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-05-27T10:24:00.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:00 smithi104 ceph-mon[34812]: pgmap v161: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-05-27T10:24:02.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:02 smithi143 ceph-mon[35545]: pgmap v162: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-05-27T10:24:02.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:02 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:24:02.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:02 smithi042 ceph-mon[31661]: pgmap v162: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-05-27T10:24:02.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:02 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:24:02.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:02 smithi104 ceph-mon[34812]: pgmap v162: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-05-27T10:24:02.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:02 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:24:04.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:04 smithi042 ceph-mon[31661]: pgmap v163: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-05-27T10:24:04.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:04 smithi143 ceph-mon[35545]: pgmap v163: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-05-27T10:24:04.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:04 smithi104 ceph-mon[34812]: pgmap v163: 97 pgs: 97 active+clean; 77 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-05-27T10:24:06.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:06 smithi143 ceph-mon[35545]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-05-27T10:24:06.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:06 smithi042 ceph-mon[31661]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-05-27T10:24:06.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:06 smithi104 ceph-mon[34812]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-05-27T10:24:08.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:08 smithi143 ceph-mon[35545]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-05-27T10:24:08.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:08 smithi042 ceph-mon[31661]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-05-27T10:24:08.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:08 smithi104 ceph-mon[34812]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-05-27T10:24:08.715 INFO:tasks.workunit.client.0.smithi042.stdout: 16 8941 197 6222 251 1271 84 2024-05-27T10:24:10.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:10 smithi143 ceph-mon[35545]: pgmap v166: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 140 op/s 2024-05-27T10:24:10.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:10 smithi042 ceph-mon[31661]: pgmap v166: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 140 op/s 2024-05-27T10:24:10.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:10 smithi104 ceph-mon[34812]: pgmap v166: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 140 op/s 2024-05-27T10:24:12.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:12 smithi143 ceph-mon[35545]: pgmap v167: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-05-27T10:24:12.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:12 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:24:12.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:12 smithi042 ceph-mon[31661]: pgmap v167: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-05-27T10:24:12.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:12 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:24:12.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:12 smithi104 ceph-mon[34812]: pgmap v167: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-05-27T10:24:12.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:12 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:24:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.796+0000 7fc68d400640 1 -- 172.21.15.42:0/3962494332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6881075d0 msgr2=0x7fc6881079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.796+0000 7fc68d400640 1 --2- 172.21.15.42:0/3962494332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6881075d0 0x7fc6881079e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc674009900 tx=0x7fc67402a400 comp rx=0 tx=0).stop 2024-05-27T10:24:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 -- 172.21.15.42:0/3962494332 shutdown_connections 2024-05-27T10:24:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 --2- 172.21.15.42:0/3962494332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 0x7fc688108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 --2- 172.21.15.42:0/3962494332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6881075d0 0x7fc6881079e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 --2- 172.21.15.42:0/3962494332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc688102b80 0x7fc688102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 -- 172.21.15.42:0/3962494332 >> 172.21.15.42:0/3962494332 conn(0x7fc6880782d0 msgr2=0x7fc688078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 -- 172.21.15.42:0/3962494332 shutdown_connections 2024-05-27T10:24:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.797+0000 7fc68d400640 1 -- 172.21.15.42:0/3962494332 wait complete. 2024-05-27T10:24:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.799+0000 7fc68d400640 1 Processor -- start 2024-05-27T10:24:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.799+0000 7fc68d400640 1 -- start start 2024-05-27T10:24:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc68d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc688102b80 0x7fc6881a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc68d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6881075d0 0x7fc6881a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc68d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 0x7fc6881a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc68d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc68811a310 con 0x7fc688102b80 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc68d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc68811a190 con 0x7fc688108000 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc68d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc68811a490 con 0x7fc6881075d0 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc687200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 0x7fc6881a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc687200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 0x7fc6881a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:60544/0 (socket says 172.21.15.42:60544) 2024-05-27T10:24:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc687200640 1 -- 172.21.15.42:0/1418583200 learned_addr learned my addr 172.21.15.42:0/1418583200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc686800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc688102b80 0x7fc6881a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.800+0000 7fc685e00640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6881075d0 0x7fc6881a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.801+0000 7fc686800640 1 -- 172.21.15.42:0/1418583200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6881075d0 msgr2=0x7fc6881a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.801+0000 7fc686800640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6881075d0 0x7fc6881a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.801+0000 7fc686800640 1 -- 172.21.15.42:0/1418583200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 msgr2=0x7fc6881a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.801+0000 7fc686800640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 0x7fc6881a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.801+0000 7fc686800640 1 -- 172.21.15.42:0/1418583200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6881a5c10 con 0x7fc688102b80 2024-05-27T10:24:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.801+0000 7fc686800640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc688102b80 0x7fc6881a0db0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc67800e9f0 tx=0x7fc67800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.802+0000 7fc67f400640 1 -- 172.21.15.42:0/1418583200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67800ccc0 con 0x7fc688102b80 2024-05-27T10:24:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.802+0000 7fc67f400640 1 -- 172.21.15.42:0/1418583200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6780108c0 con 0x7fc688102b80 2024-05-27T10:24:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.802+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6881a5f00 con 0x7fc688102b80 2024-05-27T10:24:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.802+0000 7fc67f400640 1 -- 172.21.15.42:0/1418583200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780187e0 con 0x7fc688102b80 2024-05-27T10:24:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.803+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6881ad880 con 0x7fc688102b80 2024-05-27T10:24:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.807+0000 7fc67f400640 1 -- 172.21.15.42:0/1418583200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc678010430 con 0x7fc688102b80 2024-05-27T10:24:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.808+0000 7fc69444c740 1 Processor -- start 2024-05-27T10:24:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 -- start start 2024-05-27T10:24:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc67f400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc66c0b2a50 0x7fc66c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc67f400640 1 -- 172.21.15.42:0/1418583200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc6780e2930 con 0x7fc688102b80 2024-05-27T10:24:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef82860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 0x55e6aef82da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebbf4d0 0x55e6aef832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e6aeef3b90 con 0x55e6aebbf4d0 2024-05-27T10:24:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e6aeee3730 con 0x55e6aebb72f0 2024-05-27T10:24:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.809+0000 7fc69444c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e6aeee7770 con 0x55e6aebe01b0 2024-05-27T10:24:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.810+0000 7fc685e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 0x55e6aef82da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.810+0000 7fc687200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebbf4d0 0x55e6aef832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.810+0000 7fc685e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 0x55e6aef82da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:60554/0 (socket says 172.21.15.42:60554) 2024-05-27T10:24:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.810+0000 7fc687200640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc66c0b2a50 0x7fc66c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.810+0000 7fc685e00640 1 -- 172.21.15.42:0/720244990 learned_addr learned my addr 172.21.15.42:0/720244990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 -- 172.21.15.42:0/720244990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 msgr2=0x55e6aef82860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc686800640 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef82860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef82860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 -- 172.21.15.42:0/720244990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 msgr2=0x55e6aef82da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 0x55e6aef82da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 -- 172.21.15.42:0/720244990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6aee7c8b0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc686800640 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef82860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:24:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc685e00640 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 0x55e6aef82da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:24:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebbf4d0 0x55e6aef832e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc6800049d0 tx=0x7fc6800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181ad58333045ce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.811+0000 7fc687200640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc66c0b2a50 0x7fc66c0b4f60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc6800094d0 tx=0x7fc680008000 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.812+0000 7fc673400640 1 -- 172.21.15.42:0/720244990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680025070 con 0x55e6aebbf4d0 2024-05-27T10:24:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.812+0000 7fc673400640 1 -- 172.21.15.42:0/720244990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc680013920 con 0x55e6aebbf4d0 2024-05-27T10:24:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.812+0000 7fc673400640 1 -- 172.21.15.42:0/720244990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680021b50 con 0x55e6aebbf4d0 2024-05-27T10:24:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 -- 172.21.15.42:0/720244990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebbf4d0 msgr2=0x55e6aef832e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebbf4d0 0x55e6aef832e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc6800049d0 tx=0x7fc6800107a0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 -- 172.21.15.42:0/720244990 shutdown_connections 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebbf4d0 0x55e6aef832e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebb72f0 0x55e6aef82da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 --2- 172.21.15.42:0/720244990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef82860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.813+0000 7fc69444c740 1 -- 172.21.15.42:0/720244990 >> 172.21.15.42:0/720244990 conn(0x55e6aebd4300 msgr2=0x55e6aef7f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.814+0000 7fc69444c740 1 -- 172.21.15.42:0/720244990 shutdown_connections 2024-05-27T10:24:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.814+0000 7fc69444c740 1 -- 172.21.15.42:0/720244990 wait complete. 2024-05-27T10:24:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.815+0000 7fc69444c740 1 -- start start 2024-05-27T10:24:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.815+0000 7fc69444c740 1 Processor -- start 2024-05-27T10:24:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.815+0000 7fc69444c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebb72f0 0x55e6aef8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.815+0000 7fc69444c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc69444c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 0x55e6aef98c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc69444c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e6aebc9790 con 0x55e6aebe01b0 2024-05-27T10:24:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc687200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 0x55e6aef98c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc685e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc687200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 0x55e6aef98c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39770/0 (socket says 172.21.15.42:39770) 2024-05-27T10:24:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc687200640 1 -- 172.21.15.42:0/3257133490 learned_addr learned my addr 172.21.15.42:0/3257133490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc686800640 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebb72f0 0x55e6aef8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e6aee8bbc0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.816+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e6aef8f930 con 0x55e6aebb72f0 2024-05-27T10:24:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.817+0000 7fc69444c740 10 client.0 _ll_register_callbacks cb 0x55e6aee69400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.817+0000 7fc687200640 1 -- 172.21.15.42:0/3257133490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebb72f0 msgr2=0x55e6aef8ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.817+0000 7fc687200640 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebb72f0 0x55e6aef8ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.817+0000 7fc687200640 1 -- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 msgr2=0x55e6aef8d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.817+0000 7fc687200640 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.817+0000 7fc687200640 1 -- 172.21.15.42:0/3257133490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6aebf8210 con 0x55e6aebe01b0 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.818+0000 7fc687200640 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 0x55e6aef98c60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc680010ca0 tx=0x7fc680009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e528604366f1d98f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.818+0000 7fc667e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.818+0000 7fc667e00640 1 -- 172.21.15.42:0/3257133490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68002e070 con 0x55e6aebe01b0 2024-05-27T10:24:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.818+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc680013920 con 0x55e6aebe01b0 2024-05-27T10:24:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.819+0000 7fc667e00640 1 -- 172.21.15.42:0/3257133490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc680004a00 con 0x55e6aebe01b0 2024-05-27T10:24:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.819+0000 7fc667e00640 1 -- 172.21.15.42:0/3257133490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc68003a050 con 0x55e6aebe01b0 2024-05-27T10:24:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.819+0000 7fc69444c740 10 client.16413 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.819+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e6aecc4620 con 0x55e6aebe01b0 2024-05-27T10:24:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc667e00640 1 -- 172.21.15.42:0/3257133490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc680013920 con 0x55e6aebe01b0 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc667e00640 1 -- 172.21.15.42:0/3257133490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc680020070 con 0x55e6aebe01b0 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc69444c740 10 client.16413 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc69444c740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc69444c740 20 client.16413 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc69444c740 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6aeb9ed80 0x55e6aef9eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc69444c740 4 client.16413 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.820+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6aeef5ae0 con 0x55e6aeb9ed80 2024-05-27T10:24:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.821+0000 7fc686800640 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6aeb9ed80 0x55e6aef9eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.822+0000 7fc686800640 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6aeb9ed80 0x55e6aef9eb90 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff73aead09d71ed2 server_cookie=b901da9f6d6b9b91 in_seq=0 out_seq=0 2024-05-27T10:24:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.822+0000 7fc667e00640 10 client.16413 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.824+0000 7fc667e00640 1 -- 172.21.15.42:0/3257133490 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e6aeef5ae0 con 0x55e6aeb9ed80 2024-05-27T10:24:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.824+0000 7fc667e00640 10 client.16413 handle_command_reply: tid=0 2024-05-27T10:24:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.824+0000 7fc69444c740 1 client.16413 shutdown 2024-05-27T10:24:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.824+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 msgr2=0x55e6aef98c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.824+0000 7fc69444c740 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 0x55e6aef98c60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc680010ca0 tx=0x7fc680009d50 comp rx=0 tx=0).stop 2024-05-27T10:24:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.824+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 shutdown_connections 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebe01b0 0x55e6aef98c60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6aeb9ed80 0x55e6aef9eb90 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 --2- 172.21.15.42:0/3257133490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebb72f0 0x55e6aef8ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 >> 172.21.15.42:0/3257133490 conn(0x55e6aebd4300 msgr2=0x55e6aef84580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc667e00640 0 client.16413 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 shutdown_connections 2024-05-27T10:24:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 1 -- 172.21.15.42:0/3257133490 wait complete. 2024-05-27T10:24:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.825+0000 7fc69444c740 20 client.16413 trim_cache size 0 max 16384 2024-05-27T10:24:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.856+0000 7fc69444c740 1 Processor -- start 2024-05-27T10:24:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.856+0000 7fc69444c740 1 -- start start 2024-05-27T10:24:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.858+0000 7fc69444c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 0x55e6aef7e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.858+0000 7fc69444c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.858+0000 7fc69444c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef98bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.858+0000 7fc69444c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881b88f0 con 0x55e6aebb72f0 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.858+0000 7fc69444c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881b6160 con 0x55e6aebbf4d0 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.858+0000 7fc69444c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881b3fb0 con 0x55e6aebe01b0 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc686800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 0x55e6aef7e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc686800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 0x55e6aef7e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60122/0 (socket says 172.21.15.42:60122) 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc686800640 1 -- 172.21.15.42:0/1834908355 learned_addr learned my addr 172.21.15.42:0/1834908355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc685e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc687200640 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef98bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc686800640 1 -- 172.21.15.42:0/1834908355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 msgr2=0x55e6aef98bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.859+0000 7fc686800640 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef98bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.860+0000 7fc686800640 1 -- 172.21.15.42:0/1834908355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 msgr2=0x55e6aef8d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.860+0000 7fc686800640 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.860+0000 7fc686800640 1 -- 172.21.15.42:0/1834908355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6aef83730 con 0x55e6aebb72f0 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.860+0000 7fc685e00640 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.860+0000 7fc687200640 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef98bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:24:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.861+0000 7fc686800640 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 0x55e6aef7e330 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc6780a9b90 tx=0x7fc678009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f945e61452f4d141 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.861+0000 7fc672a00640 1 -- 172.21.15.42:0/1834908355 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780adfb0 con 0x55e6aebb72f0 2024-05-27T10:24:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.861+0000 7fc672a00640 1 -- 172.21.15.42:0/1834908355 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6780b2a60 con 0x55e6aebb72f0 2024-05-27T10:24:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.861+0000 7fc672a00640 1 -- 172.21.15.42:0/1834908355 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a5570 con 0x55e6aebb72f0 2024-05-27T10:24:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.862+0000 7fc69444c740 1 -- 172.21.15.42:0/1834908355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 msgr2=0x55e6aef7e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.862+0000 7fc69444c740 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 0x55e6aef7e330 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc6780a9b90 tx=0x7fc678009700 comp rx=0 tx=0).stop 2024-05-27T10:24:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.862+0000 7fc69444c740 1 -- 172.21.15.42:0/1834908355 shutdown_connections 2024-05-27T10:24:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.862+0000 7fc69444c740 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebe01b0 0x55e6aef98bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.863+0000 7fc69444c740 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebbf4d0 0x55e6aef8d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.863+0000 7fc69444c740 1 --2- 172.21.15.42:0/1834908355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aebb72f0 0x55e6aef7e330 secure :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc6780a9b90 tx=0x7fc678009700 comp rx=0 tx=0).stop 2024-05-27T10:24:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.863+0000 7fc69444c740 1 -- 172.21.15.42:0/1834908355 >> 172.21.15.42:0/1834908355 conn(0x55e6aebd4300 msgr2=0x55e6aef83020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.863+0000 7fc69444c740 1 -- 172.21.15.42:0/1834908355 shutdown_connections 2024-05-27T10:24:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.863+0000 7fc69444c740 1 -- 172.21.15.42:0/1834908355 wait complete. 2024-05-27T10:24:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.864+0000 7fc69444c740 1 -- start start 2024-05-27T10:24:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.864+0000 7fc69444c740 1 Processor -- start 2024-05-27T10:24:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.865+0000 7fc69444c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 0x55e6aefa1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc69444c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebe01b0 0x55e6aefa23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc69444c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aeb741c0 0x55e6aef9d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc69444c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881b4620 con 0x55e6aeb741c0 2024-05-27T10:24:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc69444c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881b5a90 con 0x55e6aebe01b0 2024-05-27T10:24:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc686800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 0x55e6aefa1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc686800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 0x55e6aefa1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:37678/0 (socket says 172.21.15.42:37678) 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc686800640 1 -- 172.21.15.42:0/2576247867 learned_addr learned my addr 172.21.15.42:0/2576247867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc685e00640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebe01b0 0x55e6aefa23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc687200640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aeb741c0 0x55e6aef9d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.866+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881b4cf0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.867+0000 7fc69444c740 10 client.0 _ll_register_callbacks cb 0x7fc6800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.867+0000 7fc686800640 1 -- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebe01b0 msgr2=0x55e6aefa23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.867+0000 7fc686800640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebe01b0 0x55e6aefa23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.867+0000 7fc686800640 1 -- 172.21.15.42:0/2576247867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aeb741c0 msgr2=0x55e6aef9d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.867+0000 7fc686800640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aeb741c0 0x55e6aef9d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.867+0000 7fc686800640 1 -- 172.21.15.42:0/2576247867 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6aef9da00 con 0x55e6aebbf4d0 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.868+0000 7fc686800640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 0x55e6aefa1e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc67800cfd0 tx=0x7fc6780a9680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37be8bfc1eb72752 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.868+0000 7fc667e00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:24:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.868+0000 7fc667e00640 1 -- 172.21.15.42:0/2576247867 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780c42e0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.869+0000 7fc667e00640 1 -- 172.21.15.42:0/2576247867 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc6780e02f0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.869+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6780b2a60 con 0x55e6aebbf4d0 2024-05-27T10:24:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.869+0000 7fc667e00640 1 -- 172.21.15.42:0/2576247867 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc6780e04f0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.869+0000 7fc69444c740 10 client.26005 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.869+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e6af006a40 con 0x55e6aebbf4d0 2024-05-27T10:24:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.870+0000 7fc667e00640 1 -- 172.21.15.42:0/2576247867 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6780b2a60 con 0x55e6aebbf4d0 2024-05-27T10:24:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.870+0000 7fc667e00640 1 -- 172.21.15.42:0/2576247867 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc6780a88d0 con 0x55e6aebbf4d0 2024-05-27T10:24:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc69444c740 10 client.26005 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc69444c740 10 client.26005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc69444c740 20 client.26005 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc69444c740 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6af006fe0 0x55e6af009400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc69444c740 4 client.26005 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:24:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6aeeef2c0 con 0x55e6af006fe0 2024-05-27T10:24:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.871+0000 7fc685e00640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6af006fe0 0x55e6af009400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.872+0000 7fc667e00640 10 client.26005 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.872+0000 7fc685e00640 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6af006fe0 0x55e6af009400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec3b39089778950 server_cookie=f8238d066b74e80b in_seq=0 out_seq=0 2024-05-27T10:24:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.892+0000 7fc667e00640 1 -- 172.21.15.42:0/2576247867 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e6aeeef2c0 con 0x55e6af006fe0 2024-05-27T10:24:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.892+0000 7fc667e00640 10 client.26005 handle_command_reply: tid=0 2024-05-27T10:24:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.892+0000 7fc69444c740 1 client.26005 shutdown 2024-05-27T10:24:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.892+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 msgr2=0x55e6aefa1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.892+0000 7fc69444c740 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 0x55e6aefa1e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc67800cfd0 tx=0x7fc6780a9680 comp rx=0 tx=0).stop 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 shutdown_connections 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e6aeb741c0 0x55e6aef9d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6aebe01b0 0x55e6aefa23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e6af006fe0 0x55e6af009400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 --2- 172.21.15.42:0/2576247867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6aebbf4d0 0x55e6aefa1e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 >> 172.21.15.42:0/2576247867 conn(0x55e6aebd4300 msgr2=0x55e6aef82d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc667e00640 0 client.26005 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 shutdown_connections 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 1 -- 172.21.15.42:0/2576247867 wait complete. 2024-05-27T10:24:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.893+0000 7fc69444c740 20 client.26005 trim_cache size 0 max 16384 2024-05-27T10:24:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:24:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:24:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc66c0b2a50 msgr2=0x7fc66c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc66c0b2a50 0x7fc66c0b4f60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc6800094d0 tx=0x7fc680008000 comp rx=0 tx=0).stop 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc688102b80 msgr2=0x7fc6881a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc688102b80 0x7fc6881a0db0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc67800e9f0 tx=0x7fc67800eeb0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 shutdown_connections 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc66c0b2a50 0x7fc66c0b4f60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc688108000 0x7fc6881a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6881075d0 0x7fc6881a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 --2- 172.21.15.42:0/1418583200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc688102b80 0x7fc6881a0db0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 >> 172.21.15.42:0/1418583200 conn(0x7fc6880782d0 msgr2=0x7fc688105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.895+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 shutdown_connections 2024-05-27T10:24:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:12.896+0000 7fc68d400640 1 -- 172.21.15.42:0/1418583200 wait complete. 2024-05-27T10:24:13.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:13 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:13.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:13 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:13.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:13 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:13.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:13 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:13.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:13 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:13.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:13 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:14.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:14 smithi104 ceph-mon[34812]: pgmap v168: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-05-27T10:24:14.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:14 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:24:14.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:14 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:24:14.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:14 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:14.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:14 smithi143 ceph-mon[35545]: pgmap v168: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-05-27T10:24:14.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:14 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:24:14.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:14 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:24:14.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:14 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:14.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:14 smithi042 ceph-mon[31661]: pgmap v168: 97 pgs: 97 active+clean; 99 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-05-27T10:24:14.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:14 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:24:14.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:14 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:24:14.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:14 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:24:16.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:16 smithi104 ceph-mon[34812]: pgmap v169: 97 pgs: 97 active+clean; 106 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-05-27T10:24:16.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:16 smithi143 ceph-mon[35545]: pgmap v169: 97 pgs: 97 active+clean; 106 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-05-27T10:24:16.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:16 smithi042 ceph-mon[31661]: pgmap v169: 97 pgs: 97 active+clean; 106 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-05-27T10:24:18.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:18 smithi104 ceph-mon[34812]: pgmap v170: 97 pgs: 97 active+clean; 106 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.6 MiB/s wr, 92 op/s 2024-05-27T10:24:18.715 INFO:tasks.workunit.client.0.smithi042.stdout: 18 9088 166 6336 196 1490 93 2024-05-27T10:24:18.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:18 smithi143 ceph-mon[35545]: pgmap v170: 97 pgs: 97 active+clean; 106 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.6 MiB/s wr, 92 op/s 2024-05-27T10:24:18.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:18 smithi042 ceph-mon[31661]: pgmap v170: 97 pgs: 97 active+clean; 106 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.6 MiB/s wr, 92 op/s 2024-05-27T10:24:20.702 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:20 smithi104 ceph-mon[34812]: pgmap v171: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 142 op/s 2024-05-27T10:24:20.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:20 smithi143 ceph-mon[35545]: pgmap v171: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 142 op/s 2024-05-27T10:24:20.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:20 smithi042 ceph-mon[31661]: pgmap v171: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 142 op/s 2024-05-27T10:24:22.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:22 smithi104 ceph-mon[34812]: pgmap v172: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-27T10:24:22.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:22 smithi143 ceph-mon[35545]: pgmap v172: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-27T10:24:22.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:22 smithi042 ceph-mon[31661]: pgmap v172: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-27T10:24:24.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:24 smithi104 ceph-mon[34812]: pgmap v173: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-27T10:24:24.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:24 smithi143 ceph-mon[35545]: pgmap v173: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-27T10:24:24.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:24 smithi042 ceph-mon[31661]: pgmap v173: 97 pgs: 97 active+clean; 118 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.7 MiB/s wr, 96 op/s 2024-05-27T10:24:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 -- 172.21.15.42:0/429222368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 msgr2=0x7f3bb4105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 --2- 172.21.15.42:0/429222368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4105f80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009900 tx=0x7f3ba002a410 comp rx=0 tx=0).stop 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 -- 172.21.15.42:0/429222368 shutdown_connections 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 --2- 172.21.15.42:0/429222368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bb41065f0 0x7f3bb41129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 --2- 172.21.15.42:0/429222368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4105f80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 --2- 172.21.15.42:0/429222368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 0x7f3bb4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.187+0000 7f3bbac00640 1 -- 172.21.15.42:0/429222368 >> 172.21.15.42:0/429222368 conn(0x7f3bb4101520 msgr2=0x7f3bb4103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.188+0000 7f3bbac00640 1 -- 172.21.15.42:0/429222368 shutdown_connections 2024-05-27T10:24:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.188+0000 7f3bbac00640 1 -- 172.21.15.42:0/429222368 wait complete. 2024-05-27T10:24:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.189+0000 7f3bbac00640 1 Processor -- start 2024-05-27T10:24:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.189+0000 7f3bbac00640 1 -- start start 2024-05-27T10:24:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.190+0000 7f3bbac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bb41065f0 0x7f3bb4078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.190+0000 7f3bbac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 0x7f3bb4078ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.190+0000 7f3bbac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4078ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.190+0000 7f3bbac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4115f90 con 0x7f3bb410c340 2024-05-27T10:24:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.190+0000 7f3bbac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4115e10 con 0x7f3bb410bea0 2024-05-27T10:24:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.190+0000 7f3bbac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4116110 con 0x7f3bb41065f0 2024-05-27T10:24:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 0x7f3bb4078ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4078ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 0x7f3bb4078ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:50208/0 (socket says 172.21.15.42:50208) 2024-05-27T10:24:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb3400640 1 -- 172.21.15.42:0/82175697 learned_addr learned my addr 172.21.15.42:0/82175697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bb41065f0 0x7f3bb4078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb8c00640 1 -- 172.21.15.42:0/82175697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bb41065f0 msgr2=0x7f3bb4078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb8c00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bb41065f0 0x7f3bb4078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb8c00640 1 -- 172.21.15.42:0/82175697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 msgr2=0x7f3bb4078ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb8c00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 0x7f3bb4078ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.191+0000 7f3bb8c00640 1 -- 172.21.15.42:0/82175697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb41acf60 con 0x7f3bb410c340 2024-05-27T10:24:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.192+0000 7f3bb8c00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4078ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e940 tx=0x7f3bac00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.192+0000 7f3bb0c00640 1 -- 172.21.15.42:0/82175697 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac004d20 con 0x7f3bb410c340 2024-05-27T10:24:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.192+0000 7f3bb0c00640 1 -- 172.21.15.42:0/82175697 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bac005020 con 0x7f3bb410c340 2024-05-27T10:24:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.192+0000 7f3bb0c00640 1 -- 172.21.15.42:0/82175697 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0116d0 con 0x7f3bb410c340 2024-05-27T10:24:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.192+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb41ad250 con 0x7f3bb410c340 2024-05-27T10:24:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.193+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41ad720 con 0x7f3bb410c340 2024-05-27T10:24:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.196+0000 7f3bc1c2b740 1 Processor -- start 2024-05-27T10:24:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.198+0000 7f3bb0c00640 1 -- 172.21.15.42:0/82175697 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3bac00cb20 con 0x7f3bb410c340 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.198+0000 7f3bc1c2b740 1 -- start start 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e13860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e13da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5654a2d84b90 con 0x5654a2a504d0 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654a2d74730 con 0x5654a2a482f0 2024-05-27T10:24:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654a2d78770 con 0x5654a2a711b0 2024-05-27T10:24:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e13da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e13da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:50210/0 (socket says 172.21.15.42:50210) 2024-05-27T10:24:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.199+0000 7f3bb3e00640 1 -- 172.21.15.42:0/3029772504 learned_addr learned my addr 172.21.15.42:0/3029772504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3400640 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3e00640 1 -- 172.21.15.42:0/3029772504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 msgr2=0x5654a2e13860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e13860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3e00640 1 -- 172.21.15.42:0/3029772504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 msgr2=0x5654a2e142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb0c00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3b980b29d0 0x7f3b980b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3e00640 1 -- 172.21.15.42:0/3029772504 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a2d0d8b0 con 0x5654a2a482f0 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb3400640 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:24:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.200+0000 7f3bb0c00640 1 -- 172.21.15.42:0/82175697 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3bac0e29e0 con 0x7f3bb410c340 2024-05-27T10:24:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.201+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e13da0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009ba0 tx=0x7f3ba40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c556d6fe7b45e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.201+0000 7f3bb3400640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3b980b29d0 0x7f3b980b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.201+0000 7f3ba9200640 1 -- 172.21.15.42:0/3029772504 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4012450 con 0x5654a2a482f0 2024-05-27T10:24:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.201+0000 7f3ba9200640 1 -- 172.21.15.42:0/3029772504 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ba4012a80 con 0x5654a2a482f0 2024-05-27T10:24:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.201+0000 7f3ba9200640 1 -- 172.21.15.42:0/3029772504 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400f9f0 con 0x5654a2a482f0 2024-05-27T10:24:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.202+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/3029772504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 msgr2=0x5654a2e13da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.202+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e13da0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009ba0 tx=0x7f3ba40117b0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.202+0000 7f3bb3400640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3b980b29d0 0x7f3b980b4ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4079c90 tx=0x7f3ba0005ed0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.203+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/3029772504 shutdown_connections 2024-05-27T10:24:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.203+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.203+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e13da0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.203+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/3029772504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e13860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.203+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/3029772504 >> 172.21.15.42:0/3029772504 conn(0x5654a2a65300 msgr2=0x5654a2e10010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.204+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/3029772504 shutdown_connections 2024-05-27T10:24:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.204+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/3029772504 wait complete. 2024-05-27T10:24:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.205+0000 7f3bc1c2b740 1 -- start start 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.205+0000 7f3bc1c2b740 1 Processor -- start 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.205+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 0x5654a2e1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a504d0 0x5654a2e1def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e29cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5654a2a5a790 con 0x5654a2a482f0 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5654a2d1cbc0 con 0x5654a2a711b0 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654a2e20990 con 0x5654a2a504d0 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bc1c2b740 10 client.0 _ll_register_callbacks cb 0x5654a2cfa400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 0x5654a2e1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 0x5654a2e1d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57892/0 (socket says 172.21.15.42:57892) 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.206+0000 7f3bb3e00640 1 -- 172.21.15.42:0/38074932 learned_addr learned my addr 172.21.15.42:0/38074932 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb8c00640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e29cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb3e00640 1 -- 172.21.15.42:0/38074932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a504d0 msgr2=0x5654a2e1def0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a504d0 0x5654a2e1def0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb3e00640 1 -- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 msgr2=0x5654a2e29cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e29cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb3e00640 1 -- 172.21.15.42:0/38074932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a2a89210 con 0x5654a2a482f0 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb8c00640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e29cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 0x5654a2e1d9b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ba401b000 tx=0x7f3ba4012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e1ec5cf945b942 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.207+0000 7f3b9d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.208+0000 7f3b9d600640 1 -- 172.21.15.42:0/38074932 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4012860 con 0x5654a2a482f0 2024-05-27T10:24:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.208+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4012a80 con 0x5654a2a482f0 2024-05-27T10:24:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.208+0000 7f3b9d600640 1 -- 172.21.15.42:0/38074932 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3ba4004030 con 0x5654a2a482f0 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.208+0000 7f3b9d600640 1 -- 172.21.15.42:0/38074932 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ba4033050 con 0x5654a2a482f0 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.208+0000 7f3bc1c2b740 10 client.16431 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.208+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5654a2b55620 con 0x5654a2a482f0 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.209+0000 7f3b9d600640 1 -- 172.21.15.42:0/38074932 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ba4012a80 con 0x5654a2a482f0 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.209+0000 7f3b9d600640 1 -- 172.21.15.42:0/38074932 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3ba400f710 con 0x5654a2a482f0 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bc1c2b740 10 client.16431 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bc1c2b740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bc1c2b740 20 client.16431 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a2fd80 0x5654a2e2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bc1c2b740 4 client.16431 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a2d86ae0 con 0x5654a2a2fd80 2024-05-27T10:24:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.210+0000 7f3bb3400640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a2fd80 0x5654a2e2fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.212+0000 7f3bb3400640 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a2fd80 0x5654a2e2fb60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2630917f647eb62e server_cookie=c7f95ad7e794280d in_seq=0 out_seq=0 2024-05-27T10:24:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.212+0000 7f3b9d600640 10 client.16431 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.213+0000 7f3b9d600640 1 -- 172.21.15.42:0/38074932 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654a2d86ae0 con 0x5654a2a2fd80 2024-05-27T10:24:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.213+0000 7f3b9d600640 10 client.16431 handle_command_reply: tid=0 2024-05-27T10:24:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.213+0000 7f3bc1c2b740 1 client.16431 shutdown 2024-05-27T10:24:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.214+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 msgr2=0x5654a2e1d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.214+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 0x5654a2e1d9b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ba401b000 tx=0x7f3ba4012e60 comp rx=0 tx=0).stop 2024-05-27T10:24:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 shutdown_connections 2024-05-27T10:24:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e29cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a2fd80 0x5654a2e2fb60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a504d0 0x5654a2e1def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/38074932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a482f0 0x5654a2e1d9b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3b9d600640 0 client.16431 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 >> 172.21.15.42:0/38074932 conn(0x5654a2a65300 msgr2=0x5654a2e15580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 shutdown_connections 2024-05-27T10:24:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.215+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/38074932 wait complete. 2024-05-27T10:24:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.216+0000 7f3bc1c2b740 20 client.16431 trim_cache size 0 max 16384 2024-05-27T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 Processor -- start 2024-05-27T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 -- start start 2024-05-27T10:24:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e1fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b89f0 con 0x5654a2a504d0 2024-05-27T10:24:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b6260 con 0x5654a2a482f0 2024-05-27T10:24:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.244+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b40b0 con 0x5654a2a711b0 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e1d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57906/0 (socket says 172.21.15.42:57906) 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 -- 172.21.15.42:0/1038066477 learned_addr learned my addr 172.21.15.42:0/1038066477 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e1fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb8c00640 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 -- 172.21.15.42:0/1038066477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 msgr2=0x5654a2e1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e1de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 -- 172.21.15.42:0/1038066477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 msgr2=0x5654a2e1fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e1fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.245+0000 7f3bb3400640 1 -- 172.21.15.42:0/1038066477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a2e1bf00 con 0x5654a2a504d0 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.246+0000 7f3bb3400640 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e1d950 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0003820 tx=0x7f3ba003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2335575ff922092a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.246+0000 7f3b9fe00640 1 -- 172.21.15.42:0/1038066477 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0004840 con 0x5654a2a504d0 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.246+0000 7f3b9fe00640 1 -- 172.21.15.42:0/1038066477 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ba0004e70 con 0x5654a2a504d0 2024-05-27T10:24:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.246+0000 7f3b9fe00640 1 -- 172.21.15.42:0/1038066477 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0053980 con 0x5654a2a504d0 2024-05-27T10:24:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1038066477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 msgr2=0x5654a2e1d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e1d950 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0003820 tx=0x7f3ba003d000 comp rx=0 tx=0).stop 2024-05-27T10:24:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1038066477 shutdown_connections 2024-05-27T10:24:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a711b0 0x5654a2e1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e1d950 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1038066477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a482f0 0x5654a2e1fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.247+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1038066477 >> 172.21.15.42:0/1038066477 conn(0x5654a2a65300 msgr2=0x5654a2e1d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.248+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1038066477 shutdown_connections 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.248+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1038066477 wait complete. 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.248+0000 7f3bc1c2b740 1 -- start start 2024-05-27T10:24:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.249+0000 7f3bc1c2b740 1 Processor -- start 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.250+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a482f0 0x5654a2e20be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.250+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e21120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.250+0000 7f3bc1c2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e21660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.250+0000 7f3bb3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e21120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.250+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a482f0 0x5654a2e20be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.250+0000 7f3bb3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e21120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57908/0 (socket says 172.21.15.42:57908) 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.251+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a482f0 0x5654a2e20be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:40236/0 (socket says 172.21.15.42:40236) 2024-05-27T10:24:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.251+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b4720 con 0x5654a2a504d0 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.251+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b5b90 con 0x5654a2a711b0 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.251+0000 7f3bc1c2b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b4df0 con 0x5654a2a482f0 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.251+0000 7f3bc1c2b740 10 client.0 _ll_register_callbacks cb 0x7f3ba4009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.251+0000 7f3bb3400640 1 -- 172.21.15.42:0/1154566480 learned_addr learned my addr 172.21.15.42:0/1154566480 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.252+0000 7f3bb3400640 1 -- 172.21.15.42:0/1154566480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a482f0 msgr2=0x5654a2e20be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.252+0000 7f3bb3400640 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a482f0 0x5654a2e20be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.252+0000 7f3bb3400640 1 -- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 msgr2=0x5654a2e21660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.252+0000 7f3bb3400640 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e21660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.252+0000 7f3bb3400640 1 -- 172.21.15.42:0/1154566480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a2e96020 con 0x5654a2a504d0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.253+0000 7f3bb3400640 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e21120 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0004650 tx=0x7f3ba0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e106b9017cb705b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.253+0000 7f3b9d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.254+0000 7f3b9d600640 1 -- 172.21.15.42:0/1154566480 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0004c70 con 0x5654a2a504d0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.254+0000 7f3b9d600640 1 -- 172.21.15.42:0/1154566480 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f3ba00535b0 con 0x5654a2a504d0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.254+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba0004e70 con 0x5654a2a504d0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.254+0000 7f3b9d600640 1 -- 172.21.15.42:0/1154566480 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ba00537b0 con 0x5654a2a504d0 2024-05-27T10:24:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.254+0000 7f3bc1c2b740 10 client.16437 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.254+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5654a2e97250 con 0x5654a2a504d0 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3b9d600640 1 -- 172.21.15.42:0/1154566480 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ba0004e70 con 0x5654a2a504d0 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3b9d600640 1 -- 172.21.15.42:0/1154566480 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3ba0040460 con 0x5654a2a504d0 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3bc1c2b740 10 client.16437 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3bc1c2b740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3bc1c2b740 20 client.16437 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a051c0 0x5654a2e99910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3bc1c2b740 4 client.16437 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:24:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.255+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654a2d802c0 con 0x5654a2a051c0 2024-05-27T10:24:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.256+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a051c0 0x5654a2e99910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.257+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a051c0 0x5654a2e99910 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115cafd272c3ddcf server_cookie=b67f7a007c4a4e08 in_seq=0 out_seq=0 2024-05-27T10:24:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.257+0000 7f3b9d600640 10 client.16437 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.274+0000 7f3b9d600640 1 -- 172.21.15.42:0/1154566480 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5654a2d802c0 con 0x5654a2a051c0 2024-05-27T10:24:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.274+0000 7f3b9d600640 10 client.16437 handle_command_reply: tid=0 2024-05-27T10:24:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.274+0000 7f3bc1c2b740 1 client.16437 shutdown 2024-05-27T10:24:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 msgr2=0x5654a2e21120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e21120 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0004650 tx=0x7f3ba0009260 comp rx=0 tx=0).stop 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 shutdown_connections 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5654a2a711b0 0x5654a2e21660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654a2a504d0 0x5654a2e21120 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5654a2a051c0 0x5654a2e99910 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 --2- 172.21.15.42:0/1154566480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654a2a482f0 0x5654a2e20be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3b9d600640 0 client.16437 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 >> 172.21.15.42:0/1154566480 conn(0x5654a2a65300 msgr2=0x5654a2e1dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 shutdown_connections 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.275+0000 7f3bc1c2b740 1 -- 172.21.15.42:0/1154566480 wait complete. 2024-05-27T10:24:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.276+0000 7f3bc1c2b740 20 client.16437 trim_cache size 0 max 16384 2024-05-27T10:24:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:24:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:24:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.276+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3b980b29d0 msgr2=0x7f3b980b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.276+0000 7f3bbac00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3b980b29d0 0x7f3b980b4ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4079c90 tx=0x7f3ba0005ed0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.276+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 msgr2=0x7f3bb4078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.276+0000 7f3bbac00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4078ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e940 tx=0x7f3bac00ee00 comp rx=0 tx=0).stop 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 shutdown_connections 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f3b980b29d0 0x7f3b980b4ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb410c340 0x7f3bb4078ff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb410bea0 0x7f3bb4078ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 --2- 172.21.15.42:0/82175697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bb41065f0 0x7f3bb4078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 >> 172.21.15.42:0/82175697 conn(0x7f3bb4101520 msgr2=0x7f3bb4102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 shutdown_connections 2024-05-27T10:24:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:26.277+0000 7f3bbac00640 1 -- 172.21.15.42:0/82175697 wait complete. 2024-05-27T10:24:26.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:26 smithi143 ceph-mon[35545]: pgmap v174: 97 pgs: 97 active+clean; 129 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-05-27T10:24:26.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:26 smithi042 ceph-mon[31661]: pgmap v174: 97 pgs: 97 active+clean; 129 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-05-27T10:24:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:26 smithi104 ceph-mon[34812]: pgmap v174: 97 pgs: 97 active+clean; 129 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-05-27T10:24:28.716 INFO:tasks.workunit.client.0.smithi042.stdout: 21 9182 184 6522 163 1086 81 2024-05-27T10:24:28.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:28 smithi143 ceph-mon[35545]: pgmap v175: 97 pgs: 97 active+clean; 129 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-05-27T10:24:28.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:28 smithi042 ceph-mon[31661]: pgmap v175: 97 pgs: 97 active+clean; 129 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-05-27T10:24:28.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:28 smithi104 ceph-mon[34812]: pgmap v175: 97 pgs: 97 active+clean; 129 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 102 op/s 2024-05-27T10:24:30.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:30 smithi143 ceph-mon[35545]: pgmap v176: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-05-27T10:24:30.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:30 smithi042 ceph-mon[31661]: pgmap v176: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-05-27T10:24:30.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:30 smithi104 ceph-mon[34812]: pgmap v176: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-05-27T10:24:33.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:32 smithi143 ceph-mon[35545]: pgmap v177: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-05-27T10:24:33.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:32 smithi042 ceph-mon[31661]: pgmap v177: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-05-27T10:24:33.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:32 smithi104 ceph-mon[34812]: pgmap v177: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-05-27T10:24:35.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:34 smithi143 ceph-mon[35545]: pgmap v178: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-05-27T10:24:35.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:34 smithi042 ceph-mon[31661]: pgmap v178: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-05-27T10:24:35.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:34 smithi104 ceph-mon[34812]: pgmap v178: 97 pgs: 97 active+clean; 137 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-05-27T10:24:37.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:36 smithi143 ceph-mon[35545]: pgmap v179: 97 pgs: 97 active+clean; 146 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 137 op/s 2024-05-27T10:24:37.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:36 smithi042 ceph-mon[31661]: pgmap v179: 97 pgs: 97 active+clean; 146 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 137 op/s 2024-05-27T10:24:37.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:36 smithi104 ceph-mon[34812]: pgmap v179: 97 pgs: 97 active+clean; 146 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 137 op/s 2024-05-27T10:24:38.716 INFO:tasks.workunit.client.0.smithi042.stdout: 23 9098 182 6566 162 1670 54 2024-05-27T10:24:39.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:38 smithi143 ceph-mon[35545]: pgmap v180: 97 pgs: 97 active+clean; 146 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-05-27T10:24:39.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:38 smithi042 ceph-mon[31661]: pgmap v180: 97 pgs: 97 active+clean; 146 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-05-27T10:24:39.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:38 smithi104 ceph-mon[34812]: pgmap v180: 97 pgs: 97 active+clean; 146 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-05-27T10:24:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 -- 172.21.15.42:0/2270945684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec106420 msgr2=0x7fc0ec1127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 --2- 172.21.15.42:0/2270945684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec106420 0x7fc0ec1127f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009900 tx=0x7fc0d802a410 comp rx=0 tx=0).stop 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 -- 172.21.15.42:0/2270945684 shutdown_connections 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 --2- 172.21.15.42:0/2270945684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec106420 0x7fc0ec1127f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 --2- 172.21.15.42:0/2270945684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0ec10c170 0x7fc0ec105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 --2- 172.21.15.42:0/2270945684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 -- 172.21.15.42:0/2270945684 >> 172.21.15.42:0/2270945684 conn(0x7fc0ec1013d0 msgr2=0x7fc0ec103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.564+0000 7fc0eb200640 1 -- 172.21.15.42:0/2270945684 shutdown_connections 2024-05-27T10:24:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.565+0000 7fc0eb200640 1 -- 172.21.15.42:0/2270945684 wait complete. 2024-05-27T10:24:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.566+0000 7fc0eb200640 1 Processor -- start 2024-05-27T10:24:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.566+0000 7fc0eb200640 1 -- start start 2024-05-27T10:24:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0eb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec106420 0x7fc0ec19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0eb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0eb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0ec10c170 0x7fc0ec1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0eb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec115d70 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0eb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec115bf0 con 0x7fc0ec10c170 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0eb200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec115ef0 con 0x7fc0ec106420 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39740/0 (socket says 172.21.15.42:39740) 2024-05-27T10:24:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0e9400640 1 -- 172.21.15.42:0/1050332233 learned_addr learned my addr 172.21.15.42:0/1050332233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.567+0000 7fc0ea800640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0ec10c170 0x7fc0ec1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.568+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec106420 0x7fc0ec19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.568+0000 7fc0e9400640 1 -- 172.21.15.42:0/1050332233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec106420 msgr2=0x7fc0ec19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.568+0000 7fc0e9400640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec106420 0x7fc0ec19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.568+0000 7fc0e9400640 1 -- 172.21.15.42:0/1050332233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0ec10c170 msgr2=0x7fc0ec1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.568+0000 7fc0e9400640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0ec10c170 0x7fc0ec1a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.568+0000 7fc0e9400640 1 -- 172.21.15.42:0/1050332233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec1a15d0 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.569+0000 7fc0e9400640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec19ccd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9f0 tx=0x7fc0dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.569+0000 7fc0e2a00640 1 -- 172.21.15.42:0/1050332233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00ccc0 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.569+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec1a18c0 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.569+0000 7fc0e2a00640 1 -- 172.21.15.42:0/1050332233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0dc0044b0 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.569+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec1a9240 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.569+0000 7fc0e2a00640 1 -- 172.21.15.42:0/1050332233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc010430 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.574+0000 7fc0f7fed740 1 Processor -- start 2024-05-27T10:24:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.574+0000 7fc0e2a00640 1 -- 172.21.15.42:0/1050332233 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc0dc003f90 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.574+0000 7fc0f7fed740 1 -- start start 2024-05-27T10:24:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.575+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802eeea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.575+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802eeefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.575+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802eef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.575+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c802e5fb90 con 0x55c802b232f0 2024-05-27T10:24:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.575+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c802e4f730 con 0x55c802b2b4d0 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.575+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c802e53770 con 0x55c802b4c1b0 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.576+0000 7fc0e2a00640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc0d00b29a0 0x7fc0d00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.576+0000 7fc0e2a00640 1 -- 172.21.15.42:0/1050332233 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc0dc0e2860 con 0x7fc0ec10bcd0 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.576+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802eef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.576+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802eef4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:55968/0 (socket says 172.21.15.42:55968) 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.576+0000 7fc0e9400640 1 -- 172.21.15.42:0/4162818806 learned_addr learned my addr 172.21.15.42:0/4162818806 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.576+0000 7fc0ea800640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc0d00b29a0 0x7fc0d00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.577+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802eeefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.577+0000 7fc0e9e00640 1 -- 172.21.15.42:0/4162818806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 msgr2=0x55c802eeea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.578+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802eeea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.578+0000 7fc0e9e00640 1 -- 172.21.15.42:0/4162818806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 msgr2=0x55c802eef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.578+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802eef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.578+0000 7fc0e9e00640 1 -- 172.21.15.42:0/4162818806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c802de88b0 con 0x55c802b232f0 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.578+0000 7fc0e9400640 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802eef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.578+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802eeefa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005fd0 tx=0x7fc0e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb0f893d85f5ec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.579+0000 7fc0d7400640 1 -- 172.21.15.42:0/4162818806 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4003c30 con 0x55c802b232f0 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.579+0000 7fc0d7400640 1 -- 172.21.15.42:0/4162818806 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e4003dd0 con 0x55c802b232f0 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.579+0000 7fc0d7400640 1 -- 172.21.15.42:0/4162818806 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40197d0 con 0x55c802b232f0 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 -- 172.21.15.42:0/4162818806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 msgr2=0x55c802eeefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802eeefa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005fd0 tx=0x7fc0e4011810 comp rx=0 tx=0).stop 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0ea800640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc0d00b29a0 0x7fc0d00b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0d802a980 tx=0x7fc0d8005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 -- 172.21.15.42:0/4162818806 shutdown_connections 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802eef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802eeefa0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/4162818806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802eeea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 -- 172.21.15.42:0/4162818806 >> 172.21.15.42:0/4162818806 conn(0x55c802b40300 msgr2=0x55c802eeb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.580+0000 7fc0f7fed740 1 -- 172.21.15.42:0/4162818806 shutdown_connections 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.581+0000 7fc0f7fed740 1 -- 172.21.15.42:0/4162818806 wait complete. 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.581+0000 7fc0f7fed740 1 -- start start 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.582+0000 7fc0f7fed740 1 Processor -- start 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.582+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802ef8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.582+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802ef91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802f0d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c802b35790 con 0x55c802b232f0 2024-05-27T10:24:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c802df7bc0 con 0x55c802b2b4d0 2024-05-27T10:24:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c802efbb90 con 0x55c802b4c1b0 2024-05-27T10:24:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0f7fed740 10 client.0 _ll_register_callbacks cb 0x55c802dd5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0e9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802ef8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802ef91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0e9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802ef8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39768/0 (socket says 172.21.15.42:39768) 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0e9e00640 1 -- 172.21.15.42:0/2991543790 learned_addr learned my addr 172.21.15.42:0/2991543790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.583+0000 7fc0ea800640 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802f0d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.584+0000 7fc0e9e00640 1 -- 172.21.15.42:0/2991543790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 msgr2=0x55c802f0d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.584+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802f0d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.584+0000 7fc0e9e00640 1 -- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 msgr2=0x55c802ef91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.584+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802ef91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.584+0000 7fc0e9e00640 1 -- 172.21.15.42:0/2991543790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c802b64210 con 0x55c802b232f0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.585+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802ef8c70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004db0 tx=0x7fc0e4004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48097b472cd44803 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.585+0000 7fc0cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.586+0000 7fc0cbe00640 1 -- 172.21.15.42:0/2991543790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e401f040 con 0x55c802b232f0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.586+0000 7fc0cbe00640 1 -- 172.21.15.42:0/2991543790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc0e4004200 con 0x55c802b232f0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.586+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e4003dd0 con 0x55c802b232f0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.586+0000 7fc0cbe00640 1 -- 172.21.15.42:0/2991543790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc0e4004400 con 0x55c802b232f0 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.586+0000 7fc0f7fed740 10 client.16449 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.586+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c802c30620 con 0x55c802b232f0 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0cbe00640 1 -- 172.21.15.42:0/2991543790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e4003dd0 con 0x55c802b232f0 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0cbe00640 1 -- 172.21.15.42:0/2991543790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc0e4018070 con 0x55c802b232f0 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0f7fed740 10 client.16449 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0f7fed740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0f7fed740 20 client.16449 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802b0ad80 0x55c802f135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0f7fed740 4 client.16449 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.587+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c802e61ae0 con 0x55c802b0ad80 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.588+0000 7fc0e9400640 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802b0ad80 0x55c802f135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.589+0000 7fc0e9400640 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802b0ad80 0x55c802f135d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faef6ab10cf6f1 server_cookie=4b267003646f7f32 in_seq=0 out_seq=0 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.589+0000 7fc0cbe00640 10 client.16449 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0cbe00640 1 -- 172.21.15.42:0/2991543790 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c802e61ae0 con 0x55c802b0ad80 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0cbe00640 10 client.16449 handle_command_reply: tid=0 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 client.16449 shutdown 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 msgr2=0x55c802ef8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802ef8c70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004db0 tx=0x7fc0e4004de0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 shutdown_connections 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802b0ad80 0x55c802f135d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b4c1b0 0x55c802f0d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b2b4d0 0x55c802ef91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2991543790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b232f0 0x55c802ef8c70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0cbe00640 0 client.16449 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.591+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 >> 172.21.15.42:0/2991543790 conn(0x55c802b40300 msgr2=0x55c802ef0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.592+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 shutdown_connections 2024-05-27T10:24:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.592+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2991543790 wait complete. 2024-05-27T10:24:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.592+0000 7fc0f7fed740 20 client.16449 trim_cache size 0 max 16384 2024-05-27T10:24:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.622+0000 7fc0f7fed740 1 Processor -- start 2024-05-27T10:24:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.622+0000 7fc0f7fed740 1 -- start start 2024-05-27T10:24:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.622+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b232f0 0x55c802eef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.622+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 0x55c802f0d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.622+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b4c1b0 0x55c802f0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1aff00 con 0x55c802b2b4d0 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1ad770 con 0x55c802b4c1b0 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1ab5e0 con 0x55c802b232f0 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 0x55c802f0d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 0x55c802f0d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39772/0 (socket says 172.21.15.42:39772) 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0e9400640 1 -- 172.21.15.42:0/2226263771 learned_addr learned my addr 172.21.15.42:0/2226263771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0ea800640 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b4c1b0 0x55c802f0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.623+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b232f0 0x55c802eef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9400640 1 -- 172.21.15.42:0/2226263771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b232f0 msgr2=0x55c802eef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9400640 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b232f0 0x55c802eef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9400640 1 -- 172.21.15.42:0/2226263771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b4c1b0 msgr2=0x55c802f0da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9400640 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b4c1b0 0x55c802f0da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9400640 1 -- 172.21.15.42:0/2226263771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c802efb3a0 con 0x55c802b2b4d0 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b232f0 0x55c802eef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0e9400640 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 0x55c802f0d4c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a9890 tx=0x7fc0dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a51b6354bfe738 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.624+0000 7fc0d6a00640 1 -- 172.21.15.42:0/2226263771 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0c3070 con 0x55c802b2b4d0 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.625+0000 7fc0d6a00640 1 -- 172.21.15.42:0/2226263771 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0dc0a96a0 con 0x55c802b2b4d0 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.625+0000 7fc0d6a00640 1 -- 172.21.15.42:0/2226263771 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0b29f0 con 0x55c802b2b4d0 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2226263771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 msgr2=0x55c802f0d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 0x55c802f0d4c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a9890 tx=0x7fc0dc000bc0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2226263771 shutdown_connections 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b4c1b0 0x55c802f0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b2b4d0 0x55c802f0d4c0 secure :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a9890 tx=0x7fc0dc000bc0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/2226263771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802b232f0 0x55c802eef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2226263771 >> 172.21.15.42:0/2226263771 conn(0x55c802b40300 msgr2=0x55c802ef8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.626+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2226263771 shutdown_connections 2024-05-27T10:24:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.627+0000 7fc0f7fed740 1 -- 172.21.15.42:0/2226263771 wait complete. 2024-05-27T10:24:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.627+0000 7fc0f7fed740 1 -- start start 2024-05-27T10:24:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.627+0000 7fc0f7fed740 1 Processor -- start 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.628+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b232f0 0x55c802f09ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.628+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 0x55c802f04e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.628+0000 7fc0f7fed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802ae01c0 0x55c802f05390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1abc30 con 0x55c802b4c1b0 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1ad0a0 con 0x55c802b232f0 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0f7fed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1ac300 con 0x55c802ae01c0 2024-05-27T10:24:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0f7fed740 10 client.0 _ll_register_callbacks cb 0x7fc0e4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 0x55c802f04e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 0x55c802f04e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39788/0 (socket says 172.21.15.42:39788) 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 -- 172.21.15.42:0/1001530607 learned_addr learned my addr 172.21.15.42:0/1001530607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b232f0 0x55c802f09ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0ea800640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802ae01c0 0x55c802f05390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 -- 172.21.15.42:0/1001530607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802ae01c0 msgr2=0x55c802f05390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802ae01c0 0x55c802f05390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 -- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b232f0 msgr2=0x55c802f09ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b232f0 0x55c802f09ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.629+0000 7fc0e9400640 1 -- 172.21.15.42:0/1001530607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c802f058d0 con 0x55c802b4c1b0 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.630+0000 7fc0e9400640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 0x55c802f04e50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a5c10 tx=0x7fc0dc0ae030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c4e983266c4c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.630+0000 7fc0cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0cbe00640 1 -- 172.21.15.42:0/1001530607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0c9060 con 0x55c802b4c1b0 2024-05-27T10:24:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0cbe00640 1 -- 172.21.15.42:0/1001530607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fc0dc0a6180 con 0x55c802b4c1b0 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0dc0a96a0 con 0x55c802b4c1b0 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0cbe00640 1 -- 172.21.15.42:0/1001530607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc0dc0c3030 con 0x55c802b4c1b0 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0f7fed740 10 client.16455 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c802f7b760 con 0x55c802b4c1b0 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0cbe00640 1 -- 172.21.15.42:0/1001530607 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0dc0a96a0 con 0x55c802b4c1b0 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.631+0000 7fc0cbe00640 1 -- 172.21.15.42:0/1001530607 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc0dc0e0280 con 0x55c802b4c1b0 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0f7fed740 10 client.16455 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0f7fed740 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0f7fed740 20 client.16455 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802f7bda0 0x55c802f7e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0f7fed740 4 client.16455 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:24:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c802e5b2c0 con 0x55c802f7bda0 2024-05-27T10:24:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.632+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802f7bda0 0x55c802f7e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.634+0000 7fc0cbe00640 10 client.16455 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.634+0000 7fc0e9e00640 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802f7bda0 0x55c802f7e1c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf228d2f2e729bb server_cookie=dae56de5ad91e782 in_seq=0 out_seq=0 2024-05-27T10:24:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.655+0000 7fc0cbe00640 1 -- 172.21.15.42:0/1001530607 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c802e5b2c0 con 0x55c802f7bda0 2024-05-27T10:24:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.655+0000 7fc0cbe00640 10 client.16455 handle_command_reply: tid=0 2024-05-27T10:24:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.655+0000 7fc0f7fed740 1 client.16455 shutdown 2024-05-27T10:24:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.655+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 msgr2=0x55c802f04e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.655+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 0x55c802f04e50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a5c10 tx=0x7fc0dc0ae030 comp rx=0 tx=0).stop 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 shutdown_connections 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c802f7bda0 0x55c802f7e1c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c802ae01c0 0x55c802f05390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c802b4c1b0 0x55c802f04e50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 --2- 172.21.15.42:0/1001530607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c802b232f0 0x55c802f09ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0cbe00640 0 client.16455 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 >> 172.21.15.42:0/1001530607 conn(0x55c802b40300 msgr2=0x55c802ef8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 shutdown_connections 2024-05-27T10:24:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 1 -- 172.21.15.42:0/1001530607 wait complete. 2024-05-27T10:24:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.656+0000 7fc0f7fed740 20 client.16455 trim_cache size 0 max 16384 2024-05-27T10:24:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:24:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:24:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:24:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.657+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc0d00b29a0 msgr2=0x7fc0d00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.657+0000 7fc0eb200640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc0d00b29a0 0x7fc0d00b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0d802a980 tx=0x7fc0d8005fd0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.657+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 msgr2=0x7fc0ec19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.657+0000 7fc0eb200640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec19ccd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9f0 tx=0x7fc0dc00eeb0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 shutdown_connections 2024-05-27T10:24:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc0d00b29a0 0x7fc0d00b4eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0ec10c170 0x7fc0ec1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0ec10bcd0 0x7fc0ec19ccd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 --2- 172.21.15.42:0/1050332233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec106420 0x7fc0ec19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 >> 172.21.15.42:0/1050332233 conn(0x7fc0ec1013d0 msgr2=0x7fc0ec108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 shutdown_connections 2024-05-27T10:24:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:39.658+0000 7fc0eb200640 1 -- 172.21.15.42:0/1050332233 wait complete. 2024-05-27T10:24:41.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:40 smithi143 ceph-mon[35545]: pgmap v181: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 121 op/s 2024-05-27T10:24:41.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:40 smithi042 ceph-mon[31661]: pgmap v181: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 121 op/s 2024-05-27T10:24:41.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:40 smithi104 ceph-mon[34812]: pgmap v181: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 121 op/s 2024-05-27T10:24:43.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:42 smithi143 ceph-mon[35545]: pgmap v182: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.6 MiB/s wr, 70 op/s 2024-05-27T10:24:43.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:42 smithi042 ceph-mon[31661]: pgmap v182: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.6 MiB/s wr, 70 op/s 2024-05-27T10:24:43.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:42 smithi104 ceph-mon[34812]: pgmap v182: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.6 MiB/s wr, 70 op/s 2024-05-27T10:24:45.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:44 smithi042 ceph-mon[31661]: pgmap v183: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.6 MiB/s wr, 70 op/s 2024-05-27T10:24:45.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:44 smithi104 ceph-mon[34812]: pgmap v183: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.6 MiB/s wr, 70 op/s 2024-05-27T10:24:45.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:44 smithi143 ceph-mon[35545]: pgmap v183: 97 pgs: 97 active+clean; 156 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.6 MiB/s wr, 70 op/s 2024-05-27T10:24:47.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:46 smithi042 ceph-mon[31661]: pgmap v184: 97 pgs: 97 active+clean; 166 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 115 op/s 2024-05-27T10:24:47.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:46 smithi104 ceph-mon[34812]: pgmap v184: 97 pgs: 97 active+clean; 166 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 115 op/s 2024-05-27T10:24:47.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:46 smithi143 ceph-mon[35545]: pgmap v184: 97 pgs: 97 active+clean; 166 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 115 op/s 2024-05-27T10:24:48.716 INFO:tasks.workunit.client.0.smithi042.stdout: 26 9377 168 6823 136 1761 171 2024-05-27T10:24:49.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:48 smithi042 ceph-mon[31661]: pgmap v185: 97 pgs: 97 active+clean; 166 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-05-27T10:24:49.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:48 smithi104 ceph-mon[34812]: pgmap v185: 97 pgs: 97 active+clean; 166 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-05-27T10:24:49.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:48 smithi143 ceph-mon[35545]: pgmap v185: 97 pgs: 97 active+clean; 166 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 81 op/s 2024-05-27T10:24:51.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:50 smithi042 ceph-mon[31661]: pgmap v186: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 127 op/s 2024-05-27T10:24:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:50 smithi104 ceph-mon[34812]: pgmap v186: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 127 op/s 2024-05-27T10:24:51.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:50 smithi143 ceph-mon[35545]: pgmap v186: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 127 op/s 2024-05-27T10:24:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 -- 172.21.15.42:0/3026300645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 msgr2=0x7f5df4103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3026300645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 0x7f5df4103bb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009980 tx=0x7f5de002e830 comp rx=0 tx=0).stop 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 -- 172.21.15.42:0/3026300645 shutdown_connections 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3026300645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4104220 0x7f5df41105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3026300645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 0x7f5df4103bb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3026300645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4109ad0 0x7f5df4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 -- 172.21.15.42:0/3026300645 >> 172.21.15.42:0/3026300645 conn(0x7f5df4079100 msgr2=0x7f5df4079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 -- 172.21.15.42:0/3026300645 shutdown_connections 2024-05-27T10:24:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.958+0000 7f5df3e00640 1 -- 172.21.15.42:0/3026300645 wait complete. 2024-05-27T10:24:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.959+0000 7f5df3e00640 1 Processor -- start 2024-05-27T10:24:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 -- start start 2024-05-27T10:24:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 0x7f5df41a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4109ad0 0x7f5df41a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 0x7f5df41a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41119e0 con 0x7f5df4109f70 2024-05-27T10:24:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4111860 con 0x7f5df4104220 2024-05-27T10:24:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.960+0000 7f5df3e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4111b60 con 0x7f5df4109ad0 2024-05-27T10:24:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.961+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 0x7f5df41a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.961+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 0x7f5df41a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33464/0 (socket says 172.21.15.42:33464) 2024-05-27T10:24:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.961+0000 7f5df2a00640 1 -- 172.21.15.42:0/3511188640 learned_addr learned my addr 172.21.15.42:0/3511188640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.961+0000 7f5df2000640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4109ad0 0x7f5df41a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.961+0000 7f5df3400640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 0x7f5df41a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.962+0000 7f5df2a00640 1 -- 172.21.15.42:0/3511188640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4109ad0 msgr2=0x7f5df41a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.962+0000 7f5df2a00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4109ad0 0x7f5df41a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.962+0000 7f5df2a00640 1 -- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 msgr2=0x7f5df41a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.962+0000 7f5df2a00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 0x7f5df41a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.962+0000 7f5df2a00640 1 -- 172.21.15.42:0/3511188640 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df41a10a0 con 0x7f5df4104220 2024-05-27T10:24:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.962+0000 7f5df2000640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4109ad0 0x7f5df41a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:24:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.963+0000 7f5df2a00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 0x7f5df41a67f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d770 tx=0x7f5de400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.964+0000 7f5deb400640 1 -- 172.21.15.42:0/3511188640 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5df4104220 2024-05-27T10:24:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.964+0000 7f5deb400640 1 -- 172.21.15.42:0/3511188640 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5de4009c30 con 0x7f5df4104220 2024-05-27T10:24:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.964+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df41a1390 con 0x7f5df4104220 2024-05-27T10:24:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.964+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df4198120 con 0x7f5df4104220 2024-05-27T10:24:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.964+0000 7f5deb400640 1 -- 172.21.15.42:0/3511188640 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400f620 con 0x7f5df4104220 2024-05-27T10:24:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.968+0000 7f5e00838740 1 Processor -- start 2024-05-27T10:24:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.968+0000 7f5e00838740 1 -- start start 2024-05-27T10:24:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5e00838740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa30a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5e00838740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa30fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5e00838740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5e00838740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56117a9a1b90 con 0x56117a68e1b0 2024-05-27T10:24:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5e00838740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56117a991730 con 0x56117a6652f0 2024-05-27T10:24:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5e00838740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56117a995770 con 0x56117a66d4d0 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5df3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa30a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5df3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa30a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38996/0 (socket says 172.21.15.42:38996) 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.969+0000 7f5df3400640 1 -- 172.21.15.42:0/2236617131 learned_addr learned my addr 172.21.15.42:0/2236617131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.970+0000 7f5df2000640 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.970+0000 7f5df3400640 1 -- 172.21.15.42:0/2236617131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 msgr2=0x56117aa314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.970+0000 7f5df3400640 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.970+0000 7f5df3400640 1 -- 172.21.15.42:0/2236617131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 msgr2=0x56117aa30fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.970+0000 7f5df3400640 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa30fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.970+0000 7f5df3400640 1 -- 172.21.15.42:0/2236617131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117a92a8b0 con 0x56117a68e1b0 2024-05-27T10:24:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.971+0000 7f5df3400640 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa30a60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5dec010370 tx=0x7f5dec010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ad4a6fbd31ebef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.971+0000 7f5deb400640 1 -- 172.21.15.42:0/3511188640 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5de401a070 con 0x7f5df4104220 2024-05-27T10:24:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.971+0000 7f5ddfe00640 1 -- 172.21.15.42:0/2236617131 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec016070 con 0x56117a68e1b0 2024-05-27T10:24:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.971+0000 7f5ddfe00640 1 -- 172.21.15.42:0/2236617131 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dec004d20 con 0x56117a68e1b0 2024-05-27T10:24:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.971+0000 7f5ddfe00640 1 -- 172.21.15.42:0/2236617131 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec005390 con 0x56117a68e1b0 2024-05-27T10:24:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 -- 172.21.15.42:0/2236617131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 msgr2=0x56117aa30a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa30a60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5dec010370 tx=0x7f5dec010830 comp rx=0 tx=0).stop 2024-05-27T10:24:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 -- 172.21.15.42:0/2236617131 shutdown_connections 2024-05-27T10:24:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa30fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 --2- 172.21.15.42:0/2236617131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa30a60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.972+0000 7f5e00838740 1 -- 172.21.15.42:0/2236617131 >> 172.21.15.42:0/2236617131 conn(0x56117a682300 msgr2=0x56117aa2d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.974+0000 7f5e00838740 1 -- 172.21.15.42:0/2236617131 shutdown_connections 2024-05-27T10:24:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.974+0000 7f5e00838740 1 -- 172.21.15.42:0/2236617131 wait complete. 2024-05-27T10:24:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.975+0000 7f5e00838740 1 -- start start 2024-05-27T10:24:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.975+0000 7f5e00838740 1 Processor -- start 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.976+0000 7f5deb400640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5dd80b29f0 0x7f5dd80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.976+0000 7f5deb400640 1 -- 172.21.15.42:0/3511188640 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5de40e1d80 con 0x7f5df4104220 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.976+0000 7f5df3400640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5dd80b29f0 0x7f5dd80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.976+0000 7f5e00838740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa3abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.976+0000 7f5e00838740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.976+0000 7f5e00838740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5e00838740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56117a677790 con 0x56117a6652f0 2024-05-27T10:24:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5e00838740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56117a939bc0 con 0x56117a66d4d0 2024-05-27T10:24:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5e00838740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56117aa3db90 con 0x56117a68e1b0 2024-05-27T10:24:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5e00838740 10 client.0 _ll_register_callbacks cb 0x56117a917400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa3abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa3abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39010/0 (socket says 172.21.15.42:39010) 2024-05-27T10:24:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 -- 172.21.15.42:0/3499045164 learned_addr learned my addr 172.21.15.42:0/3499045164 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2000640 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 -- 172.21.15.42:0/3499045164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 msgr2=0x56117aa46f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa46f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 -- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 msgr2=0x56117aa3b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.977+0000 7f5df2a00640 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5df2a00640 1 -- 172.21.15.42:0/3499045164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117a6a6210 con 0x56117a6652f0 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5df3400640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5dd80b29f0 0x7f5dd80b4f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009db0 tx=0x7f5dec009bb0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5df2a00640 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa3abd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009610 tx=0x7f5de40a8da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ee12269e1d499a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5ddcc00640 1 -- 172.21.15.42:0/3499045164 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4014030 con 0x56117a6652f0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5ddcc00640 1 -- 172.21.15.42:0/3499045164 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5de40ad080 con 0x56117a6652f0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.978+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec004d20 con 0x56117a6652f0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5ddcc00640 1 -- 172.21.15.42:0/3499045164 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5de40c8050 con 0x56117a6652f0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5e00838740 10 client.16467 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56117a772620 con 0x56117a6652f0 2024-05-27T10:24:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5ddcc00640 1 -- 172.21.15.42:0/3499045164 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dec004d20 con 0x56117a6652f0 2024-05-27T10:24:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5ddcc00640 1 -- 172.21.15.42:0/3499045164 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5de40a4ad0 con 0x56117a6652f0 2024-05-27T10:24:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5e00838740 10 client.16467 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5e00838740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5e00838740 20 client.16467 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.979+0000 7f5e00838740 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a64cd80 0x56117aa4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.980+0000 7f5e00838740 4 client.16467 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:24:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.980+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56117a9a3ae0 con 0x56117a64cd80 2024-05-27T10:24:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.980+0000 7f5df2000640 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a64cd80 0x56117aa4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.981+0000 7f5df2000640 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a64cd80 0x56117aa4cf00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601fc9318d2627a7 server_cookie=14aeb66161d4f38d in_seq=0 out_seq=0 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.982+0000 7f5ddcc00640 10 client.16467 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5ddcc00640 1 -- 172.21.15.42:0/3499045164 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56117a9a3ae0 con 0x56117a64cd80 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5ddcc00640 10 client.16467 handle_command_reply: tid=0 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 client.16467 shutdown 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 msgr2=0x56117aa3abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa3abd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009610 tx=0x7f5de40a8da0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 shutdown_connections 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a64cd80 0x56117aa4cf00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa46f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 --2- 172.21.15.42:0/3499045164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa3abd0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 >> 172.21.15.42:0/3499045164 conn(0x56117a682300 msgr2=0x56117aa30830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.983+0000 7f5ddcc00640 0 client.16467 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.984+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 shutdown_connections 2024-05-27T10:24:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.984+0000 7f5e00838740 1 -- 172.21.15.42:0/3499045164 wait complete. 2024-05-27T10:24:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:52.984+0000 7f5e00838740 20 client.16467 trim_cache size 0 max 16384 2024-05-27T10:24:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.012+0000 7f5e00838740 1 Processor -- start 2024-05-27T10:24:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.012+0000 7f5e00838740 1 -- start start 2024-05-27T10:24:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5e00838740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa2c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5e00838740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa3d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5e00838740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa46cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5e00838740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4010f20 con 0x56117a68e1b0 2024-05-27T10:24:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5e00838740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd400e790 con 0x56117a6652f0 2024-05-27T10:24:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5e00838740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd400c5e0 con 0x56117a66d4d0 2024-05-27T10:24:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5df3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa46cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5df3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa46cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39022/0 (socket says 172.21.15.42:39022) 2024-05-27T10:24:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5df3400640 1 -- 172.21.15.42:0/2883109948 learned_addr learned my addr 172.21.15.42:0/2883109948 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5df2000640 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.013+0000 7f5df2a00640 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa2c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.014+0000 7f5df2000640 1 -- 172.21.15.42:0/2883109948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 msgr2=0x56117aa2c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.014+0000 7f5df2000640 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa2c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.014+0000 7f5df2000640 1 -- 172.21.15.42:0/2883109948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 msgr2=0x56117aa46cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.014+0000 7f5df2000640 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa46cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.014+0000 7f5df2000640 1 -- 172.21.15.42:0/2883109948 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117aa272a0 con 0x56117a66d4d0 2024-05-27T10:24:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.015+0000 7f5df2000640 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa3d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5de002ee20 tx=0x7f5de0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666f926a342e41f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.015+0000 7f5ddf400640 1 -- 172.21.15.42:0/2883109948 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de003a040 con 0x56117a66d4d0 2024-05-27T10:24:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.015+0000 7f5ddf400640 1 -- 172.21.15.42:0/2883109948 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5de004d070 con 0x56117a66d4d0 2024-05-27T10:24:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.015+0000 7f5ddf400640 1 -- 172.21.15.42:0/2883109948 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00035f0 con 0x56117a66d4d0 2024-05-27T10:24:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.016+0000 7f5e00838740 1 -- 172.21.15.42:0/2883109948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 msgr2=0x56117aa3d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa3d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5de002ee20 tx=0x7f5de0005fd0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 -- 172.21.15.42:0/2883109948 shutdown_connections 2024-05-27T10:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a68e1b0 0x56117aa46cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a66d4d0 0x56117aa3d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 --2- 172.21.15.42:0/2883109948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a6652f0 0x56117aa2c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 -- 172.21.15.42:0/2883109948 >> 172.21.15.42:0/2883109948 conn(0x56117a682300 msgr2=0x56117aa3ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 -- 172.21.15.42:0/2883109948 shutdown_connections 2024-05-27T10:24:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.017+0000 7f5e00838740 1 -- 172.21.15.42:0/2883109948 wait complete. 2024-05-27T10:24:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.018+0000 7f5e00838740 1 -- start start 2024-05-27T10:24:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.018+0000 7f5e00838740 1 Processor -- start 2024-05-27T10:24:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd400cc50 con 0x56117a6652f0 2024-05-27T10:24:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd400e0c0 con 0x56117a66d4d0 2024-05-27T10:24:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd400d320 con 0x56117a68e1b0 2024-05-27T10:24:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.019+0000 7f5e00838740 10 client.0 _ll_register_callbacks cb 0x7f5de400dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:24:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa43400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39032/0 (socket says 172.21.15.42:39032) 2024-05-27T10:24:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 -- 172.21.15.42:0/2442554176 learned_addr learned my addr 172.21.15.42:0/2442554176 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:24:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2000640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 -- 172.21.15.42:0/2442554176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 msgr2=0x56117aa3ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df3400640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa3ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 -- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 msgr2=0x56117aa3e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:24:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.020+0000 7f5df2a00640 1 -- 172.21.15.42:0/2442554176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117aa3f200 con 0x56117a6652f0 2024-05-27T10:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5df2000640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5df3400640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5df2a00640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa43400 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d740 tx=0x7f5de4010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de0dadab2fdb49b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:24:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5ddcc00640 1 -- 172.21.15.42:0/2442554176 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4014070 con 0x56117a6652f0 2024-05-27T10:24:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5ddcc00640 1 -- 172.21.15.42:0/2442554176 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5de40a83c0 con 0x56117a6652f0 2024-05-27T10:24:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.021+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de004d070 con 0x56117a6652f0 2024-05-27T10:24:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.022+0000 7f5ddcc00640 1 -- 172.21.15.42:0/2442554176 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5de4021040 con 0x56117a6652f0 2024-05-27T10:24:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.022+0000 7f5e00838740 10 client.16473 fetch_fsmap learned FSMap version 33 2024-05-27T10:24:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.022+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56117aab52c0 con 0x56117a6652f0 2024-05-27T10:24:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5ddcc00640 1 -- 172.21.15.42:0/2442554176 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5de004d070 con 0x56117a6652f0 2024-05-27T10:24:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5ddcc00640 1 -- 172.21.15.42:0/2442554176 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5de40c39a0 con 0x56117a6652f0 2024-05-27T10:24:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5e00838740 10 client.16473 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:24:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5e00838740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:24:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5e00838740 20 client.16473 populate_metadata read hostname 'smithi042' 2024-05-27T10:24:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5e00838740 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a6221c0 0x56117aab7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:24:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5e00838740 4 client.16473 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:24:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.023+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56117a99d2c0 con 0x56117a6221c0 2024-05-27T10:24:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.024+0000 7f5df2000640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a6221c0 0x56117aab7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:24:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.025+0000 7f5ddcc00640 10 client.16473 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.025+0000 7f5df2000640 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a6221c0 0x56117aab7a20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ea2f99d8115cd9 server_cookie=da51278b22b54141 in_seq=0 out_seq=0 2024-05-27T10:24:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.045+0000 7f5ddcc00640 1 -- 172.21.15.42:0/2442554176 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56117a99d2c0 con 0x56117a6221c0 2024-05-27T10:24:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.045+0000 7f5ddcc00640 10 client.16473 handle_command_reply: tid=0 2024-05-27T10:24:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.045+0000 7f5e00838740 1 client.16473 shutdown 2024-05-27T10:24:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 msgr2=0x56117aa43400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa43400 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d740 tx=0x7f5de4010ee0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 shutdown_connections 2024-05-27T10:24:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56117a6221c0 0x56117aab7a20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56117a68e1b0 0x56117aa3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5ddcc00640 0 client.16473 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56117a66d4d0 0x56117aa3e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 --2- 172.21.15.42:0/2442554176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56117a6652f0 0x56117aa43400 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 >> 172.21.15.42:0/2442554176 conn(0x56117a682300 msgr2=0x56117aa307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 shutdown_connections 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.046+0000 7f5e00838740 1 -- 172.21.15.42:0/2442554176 wait complete. 2024-05-27T10:24:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.047+0000 7f5e00838740 20 client.16473 trim_cache size 0 max 16384 2024-05-27T10:24:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:24:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:24:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:24:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5dd80b29f0 msgr2=0x7f5dd80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5dd80b29f0 0x7f5dd80b4f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009db0 tx=0x7f5dec009bb0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 msgr2=0x7f5df41a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 0x7f5df41a67f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d770 tx=0x7f5de400dc30 comp rx=0 tx=0).stop 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 shutdown_connections 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f5dd80b29f0 0x7f5dd80b4f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5df4109f70 0x7f5df41a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5df4109ad0 0x7f5df41a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 --2- 172.21.15.42:0/3511188640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5df4104220 0x7f5df41a67f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:24:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.048+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 >> 172.21.15.42:0/3511188640 conn(0x7f5df4079100 msgr2=0x7f5df41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:24:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.049+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 shutdown_connections 2024-05-27T10:24:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:24:53.049+0000 7f5df3e00640 1 -- 172.21.15.42:0/3511188640 wait complete. 2024-05-27T10:24:53.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:52 smithi042 ceph-mon[31661]: pgmap v187: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-05-27T10:24:53.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:52 smithi104 ceph-mon[34812]: pgmap v187: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-05-27T10:24:53.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:52 smithi143 ceph-mon[35545]: pgmap v187: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-05-27T10:24:55.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:54 smithi042 ceph-mon[31661]: pgmap v188: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-05-27T10:24:55.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:54 smithi104 ceph-mon[34812]: pgmap v188: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-05-27T10:24:55.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:54 smithi143 ceph-mon[35545]: pgmap v188: 97 pgs: 97 active+clean; 176 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-05-27T10:24:57.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:56 smithi042 ceph-mon[31661]: pgmap v189: 97 pgs: 97 active+clean; 189 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 142 op/s 2024-05-27T10:24:57.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:56 smithi104 ceph-mon[34812]: pgmap v189: 97 pgs: 97 active+clean; 189 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 142 op/s 2024-05-27T10:24:57.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:56 smithi143 ceph-mon[35545]: pgmap v189: 97 pgs: 97 active+clean; 189 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 142 op/s 2024-05-27T10:24:58.716 INFO:tasks.workunit.client.0.smithi042.stdout: 28 8955 213 6911 211 1418 225 2024-05-27T10:24:59.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:24:58 smithi104 ceph-mon[34812]: pgmap v190: 97 pgs: 97 active+clean; 189 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-05-27T10:24:59.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:24:58 smithi143 ceph-mon[35545]: pgmap v190: 97 pgs: 97 active+clean; 189 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-05-27T10:24:59.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:24:58 smithi042 ceph-mon[31661]: pgmap v190: 97 pgs: 97 active+clean; 189 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 96 op/s 2024-05-27T10:25:01.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:00 smithi104 ceph-mon[34812]: pgmap v191: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-05-27T10:25:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:00 smithi143 ceph-mon[35545]: pgmap v191: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-05-27T10:25:01.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:00 smithi042 ceph-mon[31661]: pgmap v191: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-05-27T10:25:03.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:02 smithi143 ceph-mon[35545]: pgmap v192: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-05-27T10:25:03.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:02 smithi104 ceph-mon[34812]: pgmap v192: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-05-27T10:25:03.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:02 smithi042 ceph-mon[31661]: pgmap v192: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-05-27T10:25:05.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:04 smithi104 ceph-mon[34812]: pgmap v193: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-05-27T10:25:05.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:04 smithi143 ceph-mon[35545]: pgmap v193: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-05-27T10:25:05.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:04 smithi042 ceph-mon[31661]: pgmap v193: 97 pgs: 97 active+clean; 201 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-05-27T10:25:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.342+0000 7fe4bda00640 1 -- 172.21.15.42:0/500186736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 msgr2=0x7fe4b81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.342+0000 7fe4bda00640 1 --2- 172.21.15.42:0/500186736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81079e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009900 tx=0x7fe4a402a410 comp rx=0 tx=0).stop 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 -- 172.21.15.42:0/500186736 shutdown_connections 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 --2- 172.21.15.42:0/500186736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8108000 0x7fe4b8108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 --2- 172.21.15.42:0/500186736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81079e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 --2- 172.21.15.42:0/500186736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8102b80 0x7fe4b8102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 -- 172.21.15.42:0/500186736 >> 172.21.15.42:0/500186736 conn(0x7fe4b80782d0 msgr2=0x7fe4b8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 -- 172.21.15.42:0/500186736 shutdown_connections 2024-05-27T10:25:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.343+0000 7fe4bda00640 1 -- 172.21.15.42:0/500186736 wait complete. 2024-05-27T10:25:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.344+0000 7fe4bda00640 1 Processor -- start 2024-05-27T10:25:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.344+0000 7fe4bda00640 1 -- start start 2024-05-27T10:25:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8102b80 0x7fe4b81a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4bda00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8108000 0x7fe4b81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4bda00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811a2e0 con 0x7fe4b81075d0 2024-05-27T10:25:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4bda00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811a160 con 0x7fe4b8108000 2024-05-27T10:25:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4bda00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811a460 con 0x7fe4b8102b80 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41818/0 (socket says 172.21.15.42:41818) 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4b6000640 1 -- 172.21.15.42:0/3142617985 learned_addr learned my addr 172.21.15.42:0/3142617985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.345+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8102b80 0x7fe4b81a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b7400640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8108000 0x7fe4b81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b6000640 1 -- 172.21.15.42:0/3142617985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8102b80 msgr2=0x7fe4b81a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b6000640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8102b80 0x7fe4b81a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b6000640 1 -- 172.21.15.42:0/3142617985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8108000 msgr2=0x7fe4b81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b6000640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8108000 0x7fe4b81a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b6000640 1 -- 172.21.15.42:0/3142617985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81a1080 con 0x7fe4b81075d0 2024-05-27T10:25:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.346+0000 7fe4b6000640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81a6d10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40057c0 tx=0x7fe4a4030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.347+0000 7fe4af400640 1 -- 172.21.15.42:0/3142617985 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a403f070 con 0x7fe4b81075d0 2024-05-27T10:25:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.347+0000 7fe4af400640 1 -- 172.21.15.42:0/3142617985 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a4004460 con 0x7fe4b81075d0 2024-05-27T10:25:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.347+0000 7fe4af400640 1 -- 172.21.15.42:0/3142617985 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4035030 con 0x7fe4b81075d0 2024-05-27T10:25:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.347+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81a1310 con 0x7fe4b81075d0 2024-05-27T10:25:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.347+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81a17b0 con 0x7fe4b81075d0 2024-05-27T10:25:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.352+0000 7fe4c4998740 1 Processor -- start 2024-05-27T10:25:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.352+0000 7fe4c4998740 1 -- start start 2024-05-27T10:25:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.352+0000 7fe4af400640 1 -- 172.21.15.42:0/3142617985 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe4a4042070 con 0x7fe4b81075d0 2024-05-27T10:25:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf911b0 0x5616bb333860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb333da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb3342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4c4998740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5616bb2a4b90 con 0x5616baf682f0 2024-05-27T10:25:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4c4998740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616bb294730 con 0x5616baf704d0 2024-05-27T10:25:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4c4998740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616bb298770 con 0x5616baf911b0 2024-05-27T10:25:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb333da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb333da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41826/0 (socket says 172.21.15.42:41826) 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4b6a00640 1 -- 172.21.15.42:0/4081855445 learned_addr learned my addr 172.21.15.42:0/4081855445 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4b6000640 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb3342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.353+0000 7fe4b7400640 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf911b0 0x5616bb333860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.354+0000 7fe4b6a00640 1 -- 172.21.15.42:0/4081855445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf911b0 msgr2=0x5616bb333860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.354+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf911b0 0x5616bb333860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.354+0000 7fe4b6a00640 1 -- 172.21.15.42:0/4081855445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 msgr2=0x5616bb3342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.354+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb3342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.354+0000 7fe4b6a00640 1 -- 172.21.15.42:0/4081855445 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616bb22d8b0 con 0x5616baf682f0 2024-05-27T10:25:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.354+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb333da0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009ca0 tx=0x7fe4a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d88fff54496146b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.355+0000 7fe4a3e00640 1 -- 172.21.15.42:0/4081855445 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8005220 con 0x5616baf682f0 2024-05-27T10:25:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.355+0000 7fe4a3e00640 1 -- 172.21.15.42:0/4081855445 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a8004c30 con 0x5616baf682f0 2024-05-27T10:25:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.355+0000 7fe4a3e00640 1 -- 172.21.15.42:0/4081855445 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8019690 con 0x5616baf682f0 2024-05-27T10:25:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.355+0000 7fe4af400640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.355+0000 7fe4b7400640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.355+0000 7fe4af400640 1 -- 172.21.15.42:0/3142617985 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe4a41030f0 con 0x7fe4b81075d0 2024-05-27T10:25:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.356+0000 7fe4c4998740 1 -- 172.21.15.42:0/4081855445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 msgr2=0x5616bb333da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.356+0000 7fe4c4998740 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb333da0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009ca0 tx=0x7fe4a8011810 comp rx=0 tx=0).stop 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 -- 172.21.15.42:0/4081855445 shutdown_connections 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb3342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb333da0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 --2- 172.21.15.42:0/4081855445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf911b0 0x5616bb333860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 -- 172.21.15.42:0/4081855445 >> 172.21.15.42:0/4081855445 conn(0x5616baf85300 msgr2=0x5616bb330010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4b7400640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5616bb3309d0 tx=0x7fe4b0009500 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 -- 172.21.15.42:0/4081855445 shutdown_connections 2024-05-27T10:25:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.357+0000 7fe4c4998740 1 -- 172.21.15.42:0/4081855445 wait complete. 2024-05-27T10:25:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.358+0000 7fe4c4998740 1 -- start start 2024-05-27T10:25:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.358+0000 7fe4c4998740 1 Processor -- start 2024-05-27T10:25:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb33dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb33e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 0x5616bb349cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5616baf7a790 con 0x5616baf682f0 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 0x5616bb349cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 0x5616bb349cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:57812/0 (socket says 172.21.15.42:57812) 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4b7400640 1 -- 172.21.15.42:0/871791956 learned_addr learned my addr 172.21.15.42:0/871791956 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616bb23cbc0 con 0x5616baf911b0 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616bb340990 con 0x5616baf704d0 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4c4998740 10 client.0 _ll_register_callbacks cb 0x5616bb21a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb33dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.359+0000 7fe4b6000640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb33e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b7400640 1 -- 172.21.15.42:0/871791956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 msgr2=0x5616bb33e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b7400640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb33e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b7400640 1 -- 172.21.15.42:0/871791956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 msgr2=0x5616bb33dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b7400640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb33dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b7400640 1 -- 172.21.15.42:0/871791956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616bafa9210 con 0x5616baf911b0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb33dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4b7400640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 0x5616bb349cf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00049b0 tx=0x7fe4b0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b94f892f443d3e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.360+0000 7fe4a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.361+0000 7fe4a0c00640 1 -- 172.21.15.42:0/871791956 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0010be0 con 0x5616baf911b0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.361+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a8004c30 con 0x5616baf911b0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.361+0000 7fe4a0c00640 1 -- 172.21.15.42:0/871791956 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe4b0010d80 con 0x5616baf911b0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.362+0000 7fe4a0c00640 1 -- 172.21.15.42:0/871791956 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4b0004b20 con 0x5616baf911b0 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.362+0000 7fe4c4998740 10 client.36011 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.362+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5616bb075620 con 0x5616baf911b0 2024-05-27T10:25:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.362+0000 7fe4a0c00640 1 -- 172.21.15.42:0/871791956 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a8004c30 con 0x5616baf911b0 2024-05-27T10:25:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.362+0000 7fe4a0c00640 1 -- 172.21.15.42:0/871791956 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe4b0024070 con 0x5616baf911b0 2024-05-27T10:25:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4c4998740 10 client.36011 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4c4998740 10 client.36011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4c4998740 20 client.36011 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4c4998740 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf4fd80 0x5616bb34fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4c4998740 4 client.36011 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:25:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616bb2a6ae0 con 0x5616baf4fd80 2024-05-27T10:25:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.363+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf4fd80 0x5616bb34fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.364+0000 7fe4a0c00640 10 client.36011 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.364+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf4fd80 0x5616bb34fda0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c991fddbf996dc server_cookie=b2a143faa8699a54 in_seq=0 out_seq=0 2024-05-27T10:25:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.365+0000 7fe4a0c00640 1 -- 172.21.15.42:0/871791956 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5616bb2a6ae0 con 0x5616baf4fd80 2024-05-27T10:25:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.365+0000 7fe4a0c00640 10 client.36011 handle_command_reply: tid=0 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.365+0000 7fe4c4998740 1 client.36011 shutdown 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.365+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 msgr2=0x5616bb349cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.365+0000 7fe4c4998740 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 0x5616bb349cf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00049b0 tx=0x7fe4b0004a40 comp rx=0 tx=0).stop 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 shutdown_connections 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4c4998740 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf911b0 0x5616bb349cf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4c4998740 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf4fd80 0x5616bb34fda0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4c4998740 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb33e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4c4998740 1 --2- 172.21.15.42:0/871791956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb33dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 >> 172.21.15.42:0/871791956 conn(0x5616baf85300 msgr2=0x5616bb335580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.366+0000 7fe4a0c00640 0 client.36011 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.367+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 shutdown_connections 2024-05-27T10:25:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.367+0000 7fe4c4998740 1 -- 172.21.15.42:0/871791956 wait complete. 2024-05-27T10:25:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.368+0000 7fe4c4998740 20 client.36011 trim_cache size 0 max 16384 2024-05-27T10:25:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.396+0000 7fe4c4998740 1 Processor -- start 2024-05-27T10:25:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.396+0000 7fe4c4998740 1 -- start start 2024-05-27T10:25:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf85300 0x5616bb3392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb3341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb33f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4c4998740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b8600 con 0x5616baf682f0 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4c4998740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b5e70 con 0x5616baf704d0 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4c4998740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b3cc0 con 0x5616baf85300 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb3341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb3341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41838/0 (socket says 172.21.15.42:41838) 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4b6000640 1 -- 172.21.15.42:0/2768552215 learned_addr learned my addr 172.21.15.42:0/2768552215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.397+0000 7fe4b7400640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb33f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf85300 0x5616bb3392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6000640 1 -- 172.21.15.42:0/2768552215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf85300 msgr2=0x5616bb3392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6000640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf85300 0x5616bb3392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6000640 1 -- 172.21.15.42:0/2768552215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 msgr2=0x5616bb33f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6000640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb33f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6000640 1 -- 172.21.15.42:0/2768552215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616bb334c40 con 0x5616baf682f0 2024-05-27T10:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b7400640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb33f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:25:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf85300 0x5616bb3392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.398+0000 7fe4b6000640 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb3341c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4100930 tx=0x7fe4a4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a124fe64855d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.399+0000 7fe4a3400640 1 -- 172.21.15.42:0/2768552215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40c9db0 con 0x5616baf682f0 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.399+0000 7fe4a3400640 1 -- 172.21.15.42:0/2768552215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a4030df0 con 0x5616baf682f0 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.399+0000 7fe4a3400640 1 -- 172.21.15.42:0/2768552215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40c9bd0 con 0x5616baf682f0 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 -- 172.21.15.42:0/2768552215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 msgr2=0x5616bb3341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb3341c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4100930 tx=0x7fe4a4004610 comp rx=0 tx=0).stop 2024-05-27T10:25:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 -- 172.21.15.42:0/2768552215 shutdown_connections 2024-05-27T10:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf704d0 0x5616bb33f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb3341c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 --2- 172.21.15.42:0/2768552215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf85300 0x5616bb3392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 -- 172.21.15.42:0/2768552215 >> 172.21.15.42:0/2768552215 conn(0x5616bb335e20 msgr2=0x5616bb33f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 -- 172.21.15.42:0/2768552215 shutdown_connections 2024-05-27T10:25:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.400+0000 7fe4c4998740 1 -- 172.21.15.42:0/2768552215 wait complete. 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.401+0000 7fe4c4998740 1 -- start start 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.401+0000 7fe4c4998740 1 Processor -- start 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb346130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb341480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 0x5616bb3419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b4330 con 0x5616baf682f0 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b57a0 con 0x5616baf85300 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b4a00 con 0x5616baf704d0 2024-05-27T10:25:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4c4998740 10 client.0 _ll_register_callbacks cb 0x7fe4b0003f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 0x5616bb3419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 0x5616bb3419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:57818/0 (socket says 172.21.15.42:57818) 2024-05-27T10:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.402+0000 7fe4b7400640 1 -- 172.21.15.42:0/1922461666 learned_addr learned my addr 172.21.15.42:0/1922461666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b6000640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb341480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb346130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b7400640 1 -- 172.21.15.42:0/1922461666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 msgr2=0x5616bb341480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b7400640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb341480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b7400640 1 -- 172.21.15.42:0/1922461666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 msgr2=0x5616bb346130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b7400640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb346130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b7400640 1 -- 172.21.15.42:0/1922461666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616bb341f00 con 0x5616baf85300 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4b7400640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 0x5616bb3419c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00049b0 tx=0x7fe4b0010a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389402d1983ac264 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.403+0000 7fe4a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.404+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1922461666 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0012450 con 0x5616baf85300 2024-05-27T10:25:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.404+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5616bb2a02c0 con 0x5616baf85300 2024-05-27T10:25:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.404+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1922461666 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe4b002f920 con 0x5616baf85300 2024-05-27T10:25:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.404+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1922461666 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4b002fb40 con 0x5616baf85300 2024-05-27T10:25:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.404+0000 7fe4c4998740 10 client.36017 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.404+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5616bb3b8080 con 0x5616baf85300 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1922461666 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5616bb2a02c0 con 0x5616baf85300 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1922461666 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe4b0024040 con 0x5616baf85300 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4c4998740 10 client.36017 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4c4998740 10 client.36017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4c4998740 20 client.36017 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4c4998740 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf911b0 0x5616bb3ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4c4998740 4 client.36017 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:25:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.405+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616bb21a4b0 con 0x5616baf911b0 2024-05-27T10:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.406+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf911b0 0x5616bb3ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.407+0000 7fe4b6a00640 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf911b0 0x5616bb3ba7f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe41f37abeac297f server_cookie=164867807de50527 in_seq=0 out_seq=0 2024-05-27T10:25:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.407+0000 7fe4a0c00640 10 client.36017 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1922461666 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5616bb21a4b0 con 0x5616baf911b0 2024-05-27T10:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4a0c00640 10 client.36017 handle_command_reply: tid=0 2024-05-27T10:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 client.36017 shutdown 2024-05-27T10:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 msgr2=0x5616bb3419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 0x5616bb3419c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00049b0 tx=0x7fe4b0010a60 comp rx=0 tx=0).stop 2024-05-27T10:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 shutdown_connections 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616baf85300 0x5616bb3419c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5616baf911b0 0x5616bb3ba7f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616baf704d0 0x5616bb341480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 --2- 172.21.15.42:0/1922461666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616baf682f0 0x5616bb346130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4a0c00640 0 client.36017 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.426+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 >> 172.21.15.42:0/1922461666 conn(0x5616bb335e20 msgr2=0x5616bb333f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.427+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 shutdown_connections 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.427+0000 7fe4c4998740 1 -- 172.21.15.42:0/1922461666 wait complete. 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.427+0000 7fe4c4998740 20 client.36017 trim_cache size 0 max 16384 2024-05-27T10:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 msgr2=0x7fe49c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5616bb3309d0 tx=0x7fe4b0009500 comp rx=0 tx=0).stop 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 msgr2=0x7fe4b81a6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81a6d10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40057c0 tx=0x7fe4a4030650 comp rx=0 tx=0).stop 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 shutdown_connections 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe49c0b2aa0 0x7fe49c0b4fb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4b8108000 0x7fe4b81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4b81075d0 0x7fe4b81a6d10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 --2- 172.21.15.42:0/3142617985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4b8102b80 0x7fe4b81a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.428+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 >> 172.21.15.42:0/3142617985 conn(0x7fe4b80782d0 msgr2=0x7fe4b8105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.429+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 shutdown_connections 2024-05-27T10:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:06.429+0000 7fe4bda00640 1 -- 172.21.15.42:0/3142617985 wait complete. 2024-05-27T10:25:07.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:06 smithi104 ceph-mon[34812]: pgmap v194: 97 pgs: 97 active+clean; 213 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-05-27T10:25:07.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:06 smithi143 ceph-mon[35545]: pgmap v194: 97 pgs: 97 active+clean; 213 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-05-27T10:25:07.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:06 smithi042 ceph-mon[31661]: pgmap v194: 97 pgs: 97 active+clean; 213 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-05-27T10:25:08.716 INFO:tasks.workunit.client.0.smithi042.stdout: 32 8154 190 5977 180 1697 215 2024-05-27T10:25:09.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:08 smithi104 ceph-mon[34812]: pgmap v195: 97 pgs: 97 active+clean; 213 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-05-27T10:25:09.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:08 smithi143 ceph-mon[35545]: pgmap v195: 97 pgs: 97 active+clean; 213 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-05-27T10:25:09.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:08 smithi042 ceph-mon[31661]: pgmap v195: 97 pgs: 97 active+clean; 213 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-05-27T10:25:11.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:10 smithi104 ceph-mon[34812]: pgmap v196: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 158 op/s 2024-05-27T10:25:11.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:10 smithi143 ceph-mon[35545]: pgmap v196: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 158 op/s 2024-05-27T10:25:11.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:10 smithi042 ceph-mon[31661]: pgmap v196: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 158 op/s 2024-05-27T10:25:13.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:12 smithi104 ceph-mon[34812]: pgmap v197: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-05-27T10:25:13.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:12 smithi143 ceph-mon[35545]: pgmap v197: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-05-27T10:25:13.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:12 smithi042 ceph-mon[31661]: pgmap v197: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-05-27T10:25:15.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:14 smithi104 ceph-mon[34812]: pgmap v198: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-05-27T10:25:15.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:14 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:25:15.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:14 smithi143 ceph-mon[35545]: pgmap v198: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-05-27T10:25:15.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:14 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:25:15.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:14 smithi042 ceph-mon[31661]: pgmap v198: 97 pgs: 97 active+clean; 225 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 104 op/s 2024-05-27T10:25:15.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:14 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:25:16.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:16 smithi143 ceph-mon[35545]: pgmap v199: 97 pgs: 97 active+clean; 238 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-05-27T10:25:16.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:16 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:16 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:16 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:16 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:16 smithi042 ceph-mon[31661]: pgmap v199: 97 pgs: 97 active+clean; 238 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-05-27T10:25:16.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:16 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:16 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:16 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:16 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:16 smithi104 ceph-mon[34812]: pgmap v199: 97 pgs: 97 active+clean; 238 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-05-27T10:25:16.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:16 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:16 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:16 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:16.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:16 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:17.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:17 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:25:17.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:17 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:25:17.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:17 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:17.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:17 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:25:17.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:17 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:25:17.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:17 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:17.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:17 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:25:17.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:17 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:25:17.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:17 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:25:18.716 INFO:tasks.workunit.client.0.smithi042.stdout: 35 8645 170 6540 190 2028 268 2024-05-27T10:25:18.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:18 smithi042 ceph-mon[31661]: pgmap v200: 97 pgs: 97 active+clean; 238 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-05-27T10:25:18.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:18 smithi104 ceph-mon[34812]: pgmap v200: 97 pgs: 97 active+clean; 238 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-05-27T10:25:19.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:18 smithi143 ceph-mon[35545]: pgmap v200: 97 pgs: 97 active+clean; 238 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.722+0000 7fccb4400640 1 -- 172.21.15.42:0/1003164048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac1026b0 msgr2=0x7fccac1104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.722+0000 7fccb4400640 1 --2- 172.21.15.42:0/1003164048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac1026b0 0x7fccac1104e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc9802a400 comp rx=0 tx=0).stop 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 -- 172.21.15.42:0/1003164048 shutdown_connections 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 --2- 172.21.15.42:0/1003164048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 0x7fccac112e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 --2- 172.21.15.42:0/1003164048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac1026b0 0x7fccac1104e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 --2- 172.21.15.42:0/1003164048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac101d60 0x7fccac102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 -- 172.21.15.42:0/1003164048 >> 172.21.15.42:0/1003164048 conn(0x7fccac0779d0 msgr2=0x7fccac077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 -- 172.21.15.42:0/1003164048 shutdown_connections 2024-05-27T10:25:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.723+0000 7fccb4400640 1 -- 172.21.15.42:0/1003164048 wait complete. 2024-05-27T10:25:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.724+0000 7fccb4400640 1 Processor -- start 2024-05-27T10:25:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.724+0000 7fccb4400640 1 -- start start 2024-05-27T10:25:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb4400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 0x7fccac10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb4400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac1026b0 0x7fccac10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb4400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 0x7fccac10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb4400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccac116150 con 0x7fccac110a20 2024-05-27T10:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb4400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccac115fd0 con 0x7fccac101d60 2024-05-27T10:25:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb4400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1162d0 con 0x7fccac1026b0 2024-05-27T10:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.725+0000 7fccb1a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 0x7fccac10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb1a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 0x7fccac10da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:41450/0 (socket says 172.21.15.42:41450) 2024-05-27T10:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb1a00640 1 -- 172.21.15.42:0/3037135719 learned_addr learned my addr 172.21.15.42:0/3037135719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb2400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 0x7fccac10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb1000640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac1026b0 0x7fccac10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb2400640 1 -- 172.21.15.42:0/3037135719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac1026b0 msgr2=0x7fccac10df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb2400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac1026b0 0x7fccac10df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb2400640 1 -- 172.21.15.42:0/3037135719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 msgr2=0x7fccac10da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb2400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 0x7fccac10da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb2400640 1 -- 172.21.15.42:0/3037135719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac1ad0c0 con 0x7fccac110a20 2024-05-27T10:25:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.726+0000 7fccb1a00640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 0x7fccac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:25:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.727+0000 7fccb2400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 0x7fccac10e4e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcca400d680 tx=0x7fcca400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.727+0000 7fccaa000640 1 -- 172.21.15.42:0/3037135719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017070 con 0x7fccac110a20 2024-05-27T10:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.727+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac1ad3b0 con 0x7fccac110a20 2024-05-27T10:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.728+0000 7fccaa000640 1 -- 172.21.15.42:0/3037135719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcca4009820 con 0x7fccac110a20 2024-05-27T10:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.728+0000 7fccaa000640 1 -- 172.21.15.42:0/3037135719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4004c60 con 0x7fccac110a20 2024-05-27T10:25:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.728+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac1ad8d0 con 0x7fccac110a20 2024-05-27T10:25:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.733+0000 7fccbb365740 1 Processor -- start 2024-05-27T10:25:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.735+0000 7fccaa000640 1 -- 172.21.15.42:0/3037135719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcca40099d0 con 0x7fccac110a20 2024-05-27T10:25:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.735+0000 7fccbb365740 1 -- start start 2024-05-27T10:25:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccbb365740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce27fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccbb365740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 0x55fce27feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccbb365740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce27ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccbb365740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fce276fb90 con 0x55fce243b4d0 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccbb365740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fce275f730 con 0x55fce24332f0 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccbb365740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fce2763770 con 0x55fce245c1b0 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccb1a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 0x55fce27feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccb1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce27ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccb1a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 0x55fce27feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:41462/0 (socket says 172.21.15.42:41462) 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccb1a00640 1 -- 172.21.15.42:0/1275239017 learned_addr learned my addr 172.21.15.42:0/1275239017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.736+0000 7fccb1000640 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce27fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.737+0000 7fccb1a00640 1 -- 172.21.15.42:0/1275239017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 msgr2=0x55fce27fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.737+0000 7fccb1a00640 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce27fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.737+0000 7fccb1a00640 1 -- 172.21.15.42:0/1275239017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 msgr2=0x55fce27ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.737+0000 7fccb1a00640 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce27ff2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.737+0000 7fccb1a00640 1 -- 172.21.15.42:0/1275239017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce26f88b0 con 0x55fce24332f0 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.738+0000 7fccb1a00640 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 0x55fce27feda0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcca000f7f0 tx=0x7fcca000fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bfb49542d2bdb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.738+0000 7fccaa000640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc940b29d0 0x7fcc940b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.738+0000 7fcc9ea00640 1 -- 172.21.15.42:0/1275239017 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004d20 con 0x55fce24332f0 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.738+0000 7fccaa000640 1 -- 172.21.15.42:0/3037135719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcca40e19b0 con 0x7fccac110a20 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.738+0000 7fcc9ea00640 1 -- 172.21.15.42:0/1275239017 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcca0005020 con 0x55fce24332f0 2024-05-27T10:25:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.738+0000 7fcc9ea00640 1 -- 172.21.15.42:0/1275239017 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0010690 con 0x55fce24332f0 2024-05-27T10:25:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.739+0000 7fccb1000640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc940b29d0 0x7fcc940b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 -- 172.21.15.42:0/1275239017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 msgr2=0x55fce27feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 0x55fce27feda0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcca000f7f0 tx=0x7fcca000fcb0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 -- 172.21.15.42:0/1275239017 shutdown_connections 2024-05-27T10:25:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce27ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce24332f0 0x55fce27feda0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 --2- 172.21.15.42:0/1275239017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce27fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 -- 172.21.15.42:0/1275239017 >> 172.21.15.42:0/1275239017 conn(0x55fce2450300 msgr2=0x55fce27fb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 -- 172.21.15.42:0/1275239017 shutdown_connections 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccbb365740 1 -- 172.21.15.42:0/1275239017 wait complete. 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.740+0000 7fccb1000640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc940b29d0 0x7fcc940b4ee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc980057c0 tx=0x7fcc98041040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.741+0000 7fccbb365740 1 -- start start 2024-05-27T10:25:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.741+0000 7fccbb365740 1 Processor -- start 2024-05-27T10:25:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce2808f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 0x55fce2814cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fce2445790 con 0x55fce243b4d0 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fce2707bc0 con 0x55fce245c1b0 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fce280b990 con 0x55fce24332f0 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.742+0000 7fccbb365740 10 client.0 _ll_register_callbacks cb 0x55fce26e5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.743+0000 7fccb2400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 0x55fce2814cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.743+0000 7fccb2400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 0x55fce2814cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:41478/0 (socket says 172.21.15.42:41478) 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.743+0000 7fccb2400640 1 -- 172.21.15.42:0/1058909728 learned_addr learned my addr 172.21.15.42:0/1058909728 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.743+0000 7fccb1a00640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.743+0000 7fccb1000640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce2808f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb2400640 1 -- 172.21.15.42:0/1058909728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 msgr2=0x55fce2808a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb2400640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb2400640 1 -- 172.21.15.42:0/1058909728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 msgr2=0x55fce2808f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb2400640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce2808f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb2400640 1 -- 172.21.15.42:0/1058909728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce2474210 con 0x55fce245c1b0 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb1000640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce2808f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fccb2400640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 0x55fce2814cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcca40a4e10 tx=0x7fcca40a8ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13333af943fd0df8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.744+0000 7fcc93400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.745+0000 7fcc93400640 1 -- 172.21.15.42:0/1058909728 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4020040 con 0x55fce245c1b0 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.745+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca0005020 con 0x55fce245c1b0 2024-05-27T10:25:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.745+0000 7fcc93400640 1 -- 172.21.15.42:0/1058909728 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcca40a5020 con 0x55fce245c1b0 2024-05-27T10:25:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.745+0000 7fcc93400640 1 -- 172.21.15.42:0/1058909728 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcca40c3370 con 0x55fce245c1b0 2024-05-27T10:25:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.745+0000 7fccbb365740 10 client.36029 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.745+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fce2540620 con 0x55fce245c1b0 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.746+0000 7fcc93400640 1 -- 172.21.15.42:0/1058909728 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcca0005020 con 0x55fce245c1b0 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fcc93400640 1 -- 172.21.15.42:0/1058909728 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcca4023060 con 0x55fce245c1b0 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fccbb365740 10 client.36029 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fccbb365740 10 client.36029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fccbb365740 20 client.36029 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fccbb365740 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce241ad80 0x55fce281aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fccbb365740 4 client.36029 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.747+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce2771ae0 con 0x55fce241ad80 2024-05-27T10:25:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.748+0000 7fccb1a00640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce241ad80 0x55fce281aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.748+0000 7fccb1a00640 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce241ad80 0x55fce281aca0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52fd8f2bbd904272 server_cookie=83123f5fb9b68218 in_seq=0 out_seq=0 2024-05-27T10:25:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.748+0000 7fcc93400640 10 client.36029 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.752+0000 7fcc93400640 1 -- 172.21.15.42:0/1058909728 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fce2771ae0 con 0x55fce241ad80 2024-05-27T10:25:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.752+0000 7fcc93400640 10 client.36029 handle_command_reply: tid=0 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.752+0000 7fccbb365740 1 client.36029 shutdown 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.752+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 msgr2=0x55fce2814cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.752+0000 7fccbb365740 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 0x55fce2814cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcca40a4e10 tx=0x7fcca40a8ed0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 shutdown_connections 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce245c1b0 0x55fce2814cc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce243b4d0 0x55fce2808f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce241ad80 0x55fce281aca0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 --2- 172.21.15.42:0/1058909728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 >> 172.21.15.42:0/1058909728 conn(0x55fce2450300 msgr2=0x55fce27fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fcc93400640 0 client.36029 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 shutdown_connections 2024-05-27T10:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 1 -- 172.21.15.42:0/1058909728 wait complete. 2024-05-27T10:25:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.753+0000 7fccbb365740 20 client.36029 trim_cache size 0 max 16384 2024-05-27T10:25:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.783+0000 7fccbb365740 1 Processor -- start 2024-05-27T10:25:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.783+0000 7fccbb365740 1 -- start start 2024-05-27T10:25:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.783+0000 7fccbb365740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.783+0000 7fccbb365740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce2820480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccbb365740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce245c1b0 0x55fce28209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccbb365740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b8870 con 0x55fce245c1b0 2024-05-27T10:25:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccbb365740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b60e0 con 0x55fce243b4d0 2024-05-27T10:25:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccbb365740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b3f30 con 0x55fce24332f0 2024-05-27T10:25:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccb1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce2820480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccb1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce2820480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:41490/0 (socket says 172.21.15.42:41490) 2024-05-27T10:25:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccb1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccb2400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce245c1b0 0x55fce28209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.784+0000 7fccb1000640 1 -- 172.21.15.42:0/1481085880 learned_addr learned my addr 172.21.15.42:0/1481085880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb1000640 1 -- 172.21.15.42:0/1481085880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 msgr2=0x55fce2808f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb1000640 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb1000640 1 -- 172.21.15.42:0/1481085880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce245c1b0 msgr2=0x55fce28209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb1000640 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce245c1b0 0x55fce28209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb1000640 1 -- 172.21.15.42:0/1481085880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce27ffc40 con 0x55fce243b4d0 2024-05-27T10:25:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb2400640 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce245c1b0 0x55fce28209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:25:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.785+0000 7fccb1000640 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce2820480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc98000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e802a3066da46 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.786+0000 7fcc9e000640 1 -- 172.21.15.42:0/1481085880 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98046070 con 0x55fce243b4d0 2024-05-27T10:25:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.786+0000 7fcc9e000640 1 -- 172.21.15.42:0/1481085880 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc98052070 con 0x55fce243b4d0 2024-05-27T10:25:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.786+0000 7fcc9e000640 1 -- 172.21.15.42:0/1481085880 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9803c450 con 0x55fce243b4d0 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.787+0000 7fccbb365740 1 -- 172.21.15.42:0/1481085880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 msgr2=0x55fce2820480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.787+0000 7fccbb365740 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce2820480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc98000bc0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.787+0000 7fccbb365740 1 -- 172.21.15.42:0/1481085880 shutdown_connections 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.787+0000 7fccbb365740 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce245c1b0 0x55fce28209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.788+0000 7fccbb365740 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce2820480 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.788+0000 7fccbb365740 1 --2- 172.21.15.42:0/1481085880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce24332f0 0x55fce2808f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.788+0000 7fccbb365740 1 -- 172.21.15.42:0/1481085880 >> 172.21.15.42:0/1481085880 conn(0x55fce2450300 msgr2=0x55fce2803270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.788+0000 7fccbb365740 1 -- 172.21.15.42:0/1481085880 shutdown_connections 2024-05-27T10:25:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.788+0000 7fccbb365740 1 -- 172.21.15.42:0/1481085880 wait complete. 2024-05-27T10:25:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.788+0000 7fccbb365740 1 -- start start 2024-05-27T10:25:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 Processor -- start 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce24332f0 0x55fce2811320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce280e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce280ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b45a0 con 0x55fce24332f0 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b5a10 con 0x55fce243b4d0 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b4c70 con 0x55fce245c1b0 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.789+0000 7fccbb365740 10 client.0 _ll_register_callbacks cb 0x7fcca40a4cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce280e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce280e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:41506/0 (socket says 172.21.15.42:41506) 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 -- 172.21.15.42:0/4294669018 learned_addr learned my addr 172.21.15.42:0/4294669018 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1a00640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce24332f0 0x55fce2811320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb2400640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce280ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 -- 172.21.15.42:0/4294669018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 msgr2=0x55fce280ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce280ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 -- 172.21.15.42:0/4294669018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce24332f0 msgr2=0x55fce2811320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce24332f0 0x55fce2811320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1000640 1 -- 172.21.15.42:0/4294669018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce27f6080 con 0x55fce243b4d0 2024-05-27T10:25:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.790+0000 7fccb1a00640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce24332f0 0x55fce2811320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:25:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.791+0000 7fccb1000640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce280e690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc9802a440 tx=0x7fcc980038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0ebfc9fe698473 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.791+0000 7fcc93400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.791+0000 7fcc93400640 1 -- 172.21.15.42:0/4294669018 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98046050 con 0x55fce243b4d0 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.791+0000 7fcc93400640 1 -- 172.21.15.42:0/4294669018 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fcc98004880 con 0x55fce243b4d0 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.792+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc98052070 con 0x55fce243b4d0 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.792+0000 7fcc93400640 1 -- 172.21.15.42:0/4294669018 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcc9806c050 con 0x55fce243b4d0 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.792+0000 7fccbb365740 10 client.36035 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.792+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fce2883080 con 0x55fce243b4d0 2024-05-27T10:25:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fcc93400640 1 -- 172.21.15.42:0/4294669018 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc98052070 con 0x55fce243b4d0 2024-05-27T10:25:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fcc93400640 1 -- 172.21.15.42:0/4294669018 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcc98049070 con 0x55fce243b4d0 2024-05-27T10:25:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fccbb365740 10 client.36035 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fccbb365740 10 client.36035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fccbb365740 20 client.36035 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fccbb365740 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce23f01c0 0x55fce2885700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fccbb365740 4 client.36035 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.793+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fce276b2c0 con 0x55fce23f01c0 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.794+0000 7fccb1a00640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce23f01c0 0x55fce2885700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.795+0000 7fccb1a00640 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce23f01c0 0x55fce2885700 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f744f0f6c13fdc2c server_cookie=2fdd0389eeb35849 in_seq=0 out_seq=0 2024-05-27T10:25:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.795+0000 7fcc93400640 10 client.36035 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.813+0000 7fcc93400640 1 -- 172.21.15.42:0/4294669018 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fce276b2c0 con 0x55fce23f01c0 2024-05-27T10:25:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.813+0000 7fcc93400640 10 client.36035 handle_command_reply: tid=0 2024-05-27T10:25:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 client.36035 shutdown 2024-05-27T10:25:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 msgr2=0x55fce280e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce280e690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc9802a440 tx=0x7fcc980038a0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 shutdown_connections 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55fce23f01c0 0x55fce2885700 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fce245c1b0 0x55fce280ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fcc93400640 0 client.36035 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fce243b4d0 0x55fce280e690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 --2- 172.21.15.42:0/4294669018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fce24332f0 0x55fce2811320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 >> 172.21.15.42:0/4294669018 conn(0x55fce2450300 msgr2=0x55fce28089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 shutdown_connections 2024-05-27T10:25:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.814+0000 7fccbb365740 1 -- 172.21.15.42:0/4294669018 wait complete. 2024-05-27T10:25:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.815+0000 7fccbb365740 20 client.36035 trim_cache size 0 max 16384 2024-05-27T10:25:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:25:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:25:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc940b29d0 msgr2=0x7fcc940b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc940b29d0 0x7fcc940b4ee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc980057c0 tx=0x7fcc98041040 comp rx=0 tx=0).stop 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 msgr2=0x7fccac10e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 0x7fccac10e4e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcca400d680 tx=0x7fcca400db40 comp rx=0 tx=0).stop 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 shutdown_connections 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fcc940b29d0 0x7fcc940b4ee0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccac110a20 0x7fccac10e4e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac1026b0 0x7fccac10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 --2- 172.21.15.42:0/3037135719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccac101d60 0x7fccac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 >> 172.21.15.42:0/3037135719 conn(0x7fccac0779d0 msgr2=0x7fccac111ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 shutdown_connections 2024-05-27T10:25:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:19.816+0000 7fccb4400640 1 -- 172.21.15.42:0/3037135719 wait complete. 2024-05-27T10:25:20.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:20 smithi042 ceph-mon[31661]: pgmap v201: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 121 op/s 2024-05-27T10:25:20.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:20 smithi104 ceph-mon[34812]: pgmap v201: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 121 op/s 2024-05-27T10:25:21.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:20 smithi143 ceph-mon[35545]: pgmap v201: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 121 op/s 2024-05-27T10:25:22.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:22 smithi042 ceph-mon[31661]: pgmap v202: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 81 op/s 2024-05-27T10:25:22.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:22 smithi104 ceph-mon[34812]: pgmap v202: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 81 op/s 2024-05-27T10:25:23.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:22 smithi143 ceph-mon[35545]: pgmap v202: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 81 op/s 2024-05-27T10:25:24.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:24 smithi042 ceph-mon[31661]: pgmap v203: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 81 op/s 2024-05-27T10:25:24.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:24 smithi104 ceph-mon[34812]: pgmap v203: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 81 op/s 2024-05-27T10:25:25.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:24 smithi143 ceph-mon[35545]: pgmap v203: 97 pgs: 97 active+clean; 250 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 81 op/s 2024-05-27T10:25:26.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:26 smithi042 ceph-mon[31661]: pgmap v204: 97 pgs: 97 active+clean; 264 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 133 op/s 2024-05-27T10:25:26.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:26 smithi104 ceph-mon[34812]: pgmap v204: 97 pgs: 97 active+clean; 264 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 133 op/s 2024-05-27T10:25:27.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:26 smithi143 ceph-mon[35545]: pgmap v204: 97 pgs: 97 active+clean; 264 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 133 op/s 2024-05-27T10:25:28.716 INFO:tasks.workunit.client.0.smithi042.stdout: 39 7350 195 5240 193 2553 253 2024-05-27T10:25:28.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:28 smithi104 ceph-mon[34812]: pgmap v205: 97 pgs: 97 active+clean; 264 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-05-27T10:25:29.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:28 smithi143 ceph-mon[35545]: pgmap v205: 97 pgs: 97 active+clean; 264 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-05-27T10:25:29.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:28 smithi042 ceph-mon[31661]: pgmap v205: 97 pgs: 97 active+clean; 264 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-05-27T10:25:30.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:30 smithi104 ceph-mon[34812]: pgmap v206: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-05-27T10:25:31.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:30 smithi143 ceph-mon[35545]: pgmap v206: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-05-27T10:25:31.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:30 smithi042 ceph-mon[31661]: pgmap v206: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-05-27T10:25:32.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:32 smithi104 ceph-mon[34812]: pgmap v207: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 99 op/s 2024-05-27T10:25:33.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:32 smithi143 ceph-mon[35545]: pgmap v207: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 99 op/s 2024-05-27T10:25:33.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:32 smithi042 ceph-mon[31661]: pgmap v207: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 99 op/s 2024-05-27T10:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 -- 172.21.15.42:0/1839931211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234077610 msgr2=0x7f5234077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 --2- 172.21.15.42:0/1839931211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234077610 0x7f5234077a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 -- 172.21.15.42:0/1839931211 shutdown_connections 2024-05-27T10:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 --2- 172.21.15.42:0/1839931211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234114940 0x7f5234116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 --2- 172.21.15.42:0/1839931211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5234111fe0 0x7f5234114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 --2- 172.21.15.42:0/1839931211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234077610 0x7f5234077a20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 -- 172.21.15.42:0/1839931211 >> 172.21.15.42:0/1839931211 conn(0x7f523407d510 msgr2=0x7f523407d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 -- 172.21.15.42:0/1839931211 shutdown_connections 2024-05-27T10:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.115+0000 7f523a800640 1 -- 172.21.15.42:0/1839931211 wait complete. 2024-05-27T10:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.116+0000 7f523a800640 1 Processor -- start 2024-05-27T10:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.117+0000 7f523a800640 1 -- start start 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.117+0000 7f523a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5234077610 0x7f52341a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.117+0000 7f523a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234111fe0 0x7f52341a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.117+0000 7f523a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234114940 0x7f52341a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f5238a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234111fe0 0x7f52341a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f5238a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234111fe0 0x7f52341a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43544/0 (socket says 172.21.15.42:43544) 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f523a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f523411a160 con 0x7f5234111fe0 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f5238a00640 1 -- 172.21.15.42:0/531207031 learned_addr learned my addr 172.21.15.42:0/531207031 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5234119fe0 con 0x7f5234114940 2024-05-27T10:25:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f523411a2e0 con 0x7f5234077610 2024-05-27T10:25:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f5239400640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5234077610 0x7f52341a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.118+0000 7f5239e00640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234114940 0x7f52341a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.119+0000 7f5239e00640 1 -- 172.21.15.42:0/531207031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5234077610 msgr2=0x7f52341a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.119+0000 7f5239e00640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5234077610 0x7f52341a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.119+0000 7f5239e00640 1 -- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234111fe0 msgr2=0x7f52341a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.119+0000 7f5239e00640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234111fe0 0x7f52341a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.119+0000 7f5239e00640 1 -- 172.21.15.42:0/531207031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52341a1140 con 0x7f5234114940 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.119+0000 7f5239e00640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234114940 0x7f52341a0aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f522c00ec50 tx=0x7f522c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.120+0000 7f5232000640 1 -- 172.21.15.42:0/531207031 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c003b70 con 0x7f5234114940 2024-05-27T10:25:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.120+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52341a1430 con 0x7f5234114940 2024-05-27T10:25:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.120+0000 7f5232000640 1 -- 172.21.15.42:0/531207031 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522c003d10 con 0x7f5234114940 2024-05-27T10:25:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.120+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341ad710 con 0x7f5234114940 2024-05-27T10:25:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.120+0000 7f5232000640 1 -- 172.21.15.42:0/531207031 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c010740 con 0x7f5234114940 2024-05-27T10:25:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.125+0000 7f5242f6c740 1 Processor -- start 2024-05-27T10:25:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.125+0000 7f5232000640 1 -- 172.21.15.42:0/531207031 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f522c003fe0 con 0x7f5234114940 2024-05-27T10:25:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.125+0000 7f5242f6c740 1 -- start start 2024-05-27T10:25:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8c5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88942f0 0x5583f8c5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5242f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5583f8bd0b90 con 0x5583f889c4d0 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5242f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5583f8bc0730 con 0x5583f88bd1b0 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5242f6c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5583f8bc4770 con 0x5583f88942f0 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.126+0000 7f5238a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43556/0 (socket says 172.21.15.42:43556) 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 -- 172.21.15.42:0/1073893710 learned_addr learned my addr 172.21.15.42:0/1073893710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5239400640 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88942f0 0x5583f8c5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 -- 172.21.15.42:0/1073893710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88942f0 msgr2=0x5583f8c5ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88942f0 0x5583f8c5ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 -- 172.21.15.42:0/1073893710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 msgr2=0x5583f8c5fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8c5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.127+0000 7f5238a00640 1 -- 172.21.15.42:0/1073893710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f8b598b0 con 0x5583f889c4d0 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.128+0000 7f5232000640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f521c0b29d0 0x7f521c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.128+0000 7f5238a00640 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c604e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f522000be00 tx=0x7f52200127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77e02ffa5f8f979 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.128+0000 7f5232000640 1 -- 172.21.15.42:0/531207031 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f522c0e2a80 con 0x7f5234114940 2024-05-27T10:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.128+0000 7f5239400640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f521c0b29d0 0x7f521c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.129+0000 7f5226600640 1 -- 172.21.15.42:0/1073893710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220010650 con 0x5583f889c4d0 2024-05-27T10:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.129+0000 7f5226600640 1 -- 172.21.15.42:0/1073893710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5220010c80 con 0x5583f889c4d0 2024-05-27T10:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.129+0000 7f5226600640 1 -- 172.21.15.42:0/1073893710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522001ab90 con 0x5583f889c4d0 2024-05-27T10:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 -- 172.21.15.42:0/1073893710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 msgr2=0x5583f8c604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c604e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f522000be00 tx=0x7f52200127c0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5239400640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f521c0b29d0 0x7f521c0b4ee0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52341a1be0 tx=0x7f52280058d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 -- 172.21.15.42:0/1073893710 shutdown_connections 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c604e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88942f0 0x5583f8c5ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 --2- 172.21.15.42:0/1073893710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8c5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.130+0000 7f5242f6c740 1 -- 172.21.15.42:0/1073893710 >> 172.21.15.42:0/1073893710 conn(0x5583f88b1300 msgr2=0x5583f8c5c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.131+0000 7f5242f6c740 1 -- 172.21.15.42:0/1073893710 shutdown_connections 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.131+0000 7f5242f6c740 1 -- 172.21.15.42:0/1073893710 wait complete. 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.131+0000 7f5242f6c740 1 -- start start 2024-05-27T10:25:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 Processor -- start 2024-05-27T10:25:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88bd1b0 0x5583f8c75f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5583f88a6790 con 0x5583f88bd1b0 2024-05-27T10:25:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5583f8b68bc0 con 0x5583f88942f0 2024-05-27T10:25:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.132+0000 7f5242f6c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5583f8c6cb90 con 0x5583f889c4d0 2024-05-27T10:25:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.133+0000 7f5242f6c740 10 client.0 _ll_register_callbacks cb 0x5583f8b46400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.133+0000 7f5239e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88bd1b0 0x5583f8c75f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.133+0000 7f5239400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.133+0000 7f5239400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c69c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:46164/0 (socket says 172.21.15.42:46164) 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.133+0000 7f5239400640 1 -- 172.21.15.42:0/2655490098 learned_addr learned my addr 172.21.15.42:0/2655490098 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.133+0000 7f5238a00640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.134+0000 7f5239e00640 1 -- 172.21.15.42:0/2655490098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 msgr2=0x5583f8c6a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.134+0000 7f5239e00640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.134+0000 7f5239e00640 1 -- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 msgr2=0x5583f8c69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.134+0000 7f5239e00640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c69c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.134+0000 7f5239e00640 1 -- 172.21.15.42:0/2655490098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f88d5210 con 0x5583f88bd1b0 2024-05-27T10:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.134+0000 7f5239400640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.135+0000 7f5239e00640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88bd1b0 0x5583f8c75f10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f522c0e01c0 tx=0x7f522c0a6260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8e862185d8a759 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.135+0000 7f521aa00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.135+0000 7f521aa00640 1 -- 172.21.15.42:0/2655490098 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0e0650 con 0x5583f88bd1b0 2024-05-27T10:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.135+0000 7f521aa00640 1 -- 172.21.15.42:0/2655490098 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f522c0b2720 con 0x5583f88bd1b0 2024-05-27T10:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.136+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5220010c80 con 0x5583f88bd1b0 2024-05-27T10:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.136+0000 7f521aa00640 1 -- 172.21.15.42:0/2655490098 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f522c0a9860 con 0x5583f88bd1b0 2024-05-27T10:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.136+0000 7f5242f6c740 10 client.16521 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.136+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5583f89a1620 con 0x5583f88bd1b0 2024-05-27T10:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.137+0000 7f521aa00640 1 -- 172.21.15.42:0/2655490098 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5220010c80 con 0x5583f88bd1b0 2024-05-27T10:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.137+0000 7f521aa00640 1 -- 172.21.15.42:0/2655490098 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f522c0a56c0 con 0x5583f88bd1b0 2024-05-27T10:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.137+0000 7f5242f6c740 10 client.16521 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.137+0000 7f5242f6c740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.137+0000 7f5242f6c740 20 client.16521 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.137+0000 7f5242f6c740 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f887bd80 0x5583f8c7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.138+0000 7f5242f6c740 4 client.16521 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:25:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.138+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f8bd2ae0 con 0x5583f887bd80 2024-05-27T10:25:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.138+0000 7f5239400640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f887bd80 0x5583f8c7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.139+0000 7f5239400640 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f887bd80 0x5583f8c7bf60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ba769d9c9ca0d5 server_cookie=5d40d615a24c8a9d in_seq=0 out_seq=0 2024-05-27T10:25:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.139+0000 7f521aa00640 10 client.16521 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.140+0000 7f521aa00640 1 -- 172.21.15.42:0/2655490098 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5583f8bd2ae0 con 0x5583f887bd80 2024-05-27T10:25:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.140+0000 7f521aa00640 10 client.16521 handle_command_reply: tid=0 2024-05-27T10:25:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 client.16521 shutdown 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88bd1b0 msgr2=0x5583f8c75f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88bd1b0 0x5583f8c75f10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f522c0e01c0 tx=0x7f522c0a6260 comp rx=0 tx=0).stop 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 shutdown_connections 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88bd1b0 0x5583f8c75f10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f887bd80 0x5583f8c7bf60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 --2- 172.21.15.42:0/2655490098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 >> 172.21.15.42:0/2655490098 conn(0x5583f88b1300 msgr2=0x5583f8c61780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.141+0000 7f521aa00640 0 client.16521 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.142+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 shutdown_connections 2024-05-27T10:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.142+0000 7f5242f6c740 1 -- 172.21.15.42:0/2655490098 wait complete. 2024-05-27T10:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.142+0000 7f5242f6c740 20 client.16521 trim_cache size 0 max 16384 2024-05-27T10:25:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 Processor -- start 2024-05-27T10:25:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 -- start start 2024-05-27T10:25:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c65440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c69720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88bd1b0 0x5583f8c637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b8640 con 0x5583f889c4d0 2024-05-27T10:25:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b5eb0 con 0x5583f88942f0 2024-05-27T10:25:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.168+0000 7f5242f6c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b3d00 con 0x5583f88bd1b0 2024-05-27T10:25:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.169+0000 7f5239400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c65440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.169+0000 7f5238a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c69720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.169+0000 7f5239400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c65440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:46170/0 (socket says 172.21.15.42:46170) 2024-05-27T10:25:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.169+0000 7f5239400640 1 -- 172.21.15.42:0/3235261114 learned_addr learned my addr 172.21.15.42:0/3235261114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.169+0000 7f5239e00640 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88bd1b0 0x5583f8c637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.170+0000 7f5238a00640 1 -- 172.21.15.42:0/3235261114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88bd1b0 msgr2=0x5583f8c637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.170+0000 7f5238a00640 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88bd1b0 0x5583f8c637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.170+0000 7f5238a00640 1 -- 172.21.15.42:0/3235261114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 msgr2=0x5583f8c65440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.170+0000 7f5238a00640 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c65440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.170+0000 7f5238a00640 1 -- 172.21.15.42:0/3235261114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f8c681b0 con 0x5583f889c4d0 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.171+0000 7f5238a00640 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c69720 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f522000b450 tx=0x7f5220001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cd700cc3ba78c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.171+0000 7f5225c00640 1 -- 172.21.15.42:0/3235261114 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52200050b0 con 0x5583f889c4d0 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.171+0000 7f5225c00640 1 -- 172.21.15.42:0/3235261114 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5220029070 con 0x5583f889c4d0 2024-05-27T10:25:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.171+0000 7f5225c00640 1 -- 172.21.15.42:0/3235261114 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522001b440 con 0x5583f889c4d0 2024-05-27T10:25:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.172+0000 7f5242f6c740 1 -- 172.21.15.42:0/3235261114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 msgr2=0x5583f8c69720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.172+0000 7f5242f6c740 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c69720 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f522000b450 tx=0x7f5220001e50 comp rx=0 tx=0).stop 2024-05-27T10:25:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 -- 172.21.15.42:0/3235261114 shutdown_connections 2024-05-27T10:25:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f88bd1b0 0x5583f8c637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f889c4d0 0x5583f8c69720 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 --2- 172.21.15.42:0/3235261114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88942f0 0x5583f8c65440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 -- 172.21.15.42:0/3235261114 >> 172.21.15.42:0/3235261114 conn(0x5583f88b1300 msgr2=0x5583f8c6bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 -- 172.21.15.42:0/3235261114 shutdown_connections 2024-05-27T10:25:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 -- 172.21.15.42:0/3235261114 wait complete. 2024-05-27T10:25:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.173+0000 7f5242f6c740 1 -- start start 2024-05-27T10:25:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 Processor -- start 2024-05-27T10:25:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 0x5583f8c6cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8ce2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b4370 con 0x5583f88942f0 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b57e0 con 0x5583f88bd1b0 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b4a40 con 0x5583f889c4d0 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.174+0000 7f5242f6c740 10 client.0 _ll_register_callbacks cb 0x7f522c00cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 0x5583f8c6cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 0x5583f8c6cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43584/0 (socket says 172.21.15.42:43584) 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 -- 172.21.15.42:0/203908501 learned_addr learned my addr 172.21.15.42:0/203908501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239e00640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8ce2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5238a00640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 -- 172.21.15.42:0/203908501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 msgr2=0x5583f8c6d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 -- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 msgr2=0x5583f8ce2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8ce2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.175+0000 7f5239400640 1 -- 172.21.15.42:0/203908501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f8c6dd40 con 0x5583f88942f0 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.176+0000 7f5239400640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 0x5583f8c6cf30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5228006010 tx=0x7f5228009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a555473e720833 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.176+0000 7f521aa00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:25:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.176+0000 7f521aa00640 1 -- 172.21.15.42:0/203908501 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035040 con 0x5583f88942f0 2024-05-27T10:25:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.176+0000 7f521aa00640 1 -- 172.21.15.42:0/203908501 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f5228003dd0 con 0x5583f88942f0 2024-05-27T10:25:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.176+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5220029070 con 0x5583f88942f0 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f521aa00640 1 -- 172.21.15.42:0/203908501 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f52280044b0 con 0x5583f88942f0 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f5242f6c740 10 client.16527 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5583f8ce3c30 con 0x5583f88942f0 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f521aa00640 1 -- 172.21.15.42:0/203908501 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5220029070 con 0x5583f88942f0 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f521aa00640 1 -- 172.21.15.42:0/203908501 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5228050050 con 0x5583f88942f0 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f5242f6c740 10 client.16527 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.177+0000 7f5242f6c740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.178+0000 7f5242f6c740 20 client.16527 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.178+0000 7f5242f6c740 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f88511c0 0x5583f8ce6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.178+0000 7f5242f6c740 4 client.16527 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:25:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.178+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583f8c6a5d0 con 0x5583f88511c0 2024-05-27T10:25:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.178+0000 7f5238a00640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f88511c0 0x5583f8ce6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.179+0000 7f5238a00640 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f88511c0 0x5583f8ce6470 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8051ca93f250584d server_cookie=403e8a6e12eb964b in_seq=0 out_seq=0 2024-05-27T10:25:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.179+0000 7f521aa00640 10 client.16527 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f521aa00640 1 -- 172.21.15.42:0/203908501 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5583f8c6a5d0 con 0x5583f88511c0 2024-05-27T10:25:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f521aa00640 10 client.16527 handle_command_reply: tid=0 2024-05-27T10:25:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 client.16527 shutdown 2024-05-27T10:25:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 msgr2=0x5583f8c6cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 0x5583f8c6cf30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5228006010 tx=0x7f5228009260 comp rx=0 tx=0).stop 2024-05-27T10:25:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 shutdown_connections 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583f88bd1b0 0x5583f8ce2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5583f88511c0 0x5583f8ce6470 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583f889c4d0 0x5583f8c6d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 --2- 172.21.15.42:0/203908501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f88942f0 0x5583f8c6cf30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f521aa00640 0 client.16527 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 >> 172.21.15.42:0/203908501 conn(0x5583f88b1300 msgr2=0x5583f8c5f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 shutdown_connections 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.202+0000 7f5242f6c740 1 -- 172.21.15.42:0/203908501 wait complete. 2024-05-27T10:25:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.203+0000 7f5242f6c740 20 client.16527 trim_cache size 0 max 16384 2024-05-27T10:25:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:25:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:25:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f521c0b29d0 msgr2=0x7f521c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f521c0b29d0 0x7f521c0b4ee0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52341a1be0 tx=0x7f52280058d0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234114940 msgr2=0x7f52341a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234114940 0x7f52341a0aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f522c00ec50 tx=0x7f522c00c6a0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 shutdown_connections 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f521c0b29d0 0x7f521c0b4ee0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5234114940 0x7f52341a0aa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5234111fe0 0x7f52341a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 --2- 172.21.15.42:0/531207031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5234077610 0x7f52341a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.204+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 >> 172.21.15.42:0/531207031 conn(0x7f523407d510 msgr2=0x7f52341123f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.205+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 shutdown_connections 2024-05-27T10:25:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:33.205+0000 7f523a800640 1 -- 172.21.15.42:0/531207031 wait complete. 2024-05-27T10:25:34.861 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:34 smithi143 ceph-mon[35545]: pgmap v208: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 99 op/s 2024-05-27T10:25:34.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:34 smithi104 ceph-mon[34812]: pgmap v208: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 99 op/s 2024-05-27T10:25:35.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:34 smithi042 ceph-mon[31661]: pgmap v208: 97 pgs: 97 active+clean; 278 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 99 op/s 2024-05-27T10:25:36.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:36 smithi104 ceph-mon[34812]: pgmap v209: 97 pgs: 97 active+clean; 292 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 149 op/s 2024-05-27T10:25:37.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:36 smithi143 ceph-mon[35545]: pgmap v209: 97 pgs: 97 active+clean; 292 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 149 op/s 2024-05-27T10:25:37.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:36 smithi042 ceph-mon[31661]: pgmap v209: 97 pgs: 97 active+clean; 292 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 149 op/s 2024-05-27T10:25:38.716 INFO:tasks.workunit.client.0.smithi042.stdout: 42 7751 42 6049 217 2094 347 2024-05-27T10:25:38.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:38 smithi104 ceph-mon[34812]: pgmap v210: 97 pgs: 97 active+clean; 292 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-27T10:25:39.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:38 smithi143 ceph-mon[35545]: pgmap v210: 97 pgs: 97 active+clean; 292 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-27T10:25:39.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:38 smithi042 ceph-mon[31661]: pgmap v210: 97 pgs: 97 active+clean; 292 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-27T10:25:40.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:40 smithi104 ceph-mon[34812]: pgmap v211: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 150 op/s 2024-05-27T10:25:41.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:40 smithi143 ceph-mon[35545]: pgmap v211: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 150 op/s 2024-05-27T10:25:41.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:40 smithi042 ceph-mon[31661]: pgmap v211: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 150 op/s 2024-05-27T10:25:42.809 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-27T10:25:42.809 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.95 2024-05-27T10:25:42.810 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:25:42.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:42 smithi042 ceph-mon[31661]: pgmap v212: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 102 op/s 2024-05-27T10:25:42.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:42 smithi104 ceph-mon[34812]: pgmap v212: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 102 op/s 2024-05-27T10:25:43.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:42 smithi143 ceph-mon[35545]: pgmap v212: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 102 op/s 2024-05-27T10:25:43.404 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:25:44.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:44 smithi104 ceph-mon[34812]: pgmap v213: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 102 op/s 2024-05-27T10:25:45.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:44 smithi143 ceph-mon[35545]: pgmap v213: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 102 op/s 2024-05-27T10:25:45.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:44 smithi042 ceph-mon[31661]: pgmap v213: 97 pgs: 97 active+clean; 305 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 102 op/s 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 -- 172.21.15.42:0/3662404350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4107de0 msgr2=0x7f9ed41081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 --2- 172.21.15.42:0/3662404350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4107de0 0x7f9ed41081f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00099c0 tx=0x7f9ec0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 -- 172.21.15.42:0/3662404350 shutdown_connections 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 --2- 172.21.15.42:0/3662404350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ed4108730 0x7f9ed4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 --2- 172.21.15.42:0/3662404350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 0x7f9ed41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 --2- 172.21.15.42:0/3662404350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4107de0 0x7f9ed41081f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.237+0000 7f9eda400700 1 -- 172.21.15.42:0/3662404350 >> 172.21.15.42:0/3662404350 conn(0x7f9ed40fedb0 msgr2=0x7f9ed41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.238+0000 7f9eda400700 1 -- 172.21.15.42:0/3662404350 shutdown_connections 2024-05-27T10:25:46.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.238+0000 7f9eda400700 1 -- 172.21.15.42:0/3662404350 wait complete. 2024-05-27T10:25:46.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.239+0000 7f9eda400700 1 Processor -- start 2024-05-27T10:25:46.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.240+0000 7f9eda400700 1 -- start start 2024-05-27T10:25:46.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.240+0000 7f9eda400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 0x7f9ed410d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.240+0000 7f9eda400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ed4107de0 0x7f9ed410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.240+0000 7f9eda400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4108730 0x7f9ed410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9eda400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41120d0 con 0x7f9ed4108730 2024-05-27T10:25:46.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9eda400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4111f50 con 0x7f9ed41034b0 2024-05-27T10:25:46.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9eda400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4112250 con 0x7f9ed4107de0 2024-05-27T10:25:46.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9ed3400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 0x7f9ed410d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9ed3400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 0x7f9ed410d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45462/0 (socket says 172.21.15.42:45462) 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9ed3400700 1 -- 172.21.15.42:0/765812656 learned_addr learned my addr 172.21.15.42:0/765812656 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9ed3e00700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4108730 0x7f9ed410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.241+0000 7f9ed2a00700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ed4107de0 0x7f9ed410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.242+0000 7f9ed3e00700 1 -- 172.21.15.42:0/765812656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ed4107de0 msgr2=0x7f9ed410d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.242+0000 7f9ed3e00700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ed4107de0 0x7f9ed410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.242+0000 7f9ed3e00700 1 -- 172.21.15.42:0/765812656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 msgr2=0x7f9ed410d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.242+0000 7f9ed3e00700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 0x7f9ed410d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.242+0000 7f9ed3e00700 1 -- 172.21.15.42:0/765812656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed41ad500 con 0x7f9ed4108730 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.242+0000 7f9ed3e00700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4108730 0x7f9ed410def0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00ab70 tx=0x7f9ecc00ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.243+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc017020 con 0x7f9ed4108730 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.243+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ecc004bb0 con 0x7f9ed4108730 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.243+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc005280 con 0x7f9ed4108730 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.243+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed41ad7f0 con 0x7f9ed4108730 2024-05-27T10:25:46.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.243+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed41adce0 con 0x7f9ed4108730 2024-05-27T10:25:46.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.244+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed4104960 con 0x7f9ed4108730 2024-05-27T10:25:46.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.250+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9ecc00c600 con 0x7f9ed4108730 2024-05-27T10:25:46.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.252+0000 7f9ecbe00700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9eb80b2af0 0x7f9eb80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.252+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f9ecc0e2110 con 0x7f9ed4108730 2024-05-27T10:25:46.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.253+0000 7f9ed3400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9eb80b2af0 0x7f9eb80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.254+0000 7f9ed3400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9eb80b2af0 0x7f9eb80b5000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009d20 tx=0x7f9ec4009260 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.257+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ecc021020 con 0x7f9ed4108730 2024-05-27T10:25:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.510+0000 7fe937e00640 1 -- 172.21.15.42:0/3246952702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810e000 msgr2=0x7fe9381104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.510+0000 7fe937e00640 1 --2- 172.21.15.42:0/3246952702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810e000 0x7fe9381104f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe930009ca0 tx=0x7fe93002a8c0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 -- 172.21.15.42:0/3246952702 shutdown_connections 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 --2- 172.21.15.42:0/3246952702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810e000 0x7fe9381104f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 --2- 172.21.15.42:0/3246952702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93810b610 0x7fe93810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 --2- 172.21.15.42:0/3246952702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe938104bb0 0x7fe938104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 -- 172.21.15.42:0/3246952702 >> 172.21.15.42:0/3246952702 conn(0x7fe9380fe210 msgr2=0x7fe938100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 -- 172.21.15.42:0/3246952702 shutdown_connections 2024-05-27T10:25:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.511+0000 7fe937e00640 1 -- 172.21.15.42:0/3246952702 wait complete. 2024-05-27T10:25:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.512+0000 7fe937e00640 1 Processor -- start 2024-05-27T10:25:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 -- start start 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe938104bb0 0x7fe9381084a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 0x7fe9381089e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe93810e000 0x7fe938108f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe938103dc0 con 0x7fe93810b610 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe938103c40 con 0x7fe93810e000 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.513+0000 7fe937e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe938103f40 con 0x7fe938104bb0 2024-05-27T10:25:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.514+0000 7fe936000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 0x7fe9381089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.514+0000 7fe936000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 0x7fe9381089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41882/0 (socket says 172.21.15.42:41882) 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.514+0000 7fe936a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe938104bb0 0x7fe9381084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.514+0000 7fe936000640 1 -- 172.21.15.42:0/3177158776 learned_addr learned my addr 172.21.15.42:0/3177158776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.514+0000 7fe937400640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe93810e000 0x7fe938108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe936000640 1 -- 172.21.15.42:0/3177158776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe938104bb0 msgr2=0x7fe9381084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe936000640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe938104bb0 0x7fe9381084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe936000640 1 -- 172.21.15.42:0/3177158776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe93810e000 msgr2=0x7fe938108f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe936000640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe93810e000 0x7fe938108f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe936000640 1 -- 172.21.15.42:0/3177158776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9381acc30 con 0x7fe93810b610 2024-05-27T10:25:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe937400640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe93810e000 0x7fe938108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:25:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe936000640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 0x7fe9381089e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe92400e9f0 tx=0x7fe92400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.515+0000 7fe92f400640 1 -- 172.21.15.42:0/3177158776 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92400cd80 con 0x7fe93810b610 2024-05-27T10:25:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.516+0000 7fe92f400640 1 -- 172.21.15.42:0/3177158776 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9240108c0 con 0x7fe93810b610 2024-05-27T10:25:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.516+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9381acf20 con 0x7fe93810b610 2024-05-27T10:25:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.516+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9381ad390 con 0x7fe93810b610 2024-05-27T10:25:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.516+0000 7fe92f400640 1 -- 172.21.15.42:0/3177158776 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240187c0 con 0x7fe93810b610 2024-05-27T10:25:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.520+0000 7fe9447ca740 1 Processor -- start 2024-05-27T10:25:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.520+0000 7fe9447ca740 1 -- start start 2024-05-27T10:25:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.520+0000 7fe92f400640 1 -- 172.21.15.42:0/3177158776 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe924018a40 con 0x7fe93810b610 2024-05-27T10:25:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e8ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e8c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe9447ca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55de35dfcb90 con 0x55de35ac84d0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe9447ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55de35dec730 con 0x55de35ae91b0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe9447ca740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55de35df0770 con 0x55de35ac02f0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe92f400640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe91c0b2950 0x7fe91c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe92f400640 1 -- 172.21.15.42:0/3177158776 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe9240e2910 con 0x7fe93810b610 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe937400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e8ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe937400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e8ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45496/0 (socket says 172.21.15.42:45496) 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe937400640 1 -- 172.21.15.42:0/3209497550 learned_addr learned my addr 172.21.15.42:0/3209497550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe937400640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe91c0b2950 0x7fe91c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.523+0000 7fe936a00640 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe936000640 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e8c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe937400640 1 -- 172.21.15.42:0/3209497550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 msgr2=0x55de35e8bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe937400640 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e8bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe937400640 1 -- 172.21.15.42:0/3209497550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 msgr2=0x55de35e8c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe937400640 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e8c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe937400640 1 -- 172.21.15.42:0/3209497550 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de35d858b0 con 0x55de35ae91b0 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe936000640 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e8c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.524+0000 7fe937400640 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e8ba60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9300094e0 tx=0x7fe930003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75434e9956e97d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.525+0000 7fe937400640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe91c0b2950 0x7fe91c0b4e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe930009ca0 tx=0x7fe930039000 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.525+0000 7fe923e00640 1 -- 172.21.15.42:0/3209497550 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93004e070 con 0x55de35ae91b0 2024-05-27T10:25:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.525+0000 7fe923e00640 1 -- 172.21.15.42:0/3209497550 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe930036040 con 0x55de35ae91b0 2024-05-27T10:25:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.527+0000 7fe9447ca740 1 -- 172.21.15.42:0/3209497550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 msgr2=0x55de35e8ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.527+0000 7fe9447ca740 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e8ba60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9300094e0 tx=0x7fe930003cc0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.527+0000 7fe9447ca740 1 -- 172.21.15.42:0/3209497550 shutdown_connections 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.527+0000 7fe9447ca740 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e8c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.527+0000 7fe9447ca740 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e8bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.528+0000 7fe9447ca740 1 --2- 172.21.15.42:0/3209497550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e8ba60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.528+0000 7fe9447ca740 1 -- 172.21.15.42:0/3209497550 >> 172.21.15.42:0/3209497550 conn(0x55de35add300 msgr2=0x55de35e88210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.528+0000 7fe9447ca740 1 -- 172.21.15.42:0/3209497550 shutdown_connections 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.528+0000 7fe9447ca740 1 -- 172.21.15.42:0/3209497550 wait complete. 2024-05-27T10:25:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.529+0000 7fe9447ca740 1 -- start start 2024-05-27T10:25:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.529+0000 7fe9447ca740 1 Processor -- start 2024-05-27T10:25:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e95bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac84d0 0x55de35e960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35ea1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55de35ad2790 con 0x55de35ac02f0 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55de35d94bc0 con 0x55de35ae91b0 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55de35e98b80 con 0x55de35ac84d0 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe9447ca740 10 client.0 _ll_register_callbacks cb 0x55de35d72400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe936a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e95bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.530+0000 7fe936a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e95bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41898/0 (socket says 172.21.15.42:41898) 2024-05-27T10:25:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe936a00640 1 -- 172.21.15.42:0/563144538 learned_addr learned my addr 172.21.15.42:0/563144538 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe937400640 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35ea1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe937400640 1 -- 172.21.15.42:0/563144538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac84d0 msgr2=0x55de35e960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe937400640 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac84d0 0x55de35e960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe937400640 1 -- 172.21.15.42:0/563144538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 msgr2=0x55de35e95bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe937400640 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e95bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.531+0000 7fe937400640 1 -- 172.21.15.42:0/563144538 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de35b01210 con 0x55de35ae91b0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.532+0000 7fe937400640 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35ea1f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe930031920 tx=0x7fe93002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671b0fa626d629fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.532+0000 7fe920c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.532+0000 7fe920c00640 1 -- 172.21.15.42:0/563144538 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93004e030 con 0x55de35ae91b0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.532+0000 7fe920c00640 1 -- 172.21.15.42:0/563144538 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe930030c80 con 0x55de35ae91b0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.533+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe930036040 con 0x55de35ae91b0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.533+0000 7fe920c00640 1 -- 172.21.15.42:0/563144538 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe930030e80 con 0x55de35ae91b0 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.533+0000 7fe9447ca740 10 client.36068 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.533+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55de35bcd620 con 0x55de35ae91b0 2024-05-27T10:25:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.534+0000 7fe920c00640 1 -- 172.21.15.42:0/563144538 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe930036040 con 0x55de35ae91b0 2024-05-27T10:25:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.534+0000 7fe920c00640 1 -- 172.21.15.42:0/563144538 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe930053cd0 con 0x55de35ae91b0 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.535+0000 7fe9447ca740 10 client.36068 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.535+0000 7fe9447ca740 10 client.36068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.535+0000 7fe9447ca740 20 client.36068 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.535+0000 7fe9447ca740 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35aa7d80 0x55de35ea7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.535+0000 7fe9447ca740 4 client.36068 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.535+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de35dfeae0 con 0x55de35aa7d80 2024-05-27T10:25:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.536+0000 7fe936a00640 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35aa7d80 0x55de35ea7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.537+0000 7fe936a00640 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35aa7d80 0x55de35ea7c80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64a82695248772c server_cookie=e0a1b0e82c47939d in_seq=0 out_seq=0 2024-05-27T10:25:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.537+0000 7fe920c00640 10 client.36068 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.539+0000 7fe920c00640 1 -- 172.21.15.42:0/563144538 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55de35dfeae0 con 0x55de35aa7d80 2024-05-27T10:25:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.539+0000 7fe920c00640 10 client.36068 handle_command_reply: tid=0 2024-05-27T10:25:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.539+0000 7fe9447ca740 1 client.36068 shutdown 2024-05-27T10:25:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.539+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 msgr2=0x55de35ea1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.539+0000 7fe9447ca740 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35ea1f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe930031920 tx=0x7fe93002a8c0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 shutdown_connections 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35ea1f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35aa7d80 0x55de35ea7c80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac84d0 0x55de35e960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 --2- 172.21.15.42:0/563144538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e95bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 >> 172.21.15.42:0/563144538 conn(0x55de35add300 msgr2=0x55de35e8b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe920c00640 0 client.36068 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.540+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 shutdown_connections 2024-05-27T10:25:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.541+0000 7fe9447ca740 1 -- 172.21.15.42:0/563144538 wait complete. 2024-05-27T10:25:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.541+0000 7fe9447ca740 20 client.36068 trim_cache size 0 max 16384 2024-05-27T10:25:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.567+0000 7fe9447ca740 1 Processor -- start 2024-05-27T10:25:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.567+0000 7fe9447ca740 1 -- start start 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ac84d0 0x55de35e94af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ae91b0 0x55de35ea1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe9447ca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe918010fd0 con 0x55de35ac02f0 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe9447ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe91800e840 con 0x55de35ac84d0 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe9447ca740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe91800c690 con 0x55de35ae91b0 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe936a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe936a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41908/0 (socket says 172.21.15.42:41908) 2024-05-27T10:25:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe936a00640 1 -- 172.21.15.42:0/1216876953 learned_addr learned my addr 172.21.15.42:0/1216876953 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe936000640 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ac84d0 0x55de35e94af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.568+0000 7fe937400640 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ae91b0 0x55de35ea1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.569+0000 7fe936a00640 1 -- 172.21.15.42:0/1216876953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ae91b0 msgr2=0x55de35ea1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.569+0000 7fe936a00640 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ae91b0 0x55de35ea1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.569+0000 7fe936a00640 1 -- 172.21.15.42:0/1216876953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ac84d0 msgr2=0x55de35e94af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.569+0000 7fe936a00640 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ac84d0 0x55de35e94af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.569+0000 7fe936a00640 1 -- 172.21.15.42:0/1216876953 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de35e8d510 con 0x55de35ac02f0 2024-05-27T10:25:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.570+0000 7fe936a00640 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e982e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55de35ea8220 tx=0x7fe92800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d37014eb7923dde server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.570+0000 7fe923400640 1 -- 172.21.15.42:0/1216876953 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928003e20 con 0x55de35ac02f0 2024-05-27T10:25:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.571+0000 7fe923400640 1 -- 172.21.15.42:0/1216876953 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9280228e0 con 0x55de35ac02f0 2024-05-27T10:25:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.571+0000 7fe923400640 1 -- 172.21.15.42:0/1216876953 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928021880 con 0x55de35ac02f0 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 -- 172.21.15.42:0/1216876953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 msgr2=0x55de35e982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e982e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55de35ea8220 tx=0x7fe92800b390 comp rx=0 tx=0).stop 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 -- 172.21.15.42:0/1216876953 shutdown_connections 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ae91b0 0x55de35ea1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ac84d0 0x55de35e94af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 --2- 172.21.15.42:0/1216876953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac02f0 0x55de35e982e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 -- 172.21.15.42:0/1216876953 >> 172.21.15.42:0/1216876953 conn(0x55de35add300 msgr2=0x55de35e8c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 -- 172.21.15.42:0/1216876953 shutdown_connections 2024-05-27T10:25:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.572+0000 7fe9447ca740 1 -- 172.21.15.42:0/1216876953 wait complete. 2024-05-27T10:25:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.573+0000 7fe9447ca740 1 -- start start 2024-05-27T10:25:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.573+0000 7fe9447ca740 1 Processor -- start 2024-05-27T10:25:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e98eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e99930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe91800cd00 con 0x55de35ac84d0 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe91800e170 con 0x55de35ae91b0 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe91800d3d0 con 0x55de35ac02f0 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe9447ca740 10 client.0 _ll_register_callbacks cb 0x7fe92800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.574+0000 7fe937400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e99930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe937400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e99930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45524/0 (socket says 172.21.15.42:45524) 2024-05-27T10:25:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe937400640 1 -- 172.21.15.42:0/2979998600 learned_addr learned my addr 172.21.15.42:0/2979998600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe937400640 1 -- 172.21.15.42:0/2979998600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 msgr2=0x55de35e98eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe937400640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e98eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe937400640 1 -- 172.21.15.42:0/2979998600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 msgr2=0x55de35e993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe936000640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.575+0000 7fe937400640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.576+0000 7fe937400640 1 -- 172.21.15.42:0/2979998600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de35f1f230 con 0x55de35ae91b0 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.576+0000 7fe936000640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:25:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.576+0000 7fe937400640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e99930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe930002230 tx=0x7fe93003b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f09a8b8ea4d289 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.576+0000 7fe920c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.577+0000 7fe920c00640 1 -- 172.21.15.42:0/2979998600 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93004e040 con 0x55de35ae91b0 2024-05-27T10:25:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.577+0000 7fe920c00640 1 -- 172.21.15.42:0/2979998600 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fe93003ba40 con 0x55de35ae91b0 2024-05-27T10:25:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.577+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9280228e0 con 0x55de35ae91b0 2024-05-27T10:25:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.577+0000 7fe920c00640 1 -- 172.21.15.42:0/2979998600 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe930003bc0 con 0x55de35ae91b0 2024-05-27T10:25:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.578+0000 7fe9447ca740 10 client.36074 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.578+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55de35f204a0 con 0x55de35ae91b0 2024-05-27T10:25:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.578+0000 7fe920c00640 1 -- 172.21.15.42:0/2979998600 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9280228e0 con 0x55de35ae91b0 2024-05-27T10:25:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.578+0000 7fe920c00640 1 -- 172.21.15.42:0/2979998600 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe9300054c0 con 0x55de35ae91b0 2024-05-27T10:25:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe9447ca740 10 client.36074 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe9447ca740 10 client.36074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:25:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe9447ca740 20 client.36074 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe9447ca740 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35a7d1c0 0x55de35f22a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe9447ca740 4 client.36074 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:25:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de35df82c0 con 0x55de35a7d1c0 2024-05-27T10:25:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.579+0000 7fe936a00640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35a7d1c0 0x55de35f22a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.581+0000 7fe936a00640 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35a7d1c0 0x55de35f22a70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa80290c5e0fff32 server_cookie=e11c9e4db74b23f2 in_seq=0 out_seq=0 2024-05-27T10:25:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.581+0000 7fe920c00640 10 client.36074 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.674+0000 7fe920c00640 1 -- 172.21.15.42:0/2979998600 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de35df82c0 con 0x55de35a7d1c0 2024-05-27T10:25:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.674+0000 7fe920c00640 10 client.36074 handle_command_reply: tid=0 2024-05-27T10:25:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.674+0000 7fe9447ca740 1 client.36074 shutdown 2024-05-27T10:25:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.674+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 msgr2=0x55de35e99930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e99930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe930002230 tx=0x7fe93003b950 comp rx=0 tx=0).stop 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 shutdown_connections 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de35ae91b0 0x55de35e99930 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de35ac84d0 0x55de35e993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55de35a7d1c0 0x55de35f22a70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 --2- 172.21.15.42:0/2979998600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55de35ac02f0 0x55de35e98eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 >> 172.21.15.42:0/2979998600 conn(0x55de35add300 msgr2=0x55de35e8c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe920c00640 0 client.36074 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 shutdown_connections 2024-05-27T10:25:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.675+0000 7fe9447ca740 1 -- 172.21.15.42:0/2979998600 wait complete. 2024-05-27T10:25:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.676+0000 7fe9447ca740 20 client.36074 trim_cache size 0 max 16384 2024-05-27T10:25:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:25:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:25:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:25:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe91c0b2950 msgr2=0x7fe91c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe91c0b2950 0x7fe91c0b4e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe930009ca0 tx=0x7fe930039000 comp rx=0 tx=0).stop 2024-05-27T10:25:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 msgr2=0x7fe9381089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 0x7fe9381089e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe92400e9f0 tx=0x7fe92400eeb0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 shutdown_connections 2024-05-27T10:25:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fe91c0b2950 0x7fe91c0b4e60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe93810e000 0x7fe938108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe93810b610 0x7fe9381089e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 --2- 172.21.15.42:0/3177158776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe938104bb0 0x7fe9381084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 >> 172.21.15.42:0/3177158776 conn(0x7fe9380fe210 msgr2=0x7fe93810e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 shutdown_connections 2024-05-27T10:25:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:46.677+0000 7fe937e00640 1 -- 172.21.15.42:0/3177158776 wait complete. 2024-05-27T10:25:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.737+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed402e700 con 0x7f9ed4108730 2024-05-27T10:25:46.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.739+0000 7f9ecbe00700 1 -- 172.21.15.42:0/765812656 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11881 (secure 0 0 0) 0x7f9ecc0a9b70 con 0x7f9ed4108730 2024-05-27T10:25:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:25:46.738 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:24:00.498568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:25:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9eb80b2af0 msgr2=0x7f9eb80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9eb80b2af0 0x7f9eb80b5000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009d20 tx=0x7f9ec4009260 comp rx=0 tx=0).stop 2024-05-27T10:25:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4108730 msgr2=0x7f9ed410def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4108730 0x7f9ed410def0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00ab70 tx=0x7f9ecc00ae80 comp rx=0 tx=0).stop 2024-05-27T10:25:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 shutdown_connections 2024-05-27T10:25:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f9eb80b2af0 0x7f9eb80b5000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ed4108730 0x7f9ed410def0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed41034b0 0x7f9ed410d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 --2- 172.21.15.42:0/765812656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ed4107de0 0x7f9ed410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 >> 172.21.15.42:0/765812656 conn(0x7f9ed40fedb0 msgr2=0x7f9ed4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.743+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 shutdown_connections 2024-05-27T10:25:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:46.744+0000 7f9eda400700 1 -- 172.21.15.42:0/765812656 wait complete. 2024-05-27T10:25:46.745 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 33 2024-05-27T10:25:46.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:46 smithi104 ceph-mon[34812]: pgmap v214: 97 pgs: 97 active+clean; 317 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 185 op/s 2024-05-27T10:25:47.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:46 smithi143 ceph-mon[35545]: pgmap v214: 97 pgs: 97 active+clean; 317 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 185 op/s 2024-05-27T10:25:47.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:46 smithi042 ceph-mon[31661]: pgmap v214: 97 pgs: 97 active+clean; 317 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 185 op/s 2024-05-27T10:25:47.676 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 138.95 --await-for 45.0' 2024-05-27T10:25:47.676 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 138.95 --await-for 45.0 2024-05-27T10:25:47.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:47 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/765812656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:25:48.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:47 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/765812656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:25:48.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:47 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/765812656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:25:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:25:48.716 INFO:tasks.workunit.client.0.smithi042.stdout: 44 6889 112 5201 144 2774 388 2024-05-27T10:25:48.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:48 smithi042 ceph-mon[31661]: pgmap v215: 97 pgs: 97 active+clean; 317 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-05-27T10:25:48.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:48 smithi104 ceph-mon[34812]: pgmap v215: 97 pgs: 97 active+clean; 317 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-05-27T10:25:49.058 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:48 smithi143 ceph-mon[35545]: pgmap v215: 97 pgs: 97 active+clean; 317 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-05-27T10:25:50.738 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:50 smithi042 ceph-mon[31661]: pgmap v216: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-05-27T10:25:50.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:50 smithi104 ceph-mon[34812]: pgmap v216: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-05-27T10:25:51.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:50 smithi143 ceph-mon[35545]: pgmap v216: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-05-27T10:25:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.246+0000 7f98e4a00700 1 -- 172.21.15.42:0/199516133 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0012650 con 0x7f98e8108760 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 -- 172.21.15.42:0/199516133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 msgr2=0x7f98e81100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 --2- 172.21.15.42:0/199516133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81100c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e000b0a0 tx=0x7f98e000b3b0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 -- 172.21.15.42:0/199516133 shutdown_connections 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 --2- 172.21.15.42:0/199516133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81100c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 --2- 172.21.15.42:0/199516133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e8103ff0 0x7f98e8104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 --2- 172.21.15.42:0/199516133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8107e10 0x7f98e8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 -- 172.21.15.42:0/199516133 >> 172.21.15.42:0/199516133 conn(0x7f98e80fe0c0 msgr2=0x7f98e81004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 -- 172.21.15.42:0/199516133 shutdown_connections 2024-05-27T10:25:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.248+0000 7f98ece00700 1 -- 172.21.15.42:0/199516133 wait complete. 2024-05-27T10:25:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.250+0000 7f98ece00700 1 Processor -- start 2024-05-27T10:25:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.250+0000 7f98ece00700 1 -- start start 2024-05-27T10:25:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.250+0000 7f98ece00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8103ff0 0x7f98e819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.250+0000 7f98ece00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e8107e10 0x7f98e819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.250+0000 7f98ece00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98ece00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81121a0 con 0x7f98e8103ff0 2024-05-27T10:25:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98ece00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8112020 con 0x7f98e8108760 2024-05-27T10:25:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98ece00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8112320 con 0x7f98e8107e10 2024-05-27T10:25:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98e6800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98e6800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:45544/0 (socket says 172.21.15.42:45544) 2024-05-27T10:25:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98e6800700 1 -- 172.21.15.42:0/435867854 learned_addr learned my addr 172.21.15.42:0/435867854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.251+0000 7f98e5400700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e8107e10 0x7f98e819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e5e00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8103ff0 0x7f98e819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e6800700 1 -- 172.21.15.42:0/435867854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e8107e10 msgr2=0x7f98e819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e6800700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e8107e10 0x7f98e819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e6800700 1 -- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8103ff0 msgr2=0x7f98e819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e6800700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8103ff0 0x7f98e819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e6800700 1 -- 172.21.15.42:0/435867854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e81a1d70 con 0x7f98e8108760 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.252+0000 7f98e6800700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81a16d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98e000b830 tx=0x7f98e0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.253+0000 7f98dea00700 1 -- 172.21.15.42:0/435867854 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0021070 con 0x7f98e8108760 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.253+0000 7f98dea00700 1 -- 172.21.15.42:0/435867854 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e0007b80 con 0x7f98e8108760 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.253+0000 7f98dea00700 1 -- 172.21.15.42:0/435867854 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00052a0 con 0x7f98e8108760 2024-05-27T10:25:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.253+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e81a2000 con 0x7f98e8108760 2024-05-27T10:25:51.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.253+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e81a24f0 con 0x7f98e8108760 2024-05-27T10:25:51.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.259+0000 7f98fb86db80 1 Processor -- start 2024-05-27T10:25:51.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.259+0000 7f98dea00700 1 -- 172.21.15.42:0/435867854 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f98e0024070 con 0x7f98e8108760 2024-05-27T10:25:51.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 -- start start 2024-05-27T10:25:51.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 0x5641f4b7fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b79fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b7a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5641f4aa7fe0 con 0x5641f49fafe0 2024-05-27T10:25:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5641f4a97b80 con 0x5641f4b7a4e0 2024-05-27T10:25:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.260+0000 7f98fb86db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5641f4a9bbc0 con 0x5641f4b80390 2024-05-27T10:25:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.261+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 0x5641f4b7fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.261+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 0x5641f4b7fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41942/0 (socket says 172.21.15.42:41942) 2024-05-27T10:25:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.261+0000 7f98e5400700 1 -- 172.21.15.42:0/2769843208 learned_addr learned my addr 172.21.15.42:0/2769843208 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.261+0000 7f98e5400700 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b7a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98e5e00700 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b79fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98dea00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f98cc0b2a70 0x7f98cc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98dea00700 1 -- 172.21.15.42:0/435867854 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f98e000f020 con 0x7f98e8108760 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98e5400700 1 -- 172.21.15.42:0/2769843208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 msgr2=0x5641f4b79fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98e5400700 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b79fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98e5400700 1 -- 172.21.15.42:0/2769843208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 msgr2=0x5641f4b7a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98e5400700 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b7a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.262+0000 7f98e5400700 1 -- 172.21.15.42:0/2769843208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f4a605c0 con 0x5641f49fafe0 2024-05-27T10:25:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.263+0000 7f98e6800700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f98cc0b2a70 0x7f98cc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.263+0000 7f98e5400700 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 0x5641f4b7fe50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005c50 tx=0x7f98d400b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e532359d71f6d543 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.263+0000 7f98e6800700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f98cc0b2a70 0x7f98cc0b4f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98e00a6fc0 tx=0x7f98e0009040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.264+0000 7f98d3400700 1 -- 172.21.15.42:0/2769843208 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4007d80 con 0x5641f49fafe0 2024-05-27T10:25:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.264+0000 7f98d3400700 1 -- 172.21.15.42:0/2769843208 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98d40044c0 con 0x5641f49fafe0 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.264+0000 7f98d3400700 1 -- 172.21.15.42:0/2769843208 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003e20 con 0x5641f49fafe0 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 -- 172.21.15.42:0/2769843208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 msgr2=0x5641f4b7fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 0x5641f4b7fe50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005c50 tx=0x7f98d400b3f0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 -- 172.21.15.42:0/2769843208 shutdown_connections 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b7a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f49fafe0 0x5641f4b7fe50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2769843208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b79fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 -- 172.21.15.42:0/2769843208 >> 172.21.15.42:0/2769843208 conn(0x5641f4a5b160 msgr2=0x5641f4b751a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.265+0000 7f98fb86db80 1 -- 172.21.15.42:0/2769843208 shutdown_connections 2024-05-27T10:25:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.266+0000 7f98fb86db80 1 -- 172.21.15.42:0/2769843208 wait complete. 2024-05-27T10:25:51.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.266+0000 7f98fb86db80 1 -- start start 2024-05-27T10:25:51.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.266+0000 7f98fb86db80 1 Processor -- start 2024-05-27T10:25:51.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b7a4e0 0x5641f4b83c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b80390 0x5641f4b84160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b98940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5641f499e140 con 0x5641f4b846a0 2024-05-27T10:25:51.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5641f4a38900 con 0x5641f4b80390 2024-05-27T10:25:51.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5641f4b86e00 con 0x5641f4b7a4e0 2024-05-27T10:25:51.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.267+0000 7f98fb86db80 10 client.0 _ll_register_callbacks cb 0x5641f4a41e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.268+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b98940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.268+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b98940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41958/0 (socket says 172.21.15.42:41958) 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.268+0000 7f98e5400700 1 -- 172.21.15.42:0/2146435358 learned_addr learned my addr 172.21.15.42:0/2146435358 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.268+0000 7f98e5e00700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b80390 0x5641f4b84160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.268+0000 7f98e5400700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b7a4e0 0x5641f4b83c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98e5e00700 1 -- 172.21.15.42:0/2146435358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b7a4e0 msgr2=0x5641f4b83c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98e5e00700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b7a4e0 0x5641f4b83c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98e5e00700 1 -- 172.21.15.42:0/2146435358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 msgr2=0x5641f4b98940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98e5e00700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b98940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98e5e00700 1 -- 172.21.15.42:0/2146435358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f49e7a70 con 0x5641f4b80390 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98e5e00700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b80390 0x5641f4b84160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98d800bf20 tx=0x7f98d8012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ec6cb391fb14c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.269+0000 7f98c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.270+0000 7f98c7e00700 1 -- 172.21.15.42:0/2146435358 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d80107e0 con 0x5641f4b80390 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.270+0000 7f98c7e00700 1 -- 172.21.15.42:0/2146435358 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f98d8010980 con 0x5641f4b80390 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.270+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d40044c0 con 0x5641f4b80390 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.270+0000 7f98c7e00700 1 -- 172.21.15.42:0/2146435358 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98d8010b80 con 0x5641f4b80390 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.270+0000 7f98fb86db80 10 client.36086 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.270+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641f4a68200 con 0x5641f4b80390 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98c7e00700 1 -- 172.21.15.42:0/2146435358 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98d40044c0 con 0x5641f4b80390 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98c7e00700 1 -- 172.21.15.42:0/2146435358 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f98d8019070 con 0x5641f4b80390 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98fb86db80 10 client.36086 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98fb86db80 10 client.36086 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98fb86db80 20 client.36086 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4b9ca40 0x5641f4b9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98fb86db80 4 client.36086 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.271+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f4aa9f30 con 0x5641f4b9ca40 2024-05-27T10:25:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.272+0000 7f98e5400700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4b9ca40 0x5641f4b9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.272+0000 7f98e5400700 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4b9ca40 0x5641f4b9ee80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e7f72b668a16de server_cookie=430f75fecbdaa273 in_seq=0 out_seq=0 2024-05-27T10:25:51.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.272+0000 7f98c7e00700 10 client.36086 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98c7e00700 1 -- 172.21.15.42:0/2146435358 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5641f4aa9f30 con 0x5641f4b9ca40 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98c7e00700 10 client.36086 handle_command_reply: tid=0 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 client.36086 shutdown 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b80390 msgr2=0x5641f4b84160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b80390 0x5641f4b84160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98d800bf20 tx=0x7f98d8012730 comp rx=0 tx=0).stop 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 shutdown_connections 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4b9ca40 0x5641f4b9ee80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b98940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b7a4e0 0x5641f4b83c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2146435358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b80390 0x5641f4b84160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 >> 172.21.15.42:0/2146435358 conn(0x5641f4a5b160 msgr2=0x5641f4b78db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98c7e00700 0 client.36086 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 shutdown_connections 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.275+0000 7f98fb86db80 1 -- 172.21.15.42:0/2146435358 wait complete. 2024-05-27T10:25:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.276+0000 7f98fb86db80 20 client.36086 trim_cache size 0 max 16384 2024-05-27T10:25:51.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.316+0000 7f98fb86db80 1 Processor -- start 2024-05-27T10:25:51.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.316+0000 7f98fb86db80 1 -- start start 2024-05-27T10:25:51.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.316+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.317+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b84070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.317+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b7df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.317+0000 7f98fb86db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81af450 con 0x5641f4b846a0 2024-05-27T10:25:51.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.317+0000 7f98fb86db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81acc80 con 0x5641f4b7a4e0 2024-05-27T10:25:51.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.317+0000 7f98fb86db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f98e819ca80 con 0x5641f4b80390 2024-05-27T10:25:51.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.317+0000 7f98e5400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.318+0000 7f98e5e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b84070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.318+0000 7f98e5e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b84070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:36710/0 (socket says 172.21.15.42:36710) 2024-05-27T10:25:51.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.318+0000 7f98e5e00700 1 -- 172.21.15.42:0/2116528805 learned_addr learned my addr 172.21.15.42:0/2116528805 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:51.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.318+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b7df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.319+0000 7f98e5400700 1 -- 172.21.15.42:0/2116528805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 msgr2=0x5641f4b84070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.319+0000 7f98e5400700 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b84070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.319+0000 7f98e5400700 1 -- 172.21.15.42:0/2116528805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 msgr2=0x5641f4b797e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.319+0000 7f98e5400700 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b797e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.319+0000 7f98e5400700 1 -- 172.21.15.42:0/2116528805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f4b76810 con 0x5641f4b846a0 2024-05-27T10:25:51.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.320+0000 7f98e5400700 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b7df80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x5641f4b9f670 tx=0x7f98d401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ff4578245a2e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:51.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.320+0000 7f98d2a00700 1 -- 172.21.15.42:0/2116528805 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4026040 con 0x5641f4b846a0 2024-05-27T10:25:51.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.321+0000 7f98d2a00700 1 -- 172.21.15.42:0/2116528805 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98d400b370 con 0x5641f4b846a0 2024-05-27T10:25:51.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.321+0000 7f98d2a00700 1 -- 172.21.15.42:0/2116528805 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d402e3e0 con 0x5641f4b846a0 2024-05-27T10:25:51.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 -- 172.21.15.42:0/2116528805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 msgr2=0x5641f4b7df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b7df80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x5641f4b9f670 tx=0x7f98d401e040 comp rx=0 tx=0).stop 2024-05-27T10:25:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 -- 172.21.15.42:0/2116528805 shutdown_connections 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b7df80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b797e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 --2- 172.21.15.42:0/2116528805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b84070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 -- 172.21.15.42:0/2116528805 >> 172.21.15.42:0/2116528805 conn(0x5641f4a5b160 msgr2=0x5641f4b7d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 -- 172.21.15.42:0/2116528805 shutdown_connections 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.322+0000 7f98fb86db80 1 -- 172.21.15.42:0/2116528805 wait complete. 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.323+0000 7f98fb86db80 1 -- start start 2024-05-27T10:25:51.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.323+0000 7f98fb86db80 1 Processor -- start 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b94920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81ab140 con 0x5641f4b846a0 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81ac5b0 con 0x5641f4b7a4e0 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81ab810 con 0x5641f4b80390 2024-05-27T10:25:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.324+0000 7f98fb86db80 10 client.0 _ll_register_callbacks cb 0x5641f4a41e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:25:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.325+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.325+0000 7f98e5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b918f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41984/0 (socket says 172.21.15.42:41984) 2024-05-27T10:25:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.325+0000 7f98e5400700 1 -- 172.21.15.42:0/1796683089 learned_addr learned my addr 172.21.15.42:0/1796683089 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.325+0000 7f98e5400700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b94920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.325+0000 7f98e5e00700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.326+0000 7f98e5400700 1 -- 172.21.15.42:0/1796683089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 msgr2=0x5641f4b94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.326+0000 7f98e5400700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b94e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.326+0000 7f98e5400700 1 -- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 msgr2=0x5641f4b94920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.326+0000 7f98e5400700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b94920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.326+0000 7f98e5400700 1 -- 172.21.15.42:0/1796683089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f4b91e30 con 0x5641f4b846a0 2024-05-27T10:25:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.327+0000 7f98e5400700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b918f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x5641f4b980f0 tx=0x7f98d4005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b47acd0d027a785 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.327+0000 7f98c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:25:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.328+0000 7f98c7e00700 1 -- 172.21.15.42:0/1796683089 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40053e0 con 0x5641f4b846a0 2024-05-27T10:25:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.328+0000 7f98c7e00700 1 -- 172.21.15.42:0/1796683089 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7f98d4005580 con 0x5641f4b846a0 2024-05-27T10:25:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.328+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d400b370 con 0x5641f4b846a0 2024-05-27T10:25:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.328+0000 7f98c7e00700 1 -- 172.21.15.42:0/1796683089 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98d4003760 con 0x5641f4b846a0 2024-05-27T10:25:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.328+0000 7f98fb86db80 10 client.16563 fetch_fsmap learned FSMap version 33 2024-05-27T10:25:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.328+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641f4c07cd0 con 0x5641f4b846a0 2024-05-27T10:25:51.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.329+0000 7f98c7e00700 1 -- 172.21.15.42:0/1796683089 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98d400b370 con 0x5641f4b846a0 2024-05-27T10:25:51.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.329+0000 7f98c7e00700 1 -- 172.21.15.42:0/1796683089 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f98d4047050 con 0x5641f4b846a0 2024-05-27T10:25:51.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.330+0000 7f98fb86db80 10 client.16563 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:25:51.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.330+0000 7f98fb86db80 10 client.16563 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:25:51.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.330+0000 7f98fb86db80 20 client.16563 populate_metadata read hostname 'smithi042' 2024-05-27T10:25:51.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.330+0000 7f98fb86db80 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4c08190 0x5641f4c0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:51.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.330+0000 7f98fb86db80 4 client.16563 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.95, "await_for": 45.0}] 2024-05-27T10:25:51.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.330+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.95, "await_for": 45.0}) -- 0x5641f4aa3710 con 0x5641f4c08190 2024-05-27T10:25:51.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.331+0000 7f98e5e00700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4c08190 0x5641f4c0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.333+0000 7f98e5e00700 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4c08190 0x5641f4c0a5b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857f7e1854630f6a server_cookie=786aaf92e6b0bd41 in_seq=0 out_seq=0 2024-05-27T10:25:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:51.333+0000 7f98c7e00700 10 client.16563 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:52.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:52 smithi104 ceph-mon[34812]: pgmap v217: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-05-27T10:25:53.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:52 smithi143 ceph-mon[35545]: pgmap v217: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-05-27T10:25:53.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:52 smithi042 ceph-mon[31661]: pgmap v217: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-05-27T10:25:54.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:54 smithi104 ceph-mon[34812]: pgmap v218: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-05-27T10:25:55.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:54 smithi143 ceph-mon[35545]: pgmap v218: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-05-27T10:25:55.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:54 smithi042 ceph-mon[31661]: pgmap v218: 97 pgs: 97 active+clean; 330 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-05-27T10:25:56.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:56 smithi104 ceph-mon[34812]: pgmap v219: 97 pgs: 97 active+clean; 338 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2024-05-27T10:25:57.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:56 smithi143 ceph-mon[35545]: pgmap v219: 97 pgs: 97 active+clean; 338 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2024-05-27T10:25:57.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:56 smithi042 ceph-mon[31661]: pgmap v219: 97 pgs: 97 active+clean; 338 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2024-05-27T10:25:57.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.256+0000 7f98c7e00700 1 -- 172.21.15.42:0/1796683089 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5641f4aa3710 con 0x5641f4c08190 2024-05-27T10:25:57.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.256+0000 7f98c7e00700 10 client.16563 handle_command_reply: tid=0 2024-05-27T10:25:57.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.256+0000 7f98fb86db80 1 client.16563 shutdown 2024-05-27T10:25:57.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 msgr2=0x5641f4b918f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:57.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b918f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x5641f4b980f0 tx=0x7f98d4005f60 comp rx=0 tx=0).stop 2024-05-27T10:25:57.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 shutdown_connections 2024-05-27T10:25:57.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5641f4c08190 0x5641f4c0a5b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5641f4b846a0 0x5641f4b918f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641f4b7a4e0 0x5641f4b94920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 --2- 172.21.15.42:0/1796683089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5641f4b80390 0x5641f4b94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 >> 172.21.15.42:0/1796683089 conn(0x5641f4a5b160 msgr2=0x5641f4b79540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:57.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.257+0000 7f98c7e00700 0 client.16563 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:25:57.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.258+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 shutdown_connections 2024-05-27T10:25:57.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.258+0000 7f98fb86db80 1 -- 172.21.15.42:0/1796683089 wait complete. 2024-05-27T10:25:57.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.258+0000 7f98fb86db80 20 client.16563 trim_cache size 0 max 16384 2024-05-27T10:25:57.259 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:25:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 33, 2024-05-27T10:25:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:25:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 18, 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "c5b78a64": { 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 18, 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 138.9, 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:25:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:25:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:25:57.262 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:25:57.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f98cc0b2a70 msgr2=0x7f98cc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:57.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f98cc0b2a70 0x7f98cc0b4f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98e00a6fc0 tx=0x7f98e0009040 comp rx=0 tx=0).stop 2024-05-27T10:25:57.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 msgr2=0x7f98e81a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:57.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81a16d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98e000b830 tx=0x7f98e0004d00 comp rx=0 tx=0).stop 2024-05-27T10:25:57.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 shutdown_connections 2024-05-27T10:25:57.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f98cc0b2a70 0x7f98cc0b4f80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98e8108760 0x7f98e81a16d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8103ff0 0x7f98e819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 --2- 172.21.15.42:0/435867854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e8107e10 0x7f98e819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:57.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.260+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 >> 172.21.15.42:0/435867854 conn(0x7f98e80fe0c0 msgr2=0x7f98e81004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:57.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.261+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 shutdown_connections 2024-05-27T10:25:57.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:25:57.261+0000 7f98ece00700 1 -- 172.21.15.42:0/435867854 wait complete. 2024-05-27T10:25:58.319 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c5b78a64', quiesce took 15.5 seconds. Will release after: 3.4499999999999993 2024-05-27T10:25:58.716 INFO:tasks.workunit.client.0.smithi042.stdout: 44 2803 48 2100 46 847 96 2024-05-27T10:25:58.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:25:58 smithi104 ceph-mon[34812]: pgmap v220: 97 pgs: 97 active+clean; 338 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-05-27T10:25:59.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:25:58 smithi143 ceph-mon[35545]: pgmap v220: 97 pgs: 97 active+clean; 338 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-05-27T10:25:59.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:25:58 smithi042 ceph-mon[31661]: pgmap v220: 97 pgs: 97 active+clean; 338 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 -- 172.21.15.42:0/1611689719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 msgr2=0x7fb8d810ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1611689719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d810ffc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009860 tx=0x7fb8c402a3d0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 -- 172.21.15.42:0/1611689719 shutdown_connections 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1611689719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8110500 0x7fb8d8112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1611689719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d810ffc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1611689719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8076510 0x7fb8d8076920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 -- 172.21.15.42:0/1611689719 >> 172.21.15.42:0/1611689719 conn(0x7fb8d80ff330 msgr2=0x7fb8d8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 -- 172.21.15.42:0/1611689719 shutdown_connections 2024-05-27T10:25:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.985+0000 7fb8dd600640 1 -- 172.21.15.42:0/1611689719 wait complete. 2024-05-27T10:25:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.986+0000 7fb8dd600640 1 Processor -- start 2024-05-27T10:25:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 -- start start 2024-05-27T10:25:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8076510 0x7fb8d81071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d8107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8110500 0x7fb8d810ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8115f40 con 0x7fb8d8110500 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8115dc0 con 0x7fb8d810dba0 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.987+0000 7fb8dd600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81160c0 con 0x7fb8d8076510 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.988+0000 7fb8d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8110500 0x7fb8d810ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.988+0000 7fb8d6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d8107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.988+0000 7fb8d6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8076510 0x7fb8d81071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.988+0000 7fb8d6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d8107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38810/0 (socket says 172.21.15.42:38810) 2024-05-27T10:25:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.988+0000 7fb8d6000640 1 -- 172.21.15.42:0/1915999775 learned_addr learned my addr 172.21.15.42:0/1915999775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.988+0000 7fb8d6000640 1 -- 172.21.15.42:0/1915999775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8076510 msgr2=0x7fb8d81071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d6000640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8076510 0x7fb8d81071c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d6000640 1 -- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8110500 msgr2=0x7fb8d810ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d6000640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8110500 0x7fb8d810ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d6000640 1 -- 172.21.15.42:0/1915999775 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d810d6d0 con 0x7fb8d810dba0 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8076510 0x7fb8d81071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d7400640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8110500 0x7fb8d810ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:25:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.989+0000 7fb8d6000640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d8107720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c00 tx=0x7fb8c40319a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.990+0000 7fb8cf400640 1 -- 172.21.15.42:0/1915999775 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4004ad0 con 0x7fb8d810dba0 2024-05-27T10:25:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.990+0000 7fb8cf400640 1 -- 172.21.15.42:0/1915999775 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8c4005630 con 0x7fb8d810dba0 2024-05-27T10:25:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.990+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d810d960 con 0x7fb8d810dba0 2024-05-27T10:25:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.990+0000 7fb8cf400640 1 -- 172.21.15.42:0/1915999775 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4044400 con 0x7fb8d810dba0 2024-05-27T10:25:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.990+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81a2b40 con 0x7fb8d810dba0 2024-05-27T10:25:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.994+0000 7fb8e4614740 1 Processor -- start 2024-05-27T10:25:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.995+0000 7fb8e4614740 1 -- start start 2024-05-27T10:25:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8e4614740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e990023b90 con 0x55e98fd101b0 2024-05-27T10:25:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8e4614740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e990013730 con 0x55e98fcef4d0 2024-05-27T10:25:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8e4614740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e990017770 con 0x55e98fce72f0 2024-05-27T10:25:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38236/0 (socket says 172.21.15.42:38236) 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8d7400640 1 -- 172.21.15.42:0/1627919969 learned_addr learned my addr 172.21.15.42:0/1627919969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.996+0000 7fb8cf400640 1 -- 172.21.15.42:0/1915999775 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb8c4039040 con 0x7fb8d810dba0 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.997+0000 7fb8d7400640 1 -- 172.21.15.42:0/1627919969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 msgr2=0x55e9900b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.997+0000 7fb8d7400640 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.997+0000 7fb8d7400640 1 -- 172.21.15.42:0/1627919969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 msgr2=0x55e9900b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.997+0000 7fb8d7400640 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:25:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.997+0000 7fb8d7400640 1 -- 172.21.15.42:0/1627919969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98ffac8b0 con 0x55e98fd101b0 2024-05-27T10:25:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.997+0000 7fb8d7400640 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900b2860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0011760 tx=0x7fb8d0011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab61d90e9c61bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:25:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.998+0000 7fb8c3400640 1 -- 172.21.15.42:0/1627919969 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004d20 con 0x55e98fd101b0 2024-05-27T10:25:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.998+0000 7fb8c3400640 1 -- 172.21.15.42:0/1627919969 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d0005020 con 0x55e98fd101b0 2024-05-27T10:25:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.998+0000 7fb8c3400640 1 -- 172.21.15.42:0/1627919969 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0019670 con 0x55e98fd101b0 2024-05-27T10:25:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.999+0000 7fb8cf400640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb8bc0b2950 0x7fb8bc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:25:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.999+0000 7fb8cf400640 1 -- 172.21.15.42:0/1915999775 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb8c41035b0 con 0x7fb8d810dba0 2024-05-27T10:25:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.999+0000 7fb8e4614740 1 -- 172.21.15.42:0/1627919969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 msgr2=0x55e9900b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:25:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:25:59.999+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900b2860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0011760 tx=0x7fb8d0011c20 comp rx=0 tx=0).stop 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb8bc0b2950 0x7fb8bc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 -- 172.21.15.42:0/1627919969 shutdown_connections 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1627919969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900b2860 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 -- 172.21.15.42:0/1627919969 >> 172.21.15.42:0/1627919969 conn(0x55e98fd04300 msgr2=0x55e9900af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 -- 172.21.15.42:0/1627919969 shutdown_connections 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.000+0000 7fb8e4614740 1 -- 172.21.15.42:0/1627919969 wait complete. 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.001+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb8bc0b2950 0x7fb8bc0b4e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55e9900b3c10 tx=0x7fb8c80046c0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.001+0000 7fb8e4614740 1 -- start start 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.001+0000 7fb8e4614740 1 Processor -- start 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fce72f0 0x55e9900bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fd101b0 0x55e9900c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e98fcf9790 con 0x55e98fce72f0 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e98ffbbbc0 con 0x55e98fcef4d0 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e9900bf990 con 0x55e98fd101b0 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.002+0000 7fb8e4614740 10 client.0 _ll_register_callbacks cb 0x55e98ff99400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fce72f0 0x55e9900bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900bcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38834/0 (socket says 172.21.15.42:38834) 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 -- 172.21.15.42:0/3043322651 learned_addr learned my addr 172.21.15.42:0/3043322651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d7400640 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fd101b0 0x55e9900c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 -- 172.21.15.42:0/3043322651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fd101b0 msgr2=0x55e9900c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fd101b0 0x55e9900c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 -- 172.21.15.42:0/3043322651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fce72f0 msgr2=0x55e9900bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fce72f0 0x55e9900bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.003+0000 7fb8d6000640 1 -- 172.21.15.42:0/3043322651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98fd28210 con 0x55e98fcef4d0 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.004+0000 7fb8d6000640 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900bcf60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40c7af0 tx=0x7fb8c4101260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f4b883d328f4938 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.004+0000 7fb8b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.004+0000 7fb8b7e00640 1 -- 172.21.15.42:0/3043322651 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40caf80 con 0x55e98fcef4d0 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.004+0000 7fb8b7e00640 1 -- 172.21.15.42:0/3043322651 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb8c4004c70 con 0x55e98fcef4d0 2024-05-27T10:26:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.004+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d0005020 con 0x55e98fcef4d0 2024-05-27T10:26:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.005+0000 7fb8b7e00640 1 -- 172.21.15.42:0/3043322651 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb8c40ca880 con 0x55e98fcef4d0 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.005+0000 7fb8e4614740 10 client.36101 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.005+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e98fdf4620 con 0x55e98fcef4d0 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.005+0000 7fb8b7e00640 1 -- 172.21.15.42:0/3043322651 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d0005020 con 0x55e98fcef4d0 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8b7e00640 1 -- 172.21.15.42:0/3043322651 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c40d3250 con 0x55e98fcef4d0 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8e4614740 10 client.36101 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8e4614740 10 client.36101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8e4614740 20 client.36101 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8e4614740 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fcced80 0x55e9900cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8e4614740 4 client.36101 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.006+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e990025ae0 con 0x55e98fcced80 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.007+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fcced80 0x55e9900cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.008+0000 7fb8b7e00640 10 client.36101 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.008+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fcced80 0x55e9900cec40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e247927e09f621 server_cookie=4fdd5b965bdbd586 in_seq=0 out_seq=0 2024-05-27T10:26:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.009+0000 7fb8b7e00640 1 -- 172.21.15.42:0/3043322651 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e990025ae0 con 0x55e98fcced80 2024-05-27T10:26:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.009+0000 7fb8b7e00640 10 client.36101 handle_command_reply: tid=0 2024-05-27T10:26:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 client.36101 shutdown 2024-05-27T10:26:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 msgr2=0x55e9900bcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900bcf60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40c7af0 tx=0x7fb8c4101260 comp rx=0 tx=0).stop 2024-05-27T10:26:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 shutdown_connections 2024-05-27T10:26:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fcced80 0x55e9900cec40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fd101b0 0x55e9900c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900bcf60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 --2- 172.21.15.42:0/3043322651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fce72f0 0x55e9900bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 >> 172.21.15.42:0/3043322651 conn(0x55e98fd04300 msgr2=0x55e9900b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8b7e00640 0 client.36101 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 shutdown_connections 2024-05-27T10:26:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.010+0000 7fb8e4614740 1 -- 172.21.15.42:0/3043322651 wait complete. 2024-05-27T10:26:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.011+0000 7fb8e4614740 20 client.36101 trim_cache size 0 max 16384 2024-05-27T10:26:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.037+0000 7fb8e4614740 1 Processor -- start 2024-05-27T10:26:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 -- start start 2024-05-27T10:26:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81c1040 con 0x55e98fd101b0 2024-05-27T10:26:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81be8b0 con 0x55e98fcef4d0 2024-05-27T10:26:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.038+0000 7fb8e4614740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81bc700 con 0x55e98fce72f0 2024-05-27T10:26:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38848/0 (socket says 172.21.15.42:38848) 2024-05-27T10:26:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 -- 172.21.15.42:0/1724157528 learned_addr learned my addr 172.21.15.42:0/1724157528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d7400640 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 -- 172.21.15.42:0/1724157528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 msgr2=0x55e9900b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 -- 172.21.15.42:0/1724157528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 msgr2=0x55e9900c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.039+0000 7fb8d6000640 1 -- 172.21.15.42:0/1724157528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9900b00d0 con 0x55e98fcef4d0 2024-05-27T10:26:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.040+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.040+0000 7fb8d6000640 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900c8ab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402a990 tx=0x7fb8c403cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7570da62b2618a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.040+0000 7fb8c2a00640 1 -- 172.21.15.42:0/1724157528 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40e6070 con 0x55e98fcef4d0 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.040+0000 7fb8c2a00640 1 -- 172.21.15.42:0/1724157528 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8c40caf80 con 0x55e98fcef4d0 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.040+0000 7fb8c2a00640 1 -- 172.21.15.42:0/1724157528 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4090020 con 0x55e98fcef4d0 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 -- 172.21.15.42:0/1724157528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 msgr2=0x55e9900c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900c8ab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402a990 tx=0x7fb8c403cf90 comp rx=0 tx=0).stop 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 -- 172.21.15.42:0/1724157528 shutdown_connections 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9900c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fcef4d0 0x55e9900c8ab0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 --2- 172.21.15.42:0/1724157528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fce72f0 0x55e9900b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 -- 172.21.15.42:0/1724157528 >> 172.21.15.42:0/1724157528 conn(0x55e98fd04300 msgr2=0x55e9900bcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 -- 172.21.15.42:0/1724157528 shutdown_connections 2024-05-27T10:26:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.041+0000 7fb8e4614740 1 -- 172.21.15.42:0/1724157528 wait complete. 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.042+0000 7fb8e4614740 1 -- start start 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.042+0000 7fb8e4614740 1 Processor -- start 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 0x55e9900c0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fcef4d0 0x55e9900c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9901352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81bcd70 con 0x55e98fd101b0 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81be1e0 con 0x55e98fce72f0 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81bd440 con 0x55e98fcef4d0 2024-05-27T10:26:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8e4614740 10 client.0 _ll_register_callbacks cb 0x7fb8c4004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8d6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 0x55e9900c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8d6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 0x55e9900c0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38852/0 (socket says 172.21.15.42:38852) 2024-05-27T10:26:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8d6a00640 1 -- 172.21.15.42:0/2126911464 learned_addr learned my addr 172.21.15.42:0/2126911464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.043+0000 7fb8d7400640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9901352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d6000640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fcef4d0 0x55e9900c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d6a00640 1 -- 172.21.15.42:0/2126911464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fcef4d0 msgr2=0x55e9900c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fcef4d0 0x55e9900c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d6a00640 1 -- 172.21.15.42:0/2126911464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 msgr2=0x55e9901352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9901352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d6a00640 1 -- 172.21.15.42:0/2126911464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e990135800 con 0x55e98fce72f0 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.044+0000 7fb8d7400640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9901352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.045+0000 7fb8d6a00640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 0x55e9900c0280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55e9900b0350 tx=0x7fb8c8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b221c476454e31aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.045+0000 7fb8b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:26:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.045+0000 7fb8b7e00640 1 -- 172.21.15.42:0/2126911464 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c802b070 con 0x55e98fce72f0 2024-05-27T10:26:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.045+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c40caf80 con 0x55e98fce72f0 2024-05-27T10:26:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.045+0000 7fb8b7e00640 1 -- 172.21.15.42:0/2126911464 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7fb8c8009040 con 0x55e98fce72f0 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.046+0000 7fb8b7e00640 1 -- 172.21.15.42:0/2126911464 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb8c8037050 con 0x55e98fce72f0 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.046+0000 7fb8e4614740 10 client.36107 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.046+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e990136b10 con 0x55e98fce72f0 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.046+0000 7fb8b7e00640 1 -- 172.21.15.42:0/2126911464 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8c40caf80 con 0x55e98fce72f0 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8b7e00640 1 -- 172.21.15.42:0/2126911464 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c8027680 con 0x55e98fce72f0 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8e4614740 10 client.36107 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8e4614740 10 client.36107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8e4614740 20 client.36107 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8e4614740 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fca41c0 0x55e9901392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8e4614740 4 client.36107 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:26:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e99001f2c0 con 0x55e98fca41c0 2024-05-27T10:26:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.047+0000 7fb8d6000640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fca41c0 0x55e9901392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.048+0000 7fb8d6000640 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fca41c0 0x55e9901392b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b0fbc23ffebda8 server_cookie=ff9fa4cf157faad3 in_seq=0 out_seq=0 2024-05-27T10:26:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.048+0000 7fb8b7e00640 10 client.36107 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.083+0000 7fb8b7e00640 1 -- 172.21.15.42:0/2126911464 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e99001f2c0 con 0x55e98fca41c0 2024-05-27T10:26:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.083+0000 7fb8b7e00640 10 client.36107 handle_command_reply: tid=0 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.083+0000 7fb8e4614740 1 client.36107 shutdown 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 msgr2=0x55e9900c0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 0x55e9900c0280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55e9900b0350 tx=0x7fb8c8013040 comp rx=0 tx=0).stop 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 shutdown_connections 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e98fd101b0 0x55e9901352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e98fca41c0 0x55e9901392b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e98fcef4d0 0x55e9900c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8b7e00640 0 client.36107 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 --2- 172.21.15.42:0/2126911464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e98fce72f0 0x55e9900c0280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 >> 172.21.15.42:0/2126911464 conn(0x55e98fd04300 msgr2=0x55e9900bdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 shutdown_connections 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 1 -- 172.21.15.42:0/2126911464 wait complete. 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.084+0000 7fb8e4614740 20 client.36107 trim_cache size 0 max 16384 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:26:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb8bc0b2950 msgr2=0x7fb8bc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb8bc0b2950 0x7fb8bc0b4e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55e9900b3c10 tx=0x7fb8c80046c0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 msgr2=0x7fb8d8107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d8107720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c00 tx=0x7fb8c40319a0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 shutdown_connections 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb8bc0b2950 0x7fb8bc0b4e60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8d8110500 0x7fb8d810ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8d810dba0 0x7fb8d8107720 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.085+0000 7fb8dd600640 1 --2- 172.21.15.42:0/1915999775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8d8076510 0x7fb8d81071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.086+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 >> 172.21.15.42:0/1915999775 conn(0x7fb8d80ff330 msgr2=0x7fb8d8110910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.086+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 shutdown_connections 2024-05-27T10:26:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:00.086+0000 7fb8dd600640 1 -- 172.21.15.42:0/1915999775 wait complete. 2024-05-27T10:26:00.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:00 smithi104 ceph-mon[34812]: pgmap v221: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 446 op/s 2024-05-27T10:26:01.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:00 smithi143 ceph-mon[35545]: pgmap v221: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 446 op/s 2024-05-27T10:26:01.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:00 smithi042 ceph-mon[31661]: pgmap v221: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 446 op/s 2024-05-27T10:26:01.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:26:02.362 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:26:02.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:02 smithi042 ceph-mon[31661]: pgmap v222: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.0 MiB/s wr, 238 op/s 2024-05-27T10:26:02.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:02 smithi104 ceph-mon[34812]: pgmap v222: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.0 MiB/s wr, 238 op/s 2024-05-27T10:26:03.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:02 smithi143 ceph-mon[35545]: pgmap v222: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.0 MiB/s wr, 238 op/s 2024-05-27T10:26:04.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:04 smithi104 ceph-mon[34812]: pgmap v223: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.0 MiB/s wr, 238 op/s 2024-05-27T10:26:05.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:04 smithi143 ceph-mon[35545]: pgmap v223: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.0 MiB/s wr, 238 op/s 2024-05-27T10:26:05.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:04 smithi042 ceph-mon[31661]: pgmap v223: 97 pgs: 97 active+clean; 320 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.0 MiB/s wr, 238 op/s 2024-05-27T10:26:07.006 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:06 smithi042 ceph-mon[31661]: pgmap v224: 97 pgs: 97 active+clean; 327 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 332 op/s 2024-05-27T10:26:07.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:06 smithi143 ceph-mon[35545]: pgmap v224: 97 pgs: 97 active+clean; 327 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 332 op/s 2024-05-27T10:26:07.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:06 smithi104 ceph-mon[34812]: pgmap v224: 97 pgs: 97 active+clean; 327 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 332 op/s 2024-05-27T10:26:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 -- 172.21.15.42:0/69293670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 msgr2=0x7ff6141049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 --2- 172.21.15.42:0/69293670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff6141049f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff6040099c0 tx=0x7ff604009cd0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 -- 172.21.15.42:0/69293670 shutdown_connections 2024-05-27T10:26:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 --2- 172.21.15.42:0/69293670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff614109860 0x7ff6141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 --2- 172.21.15.42:0/69293670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff6141049f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 --2- 172.21.15.42:0/69293670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff614108f10 0x7ff614109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 -- 172.21.15.42:0/69293670 >> 172.21.15.42:0/69293670 conn(0x7ff614100030 msgr2=0x7ff614102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 -- 172.21.15.42:0/69293670 shutdown_connections 2024-05-27T10:26:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.662+0000 7ff619000700 1 -- 172.21.15.42:0/69293670 wait complete. 2024-05-27T10:26:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 Processor -- start 2024-05-27T10:26:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 -- start start 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff61419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff614108f10 0x7ff61419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff614109860 0x7ff6141a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff614114340 con 0x7ff6141045e0 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141141c0 con 0x7ff614108f10 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.664+0000 7ff619000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141144c0 con 0x7ff614109860 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff61419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff61419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35906/0 (socket says 172.21.15.42:35906) 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 -- 172.21.15.42:0/2819204468 learned_addr learned my addr 172.21.15.42:0/2819204468 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff609400700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff614108f10 0x7ff61419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff612800700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff614109860 0x7ff6141a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 -- 172.21.15.42:0/2819204468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff614109860 msgr2=0x7ff6141a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff614109860 0x7ff6141a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 -- 172.21.15.42:0/2819204468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff614108f10 msgr2=0x7ff61419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff614108f10 0x7ff61419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.665+0000 7ff611e00700 1 -- 172.21.15.42:0/2819204468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6141a1ea0 con 0x7ff6141045e0 2024-05-27T10:26:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.666+0000 7ff611e00700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff61419cfd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff60400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.666+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60400f940 con 0x7ff6141045e0 2024-05-27T10:26:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.666+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff604004830 con 0x7ff6141045e0 2024-05-27T10:26:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.666+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6141a2130 con 0x7ff6141045e0 2024-05-27T10:26:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.666+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6040054b0 con 0x7ff6141045e0 2024-05-27T10:26:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.666+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6141a2600 con 0x7ff6141045e0 2024-05-27T10:26:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.668+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6140512b0 con 0x7ff6141045e0 2024-05-27T10:26:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.670+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff604029060 con 0x7ff6141045e0 2024-05-27T10:26:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.671+0000 7ff60b400700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff5f80b2a70 0x7ff5f80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.672+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8440+0+0 (secure 0 0 0) 0x7ff6040e30b0 con 0x7ff6141045e0 2024-05-27T10:26:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.672+0000 7ff609400700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff5f80b2a70 0x7ff5f80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.673+0000 7ff609400700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff5f80b2a70 0x7ff5f80b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc005950 tx=0x7ff5fc0058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:07.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.678+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff604013070 con 0x7ff6141045e0 2024-05-27T10:26:07.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.967+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6141a9b70 con 0x7ff6141045e0 2024-05-27T10:26:07.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.968+0000 7ff60b400700 1 -- 172.21.15.42:0/2819204468 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11881 (secure 0 0 0) 0x7ff6040aab70 con 0x7ff6141045e0 2024-05-27T10:26:07.967 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:26:07.968 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:24:00.498568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:26:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.972+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff5f80b2a70 msgr2=0x7ff5f80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.972+0000 7ff619000700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff5f80b2a70 0x7ff5f80b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc005950 tx=0x7ff5fc0058e0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.972+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 msgr2=0x7ff61419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.972+0000 7ff619000700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff61419cfd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff60400f510 comp rx=0 tx=0).stop 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 shutdown_connections 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff5f80b2a70 0x7ff5f80b4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff614109860 0x7ff6141a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6141045e0 0x7ff61419cfd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 --2- 172.21.15.42:0/2819204468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff614108f10 0x7ff61419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 >> 172.21.15.42:0/2819204468 conn(0x7ff614100030 msgr2=0x7ff61410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 shutdown_connections 2024-05-27T10:26:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:07.973+0000 7ff619000700 1 -- 172.21.15.42:0/2819204468 wait complete. 2024-05-27T10:26:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 33 2024-05-27T10:26:08.717 INFO:tasks.workunit.client.0.smithi042.stdout: 44 4542 0 4072 0 1638 0 2024-05-27T10:26:09.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:08 smithi042 ceph-mon[31661]: pgmap v225: 97 pgs: 97 active+clean; 327 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 855 KiB/s wr, 182 op/s 2024-05-27T10:26:09.014 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:08 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2819204468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:09.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:08 smithi143 ceph-mon[35545]: pgmap v225: 97 pgs: 97 active+clean; 327 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 855 KiB/s wr, 182 op/s 2024-05-27T10:26:09.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:08 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2819204468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:09.087 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id c5b78a64 --release --await' 2024-05-27T10:26:09.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id c5b78a64 --release --await 2024-05-27T10:26:09.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:08 smithi104 ceph-mon[34812]: pgmap v225: 97 pgs: 97 active+clean; 327 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 855 KiB/s wr, 182 op/s 2024-05-27T10:26:09.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:08 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2819204468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:09.564 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:26:11.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:10 smithi143 ceph-mon[35545]: pgmap v226: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 212 op/s 2024-05-27T10:26:11.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:10 smithi042 ceph-mon[31661]: pgmap v226: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 212 op/s 2024-05-27T10:26:11.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:10 smithi104 ceph-mon[34812]: pgmap v226: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 212 op/s 2024-05-27T10:26:11.948 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:11 smithi042 ceph-mon[31661]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T10:26:12.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:11 smithi143 ceph-mon[35545]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T10:26:12.102 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:11 smithi104 ceph-mon[34812]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T10:26:12.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 -- 172.21.15.42:0/536227748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c8108f00 msgr2=0x7f27c8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 --2- 172.21.15.42:0/536227748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c8108f00 0x7f27c8109310 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f27b40099c0 tx=0x7f27b4009cd0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 -- 172.21.15.42:0/536227748 shutdown_connections 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 --2- 172.21.15.42:0/536227748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c8109850 0x7f27c81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 --2- 172.21.15.42:0/536227748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27c8104620 0x7f27c8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 --2- 172.21.15.42:0/536227748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c8108f00 0x7f27c8109310 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 -- 172.21.15.42:0/536227748 >> 172.21.15.42:0/536227748 conn(0x7f27c8100090 msgr2=0x7f27c81024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 -- 172.21.15.42:0/536227748 shutdown_connections 2024-05-27T10:26:12.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.448+0000 7f27cfa00700 1 -- 172.21.15.42:0/536227748 wait complete. 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 Processor -- start 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 -- start start 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27c8104620 0x7f27c819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c8109850 0x7f27c819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 0x7f27c81a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8114310 con 0x7f27c81a1870 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8114190 con 0x7f27c8104620 2024-05-27T10:26:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.450+0000 7f27cfa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8114490 con 0x7f27c8109850 2024-05-27T10:26:12.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 0x7f27c81a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 0x7f27c81a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35926/0 (socket says 172.21.15.42:35926) 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 -- 172.21.15.42:0/1847210417 learned_addr learned my addr 172.21.15.42:0/1847210417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cd000700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27c8104620 0x7f27c819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27c7e00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c8109850 0x7f27c819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 -- 172.21.15.42:0/1847210417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c8109850 msgr2=0x7f27c819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c8109850 0x7f27c819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 -- 172.21.15.42:0/1847210417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27c8104620 msgr2=0x7f27c819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27c8104620 0x7f27c819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.451+0000 7f27cda00700 1 -- 172.21.15.42:0/1847210417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c81a23c0 con 0x7f27c81a1870 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.452+0000 7f27cda00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 0x7f27c81a1c80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f27c000ef30 tx=0x7f27c000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.452+0000 7f27c5600700 1 -- 172.21.15.42:0/1847210417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c00092e0 con 0x7f27c81a1870 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.452+0000 7f27c5600700 1 -- 172.21.15.42:0/1847210417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27c0004bb0 con 0x7f27c81a1870 2024-05-27T10:26:12.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.452+0000 7f27c5600700 1 -- 172.21.15.42:0/1847210417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004080 con 0x7f27c81a1870 2024-05-27T10:26:12.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.452+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c81a2650 con 0x7f27c81a1870 2024-05-27T10:26:12.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.452+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81a9db0 con 0x7f27c81a1870 2024-05-27T10:26:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.456+0000 7f27c5600700 1 -- 172.21.15.42:0/1847210417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f27c001b030 con 0x7f27c81a1870 2024-05-27T10:26:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.458+0000 7f27de592b80 1 Processor -- start 2024-05-27T10:26:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.458+0000 7f27de592b80 1 -- start start 2024-05-27T10:26:12.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.458+0000 7f27de592b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45093f40 0x55ee45218db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.458+0000 7f27de592b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee45212f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27de592b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 0x55ee45213850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27de592b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ee45140f40 con 0x55ee45213440 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27de592b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ee45130ae0 con 0x55ee45093f40 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27de592b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ee45134b20 con 0x55ee452192f0 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27c5600700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27b00b29f0 0x7f27b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 0x55ee45213850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 0x55ee45213850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35938/0 (socket says 172.21.15.42:35938) 2024-05-27T10:26:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27c5600700 1 -- 172.21.15.42:0/1847210417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f27c00e2200 con 0x7f27c81a1870 2024-05-27T10:26:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27c7e00700 1 -- 172.21.15.42:0/2538084606 learned_addr learned my addr 172.21.15.42:0/2538084606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27cd000700 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee45212f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27cda00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27b00b29f0 0x7f27b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.459+0000 7f27c7e00700 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45093f40 0x55ee45218db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cd000700 1 -- 172.21.15.42:0/2538084606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45093f40 msgr2=0x55ee45218db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cd000700 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45093f40 0x55ee45218db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cd000700 1 -- 172.21.15.42:0/2538084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 msgr2=0x55ee45213850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cd000700 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 0x55ee45213850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cd000700 1 -- 172.21.15.42:0/2538084606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee450f9520 con 0x55ee452192f0 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27c7e00700 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 0x55ee45213850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cda00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27b00b29f0 0x7f27b00b4f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27c000ef00 tx=0x7f27c0009040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.460+0000 7f27cd000700 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee45212f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00bc70 tx=0x7f27bc00bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba3f6837c8ab946c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.461+0000 7f27ba000700 1 -- 172.21.15.42:0/2538084606 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc015070 con 0x55ee452192f0 2024-05-27T10:26:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.461+0000 7f27ba000700 1 -- 172.21.15.42:0/2538084606 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27bc01a460 con 0x55ee452192f0 2024-05-27T10:26:12.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.461+0000 7f27ba000700 1 -- 172.21.15.42:0/2538084606 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0116a0 con 0x55ee452192f0 2024-05-27T10:26:12.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 -- 172.21.15.42:0/2538084606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 msgr2=0x55ee45212f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee45212f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00bc70 tx=0x7f27bc00bf80 comp rx=0 tx=0).stop 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 -- 172.21.15.42:0/2538084606 shutdown_connections 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45213440 0x55ee45213850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45093f40 0x55ee45218db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 --2- 172.21.15.42:0/2538084606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee45212f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 -- 172.21.15.42:0/2538084606 >> 172.21.15.42:0/2538084606 conn(0x55ee450f40c0 msgr2=0x55ee4520e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 -- 172.21.15.42:0/2538084606 shutdown_connections 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.462+0000 7f27de592b80 1 -- 172.21.15.42:0/2538084606 wait complete. 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.463+0000 7f27de592b80 1 -- start start 2024-05-27T10:26:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.463+0000 7f27de592b80 1 Processor -- start 2024-05-27T10:26:12.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee4521f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45213440 0x55ee4521c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee452192f0 0x55ee4521cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ee450370a0 con 0x55ee45093f40 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ee450d1860 con 0x55ee452192f0 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ee4521fdc0 con 0x55ee45213440 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27de592b80 10 client.0 _ll_register_callbacks cb 0x55ee450dad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee4521f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee4521f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35944/0 (socket says 172.21.15.42:35944) 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27c7e00700 1 -- 172.21.15.42:0/350523924 learned_addr learned my addr 172.21.15.42:0/350523924 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.464+0000 7f27cd000700 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45213440 0x55ee4521c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27c7e00700 1 -- 172.21.15.42:0/350523924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45213440 msgr2=0x55ee4521c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27c7e00700 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45213440 0x55ee4521c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27c7e00700 1 -- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee452192f0 msgr2=0x55ee4521cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27c7e00700 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee452192f0 0x55ee4521cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27c7e00700 1 -- 172.21.15.42:0/350523924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee450809d0 con 0x55ee45093f40 2024-05-27T10:26:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27c7e00700 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee4521f570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f27c806c670 tx=0x7f27b40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d609760ab6272c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27aea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:12.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27aea00700 1 -- 172.21.15.42:0/350523924 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4003bd0 con 0x55ee45093f40 2024-05-27T10:26:12.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27bc01a460 con 0x55ee45093f40 2024-05-27T10:26:12.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.465+0000 7f27aea00700 1 -- 172.21.15.42:0/350523924 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f27b4003d70 con 0x55ee45093f40 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27aea00700 1 -- 172.21.15.42:0/350523924 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27b400b790 con 0x55ee45093f40 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27de592b80 10 client.16599 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee45101160 con 0x55ee45093f40 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27aea00700 1 -- 172.21.15.42:0/350523924 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27bc01a460 con 0x55ee45093f40 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27aea00700 1 -- 172.21.15.42:0/350523924 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f27b402f6b0 con 0x55ee45093f40 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27de592b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.466+0000 7f27de592b80 10 client.16599 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.467+0000 7f27de592b80 20 client.16599 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.467+0000 7f27de592b80 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee4522cb00 0x55ee4522ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.467+0000 7f27de592b80 4 client.16599 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.467+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee45142e90 con 0x55ee4522cb00 2024-05-27T10:26:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.467+0000 7f27cd000700 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee4522cb00 0x55ee4522ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.468+0000 7f27cd000700 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee4522cb00 0x55ee4522ef20 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9667b2ba8aa3c26 server_cookie=267b5f9ac1965cee in_seq=0 out_seq=0 2024-05-27T10:26:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.468+0000 7f27aea00700 10 client.16599 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.470+0000 7f27aea00700 1 -- 172.21.15.42:0/350523924 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ee45142e90 con 0x55ee4522cb00 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.470+0000 7f27aea00700 10 client.16599 handle_command_reply: tid=0 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.470+0000 7f27de592b80 1 client.16599 shutdown 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 msgr2=0x55ee4521f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee4521f570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f27c806c670 tx=0x7f27b40095e0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 shutdown_connections 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee4522cb00 0x55ee4522ef20 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee452192f0 0x55ee4521cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee4521f570 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 --2- 172.21.15.42:0/350523924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45213440 0x55ee4521c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 >> 172.21.15.42:0/350523924 conn(0x55ee450f40c0 msgr2=0x55ee45211d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27aea00700 0 client.16599 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 shutdown_connections 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 1 -- 172.21.15.42:0/350523924 wait complete. 2024-05-27T10:26:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.471+0000 7f27de592b80 20 client.16599 trim_cache size 0 max 16384 2024-05-27T10:26:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.493+0000 7f27de592b80 1 Processor -- start 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.493+0000 7f27de592b80 1 -- start start 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27de592b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45093f40 0x55ee4521ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27de592b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45228e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27de592b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 0x55ee452181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27de592b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81b3f70 con 0x55ee452192f0 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27de592b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81b17a0 con 0x55ee45213440 2024-05-27T10:26:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27de592b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81af5f0 con 0x55ee45093f40 2024-05-27T10:26:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 0x55ee452181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 0x55ee452181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35960/0 (socket says 172.21.15.42:35960) 2024-05-27T10:26:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.494+0000 7f27c7e00700 1 -- 172.21.15.42:0/3764669263 learned_addr learned my addr 172.21.15.42:0/3764669263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45093f40 0x55ee4521ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27cd000700 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45228e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 -- 172.21.15.42:0/3764669263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45093f40 msgr2=0x55ee4521ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45093f40 0x55ee4521ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 -- 172.21.15.42:0/3764669263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 msgr2=0x55ee45228e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45228e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 -- 172.21.15.42:0/3764669263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4520e9f0 con 0x55ee452192f0 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27cd000700 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45228e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.495+0000 7f27c7e00700 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 0x55ee452181b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f27b4003760 tx=0x7f27b4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2d0176a2c51e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.496+0000 7f27b9600700 1 -- 172.21.15.42:0/3764669263 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b402f620 con 0x55ee452192f0 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.496+0000 7f27b9600700 1 -- 172.21.15.42:0/3764669263 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27b4035070 con 0x55ee452192f0 2024-05-27T10:26:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.496+0000 7f27b9600700 1 -- 172.21.15.42:0/3764669263 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b400bd00 con 0x55ee452192f0 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 -- 172.21.15.42:0/3764669263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 msgr2=0x55ee452181b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 0x55ee452181b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f27b4003760 tx=0x7f27b4005fb0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 -- 172.21.15.42:0/3764669263 shutdown_connections 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee452192f0 0x55ee452181b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee45093f40 0x55ee4521ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 --2- 172.21.15.42:0/3764669263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45228e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 -- 172.21.15.42:0/3764669263 >> 172.21.15.42:0/3764669263 conn(0x55ee450f40c0 msgr2=0x55ee45217aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 -- 172.21.15.42:0/3764669263 shutdown_connections 2024-05-27T10:26:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 -- 172.21.15.42:0/3764669263 wait complete. 2024-05-27T10:26:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.497+0000 7f27de592b80 1 -- start start 2024-05-27T10:26:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 Processor -- start 2024-05-27T10:26:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee45222650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45222b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee452975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81afc60 con 0x55ee45093f40 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81b10d0 con 0x55ee45213440 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81b0330 con 0x55ee452192f0 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27de592b80 10 client.0 _ll_register_callbacks cb 0x55ee450dad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.498+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee45222650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee45222650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35976/0 (socket says 172.21.15.42:35976) 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 -- 172.21.15.42:0/199229474 learned_addr learned my addr 172.21.15.42:0/199229474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27cd000700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45222b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee452975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 -- 172.21.15.42:0/199229474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 msgr2=0x55ee452975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee452975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 -- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 msgr2=0x55ee45222b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45222b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27c7e00700 1 -- 172.21.15.42:0/199229474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee45297b20 con 0x55ee45093f40 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.499+0000 7f27cd000700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45222b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27c7e00700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee45222650 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27b402fbf0 tx=0x7f27b4009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7449c470dcda5b31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27aea00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27aea00700 1 -- 172.21.15.42:0/199229474 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4043060 con 0x55ee45093f40 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27aea00700 1 -- 172.21.15.42:0/199229474 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f27b400bb30 con 0x55ee45093f40 2024-05-27T10:26:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b4035070 con 0x55ee45093f40 2024-05-27T10:26:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27aea00700 1 -- 172.21.15.42:0/199229474 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27b400bd30 con 0x55ee45093f40 2024-05-27T10:26:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27de592b80 10 client.16605 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.500+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee45298da0 con 0x55ee45093f40 2024-05-27T10:26:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27aea00700 1 -- 172.21.15.42:0/199229474 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27b4035070 con 0x55ee45093f40 2024-05-27T10:26:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27aea00700 1 -- 172.21.15.42:0/199229474 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f27b402e4a0 con 0x55ee45093f40 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27de592b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27de592b80 10 client.16605 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27de592b80 20 client.16605 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27de592b80 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee45299370 0x55ee4529b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27de592b80 4 client.16605 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c5b78a64", "release": true, "await": true}] 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.501+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c5b78a64", "release": true, "await": true}) -- 0x55ee4513c670 con 0x55ee45299370 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.502+0000 7f27cd000700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee45299370 0x55ee4529b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.503+0000 7f27cd000700 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee45299370 0x55ee4529b790 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7c5ef5ec8f17c5 server_cookie=480cc4dbdc6e8a73 in_seq=0 out_seq=0 2024-05-27T10:26:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.503+0000 7f27aea00700 10 client.16605 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:12.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.533+0000 7f27aea00700 1 -- 172.21.15.42:0/199229474 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ee4513c670 con 0x55ee45299370 2024-05-27T10:26:12.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.533+0000 7f27aea00700 10 client.16605 handle_command_reply: tid=0 2024-05-27T10:26:12.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.533+0000 7f27de592b80 1 client.16605 shutdown 2024-05-27T10:26:12.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 msgr2=0x55ee45222650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee45222650 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27b402fbf0 tx=0x7f27b4009230 comp rx=0 tx=0).stop 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 shutdown_connections 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55ee45299370 0x55ee4529b790 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee452192f0 0x55ee452975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee45093f40 0x55ee45222650 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 --2- 172.21.15.42:0/199229474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ee45213440 0x55ee45222b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 >> 172.21.15.42:0/199229474 conn(0x55ee450f40c0 msgr2=0x55ee45217aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27aea00700 0 client.16605 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 shutdown_connections 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 1 -- 172.21.15.42:0/199229474 wait complete. 2024-05-27T10:26:12.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.534+0000 7f27de592b80 20 client.16605 trim_cache size 0 max 16384 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 33, 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 20, 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "c5b78a64": { 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 20, 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:26:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 138.9, 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:12.536 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.535+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27b00b29f0 msgr2=0x7f27b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.535+0000 7f27cfa00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27b00b29f0 0x7f27b00b4f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27c000ef00 tx=0x7f27c0009040 comp rx=0 tx=0).stop 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.535+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 msgr2=0x7f27c81a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.535+0000 7f27cfa00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 0x7f27c81a1c80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f27c000ef30 tx=0x7f27c000c450 comp rx=0 tx=0).stop 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 shutdown_connections 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f27b00b29f0 0x7f27b00b4f00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27c81a1870 0x7f27c81a1c80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27c8104620 0x7f27c819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 --2- 172.21.15.42:0/1847210417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c8109850 0x7f27c819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:12.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 >> 172.21.15.42:0/1847210417 conn(0x7f27c8100090 msgr2=0x7f27c810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:12.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 shutdown_connections 2024-05-27T10:26:12.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:12.536+0000 7f27cfa00700 1 -- 172.21.15.42:0/1847210417 wait complete. 2024-05-27T10:26:12.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:12 smithi042 ceph-mon[31661]: pgmap v228: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 932 KiB/s wr, 147 op/s 2024-05-27T10:26:13.057 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:12 smithi143 ceph-mon[35545]: pgmap v228: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 932 KiB/s wr, 147 op/s 2024-05-27T10:26:13.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:12 smithi104 ceph-mon[34812]: pgmap v228: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 932 KiB/s wr, 147 op/s 2024-05-27T10:26:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.409+0000 7f95d1600640 1 -- 172.21.15.42:0/2057978013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc110500 msgr2=0x7f95cc112920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.409+0000 7f95d1600640 1 --2- 172.21.15.42:0/2057978013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc110500 0x7f95cc112920 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ca0 tx=0x7f95c402a8c0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 -- 172.21.15.42:0/2057978013 shutdown_connections 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 --2- 172.21.15.42:0/2057978013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc110500 0x7f95cc112920 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 --2- 172.21.15.42:0/2057978013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95cc10dba0 0x7f95cc10ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 --2- 172.21.15.42:0/2057978013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc075420 0x7f95cc075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 -- 172.21.15.42:0/2057978013 >> 172.21.15.42:0/2057978013 conn(0x7f95cc079310 msgr2=0x7f95cc079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 -- 172.21.15.42:0/2057978013 shutdown_connections 2024-05-27T10:26:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.410+0000 7f95d1600640 1 -- 172.21.15.42:0/2057978013 wait complete. 2024-05-27T10:26:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.412+0000 7f95d1600640 1 Processor -- start 2024-05-27T10:26:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.412+0000 7f95d1600640 1 -- start start 2024-05-27T10:26:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95d1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 0x7f95cc1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95d1600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95cc10dba0 0x7f95cc1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95d1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc110500 0x7f95cc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95d1600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc115f60 con 0x7f95cc075420 2024-05-27T10:26:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95d1600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc115de0 con 0x7f95cc10dba0 2024-05-27T10:26:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95d1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1160e0 con 0x7f95cc110500 2024-05-27T10:26:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.413+0000 7f95caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 0x7f95cc1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 0x7f95cc1a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44368/0 (socket says 172.21.15.42:44368) 2024-05-27T10:26:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 -- 172.21.15.42:0/2301992175 learned_addr learned my addr 172.21.15.42:0/2301992175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95ca000640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95cc10dba0 0x7f95cc1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95cb400640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc110500 0x7f95cc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 -- 172.21.15.42:0/2301992175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc110500 msgr2=0x7f95cc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc110500 0x7f95cc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 -- 172.21.15.42:0/2301992175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95cc10dba0 msgr2=0x7f95cc1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95cc10dba0 0x7f95cc1a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95caa00640 1 -- 172.21.15.42:0/2301992175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc19cea0 con 0x7f95cc075420 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.414+0000 7f95cb400640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc110500 0x7f95cc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.415+0000 7f95caa00640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 0x7f95cc1a2590 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d770 tx=0x7f95bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.415+0000 7f95c3400640 1 -- 172.21.15.42:0/2301992175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95cc075420 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.415+0000 7f95c3400640 1 -- 172.21.15.42:0/2301992175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95bc009c30 con 0x7f95cc075420 2024-05-27T10:26:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.415+0000 7f95c3400640 1 -- 172.21.15.42:0/2301992175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc003c50 con 0x7f95cc075420 2024-05-27T10:26:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.416+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc19d190 con 0x7f95cc075420 2024-05-27T10:26:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.416+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc1a93e0 con 0x7f95cc075420 2024-05-27T10:26:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.421+0000 7f95d85fb740 1 Processor -- start 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.421+0000 7f95d85fb740 1 -- start start 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.421+0000 7f95c3400640 1 -- 172.21.15.42:0/2301992175 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f95bc003df0 con 0x7f95cc075420 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.421+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c796776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.421+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 0x561c79677c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.421+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792b44d0 0x561c79678170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95d85fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561c795e8b90 con 0x561c792d51b0 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95d85fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561c795d8730 con 0x561c792ac2f0 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95d85fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561c795dc770 con 0x561c792b44d0 2024-05-27T10:26:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95caa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 0x561c79677c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95caa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 0x561c79677c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33348/0 (socket says 172.21.15.42:33348) 2024-05-27T10:26:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95caa00640 1 -- 172.21.15.42:0/1922305985 learned_addr learned my addr 172.21.15.42:0/1922305985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95cb400640 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c796776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.422+0000 7f95ca000640 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792b44d0 0x561c79678170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95caa00640 1 -- 172.21.15.42:0/1922305985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792b44d0 msgr2=0x561c79678170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95caa00640 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792b44d0 0x561c79678170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95caa00640 1 -- 172.21.15.42:0/1922305985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 msgr2=0x561c796776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95caa00640 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c796776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95caa00640 1 -- 172.21.15.42:0/1922305985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c795718b0 con 0x561c792ac2f0 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95ca000640 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792b44d0 0x561c79678170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.423+0000 7f95caa00640 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 0x561c79677c30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0df5b0 tx=0x7f95bc0ea5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaaa5007fc31f690 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.424+0000 7f95c3400640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f95b00b2aa0 0x7f95b00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.424+0000 7f95c3400640 1 -- 172.21.15.42:0/2301992175 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f95bc0e1f60 con 0x7f95cc075420 2024-05-27T10:26:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.424+0000 7f95b7400640 1 -- 172.21.15.42:0/1922305985 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0f5070 con 0x561c792ac2f0 2024-05-27T10:26:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.424+0000 7f95cb400640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f95b00b2aa0 0x7f95b00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.424+0000 7f95b7400640 1 -- 172.21.15.42:0/1922305985 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95bc0a8d10 con 0x561c792ac2f0 2024-05-27T10:26:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.424+0000 7f95b7400640 1 -- 172.21.15.42:0/1922305985 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0a8b50 con 0x561c792ac2f0 2024-05-27T10:26:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.425+0000 7f95cb400640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f95b00b2aa0 0x7f95b00b4fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95c4005e30 tx=0x7f95c4005d80 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.425+0000 7f95d85fb740 1 -- 172.21.15.42:0/1922305985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 msgr2=0x561c79677c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.425+0000 7f95d85fb740 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 0x561c79677c30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0df5b0 tx=0x7f95bc0ea5f0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 -- 172.21.15.42:0/1922305985 shutdown_connections 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792b44d0 0x561c79678170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792ac2f0 0x561c79677c30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 --2- 172.21.15.42:0/1922305985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c796776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 -- 172.21.15.42:0/1922305985 >> 172.21.15.42:0/1922305985 conn(0x561c792c9300 msgr2=0x561c79674010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 -- 172.21.15.42:0/1922305985 shutdown_connections 2024-05-27T10:26:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.426+0000 7f95d85fb740 1 -- 172.21.15.42:0/1922305985 wait complete. 2024-05-27T10:26:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.427+0000 7f95d85fb740 1 -- start start 2024-05-27T10:26:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.427+0000 7f95d85fb740 1 Processor -- start 2024-05-27T10:26:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c79692a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c79692fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561c792be790 con 0x561c792ac2f0 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561c79580bc0 con 0x561c792b44d0 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561c79695a00 con 0x561c792d51b0 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95d85fb740 10 client.0 _ll_register_callbacks cb 0x561c7955e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c79692a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.428+0000 7f95ca000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c79692fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95ca000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c79692fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33364/0 (socket says 172.21.15.42:33364) 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c79692a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44396/0 (socket says 172.21.15.42:44396) 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95ca000640 1 -- 172.21.15.42:0/2357551031 learned_addr learned my addr 172.21.15.42:0/2357551031 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95cb400640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 -- 172.21.15.42:0/2357551031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 msgr2=0x561c7969ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 -- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 msgr2=0x561c79692fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c79692fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 -- 172.21.15.42:0/2357551031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c792ed210 con 0x561c792ac2f0 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95cb400640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95ca000640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c79692fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95caa00640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c79692a80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0e2250 tx=0x7f95bc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32df2691e53e93af server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.429+0000 7f95abe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.430+0000 7f95abe00640 1 -- 172.21.15.42:0/2357551031 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc020070 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.430+0000 7f95abe00640 1 -- 172.21.15.42:0/2357551031 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f95bc0a5490 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.430+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95bc0a8d10 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95abe00640 1 -- 172.21.15.42:0/2357551031 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f95bc0a8970 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95d85fb740 10 client.16617 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561c793b9620 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95abe00640 1 -- 172.21.15.42:0/2357551031 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95bc0a8d10 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95abe00640 1 -- 172.21.15.42:0/2357551031 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f95bc0df8f0 con 0x561c792ac2f0 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95d85fb740 10 client.16617 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.431+0000 7f95d85fb740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.432+0000 7f95d85fb740 20 client.16617 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.432+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c79293d80 0x561c796a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.432+0000 7f95d85fb740 4 client.16617 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.432+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c795eaae0 con 0x561c79293d80 2024-05-27T10:26:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.432+0000 7f95ca000640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c79293d80 0x561c796a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.433+0000 7f95ca000640 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c79293d80 0x561c796a4af0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf1d1f59c57f274 server_cookie=e7df28edb8771d21 in_seq=0 out_seq=0 2024-05-27T10:26:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.433+0000 7f95abe00640 10 client.16617 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.435+0000 7f95abe00640 1 -- 172.21.15.42:0/2357551031 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561c795eaae0 con 0x561c79293d80 2024-05-27T10:26:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.435+0000 7f95abe00640 10 client.16617 handle_command_reply: tid=0 2024-05-27T10:26:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.435+0000 7f95d85fb740 1 client.16617 shutdown 2024-05-27T10:26:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.435+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 msgr2=0x561c79692a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.435+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c79692a80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0e2250 tx=0x7f95bc0035f0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 shutdown_connections 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c79293d80 0x561c796a4af0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c79692fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2357551031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c79692a80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 >> 172.21.15.42:0/2357551031 conn(0x561c792c9300 msgr2=0x561c79679580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95abe00640 0 client.16617 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 shutdown_connections 2024-05-27T10:26:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 1 -- 172.21.15.42:0/2357551031 wait complete. 2024-05-27T10:26:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.436+0000 7f95d85fb740 20 client.16617 trim_cache size 0 max 16384 2024-05-27T10:26:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.465+0000 7f95d85fb740 1 Processor -- start 2024-05-27T10:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.465+0000 7f95d85fb740 1 -- start start 2024-05-27T10:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.465+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c796780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c7969ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95d85fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1abe30 con 0x561c792ac2f0 2024-05-27T10:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95d85fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1a96a0 con 0x561c792b44d0 2024-05-27T10:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95d85fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc050860 con 0x561c792d51b0 2024-05-27T10:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c796780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c796780c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44402/0 (socket says 172.21.15.42:44402) 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95caa00640 1 -- 172.21.15.42:0/2272744712 learned_addr learned my addr 172.21.15.42:0/2272744712 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95cb400640 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.466+0000 7f95ca000640 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c7969ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.467+0000 7f95caa00640 1 -- 172.21.15.42:0/2272744712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 msgr2=0x561c7969f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.467+0000 7f95caa00640 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.467+0000 7f95caa00640 1 -- 172.21.15.42:0/2272744712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 msgr2=0x561c7969ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.467+0000 7f95caa00640 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c7969ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.467+0000 7f95caa00640 1 -- 172.21.15.42:0/2272744712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c79674c00 con 0x561c792ac2f0 2024-05-27T10:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.467+0000 7f95caa00640 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c796780c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0a9190 tx=0x7f95bc0a8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc4daad22d017eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.468+0000 7f95b6a00640 1 -- 172.21.15.42:0/2272744712 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0c2020 con 0x561c792ac2f0 2024-05-27T10:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.468+0000 7f95b6a00640 1 -- 172.21.15.42:0/2272744712 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95bc0a9900 con 0x561c792ac2f0 2024-05-27T10:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.468+0000 7f95b6a00640 1 -- 172.21.15.42:0/2272744712 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc020070 con 0x561c792ac2f0 2024-05-27T10:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 -- 172.21.15.42:0/2272744712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 msgr2=0x561c796780c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c796780c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0a9190 tx=0x7f95bc0a8b50 comp rx=0 tx=0).stop 2024-05-27T10:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 -- 172.21.15.42:0/2272744712 shutdown_connections 2024-05-27T10:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792d51b0 0x561c7969f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c7969ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 --2- 172.21.15.42:0/2272744712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792ac2f0 0x561c796780c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 -- 172.21.15.42:0/2272744712 >> 172.21.15.42:0/2272744712 conn(0x561c792c9300 msgr2=0x561c79692d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 -- 172.21.15.42:0/2272744712 shutdown_connections 2024-05-27T10:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.469+0000 7f95d85fb740 1 -- 172.21.15.42:0/2272744712 wait complete. 2024-05-27T10:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.470+0000 7f95d85fb740 1 -- start start 2024-05-27T10:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.470+0000 7f95d85fb740 1 Processor -- start 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792ac2f0 0x561c796a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c796a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c7970d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc0734c0 con 0x561c792d51b0 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc074930 con 0x561c792b44d0 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc073b90 con 0x561c792ac2f0 2024-05-27T10:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.471+0000 7f95d85fb740 10 client.0 _ll_register_callbacks cb 0x7f95bc0f3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.472+0000 7f95ca000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c796a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.472+0000 7f95ca000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c796a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33380/0 (socket says 172.21.15.42:33380) 2024-05-27T10:26:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.472+0000 7f95ca000640 1 -- 172.21.15.42:0/142076359 learned_addr learned my addr 172.21.15.42:0/142076359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.472+0000 7f95cb400640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c7970d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.472+0000 7f95caa00640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792ac2f0 0x561c796a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.472+0000 7f95cb400640 1 -- 172.21.15.42:0/142076359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792ac2f0 msgr2=0x561c796a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95cb400640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792ac2f0 0x561c796a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95cb400640 1 -- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 msgr2=0x561c796a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95cb400640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c796a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95cb400640 1 -- 172.21.15.42:0/142076359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7970dc80 con 0x561c792d51b0 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95ca000640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c796a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95cb400640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c7970d710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006010 tx=0x7f95c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af0950825d0344f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.473+0000 7f95abe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.474+0000 7f95abe00640 1 -- 172.21.15.42:0/142076359 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4057060 con 0x561c792d51b0 2024-05-27T10:26:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.474+0000 7f95abe00640 1 -- 172.21.15.42:0/142076359 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8528+0+0 (secure 0 0 0) 0x7f95c4004250 con 0x561c792d51b0 2024-05-27T10:26:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.474+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95bc0a9900 con 0x561c792d51b0 2024-05-27T10:26:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.474+0000 7f95abe00640 1 -- 172.21.15.42:0/142076359 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f95c4004450 con 0x561c792d51b0 2024-05-27T10:26:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.474+0000 7f95d85fb740 10 client.16623 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.474+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561c79714f90 con 0x561c792d51b0 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95abe00640 1 -- 172.21.15.42:0/142076359 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95bc0a9900 con 0x561c792d51b0 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95abe00640 1 -- 172.21.15.42:0/142076359 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f95c4030cf0 con 0x561c792d51b0 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95d85fb740 10 client.16623 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95d85fb740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95d85fb740 20 client.16623 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95d85fb740 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c792691c0 0x561c797176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95d85fb740 4 client.16623 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:26:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.475+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c795e42c0 con 0x561c792691c0 2024-05-27T10:26:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.476+0000 7f95caa00640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c792691c0 0x561c797176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.477+0000 7f95caa00640 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c792691c0 0x561c797176d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c60ba3e535b552 server_cookie=909612d7970818a5 in_seq=0 out_seq=0 2024-05-27T10:26:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.477+0000 7f95abe00640 10 client.16623 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95abe00640 1 -- 172.21.15.42:0/142076359 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561c795e42c0 con 0x561c792691c0 2024-05-27T10:26:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95abe00640 10 client.16623 handle_command_reply: tid=0 2024-05-27T10:26:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 client.16623 shutdown 2024-05-27T10:26:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 msgr2=0x561c7970d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c7970d710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006010 tx=0x7f95c40095c0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 shutdown_connections 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561c792d51b0 0x561c7970d710 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c792b44d0 0x561c796a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x561c792691c0 0x561c797176d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 --2- 172.21.15.42:0/142076359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c792ac2f0 0x561c796a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.510+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 >> 172.21.15.42:0/142076359 conn(0x561c792c9300 msgr2=0x561c79694850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.511+0000 7f95abe00640 0 client.16623 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.511+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 shutdown_connections 2024-05-27T10:26:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.511+0000 7f95d85fb740 1 -- 172.21.15.42:0/142076359 wait complete. 2024-05-27T10:26:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.511+0000 7f95d85fb740 20 client.16623 trim_cache size 0 max 16384 2024-05-27T10:26:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:26:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:26:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:26:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f95b00b2aa0 msgr2=0x7f95b00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f95b00b2aa0 0x7f95b00b4fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95c4005e30 tx=0x7f95c4005d80 comp rx=0 tx=0).stop 2024-05-27T10:26:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 msgr2=0x7f95cc1a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 0x7f95cc1a2590 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d770 tx=0x7f95bc00dc30 comp rx=0 tx=0).stop 2024-05-27T10:26:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 shutdown_connections 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f95b00b2aa0 0x7f95b00b4fb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95cc110500 0x7f95cc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95cc10dba0 0x7f95cc1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 --2- 172.21.15.42:0/2301992175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95cc075420 0x7f95cc1a2590 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 >> 172.21.15.42:0/2301992175 conn(0x7f95cc079310 msgr2=0x7f95cc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.512+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 shutdown_connections 2024-05-27T10:26:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:13.513+0000 7f95d1600640 1 -- 172.21.15.42:0/2301992175 wait complete. 2024-05-27T10:26:13.608 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c5b78a64', total seconds elapsed: 30.8 2024-05-27T10:26:13.608 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.95 2024-05-27T10:26:13.608 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:26:14.197 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:26:15.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:14 smithi143 ceph-mon[35545]: pgmap v229: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 932 KiB/s wr, 147 op/s 2024-05-27T10:26:15.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:14 smithi042 ceph-mon[31661]: pgmap v229: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 932 KiB/s wr, 147 op/s 2024-05-27T10:26:15.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:14 smithi104 ceph-mon[34812]: pgmap v229: 97 pgs: 97 active+clean; 329 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 932 KiB/s wr, 147 op/s 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 -- 172.21.15.42:0/908780010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 msgr2=0x7f718c1130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 --2- 172.21.15.42:0/908780010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c1130a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7188009f90 tx=0x7f718800b330 comp rx=0 tx=0).stop 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 -- 172.21.15.42:0/908780010 shutdown_connections 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 --2- 172.21.15.42:0/908780010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c1130a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 --2- 172.21.15.42:0/908780010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c1030d0 0x7f718c1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 --2- 172.21.15.42:0/908780010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:16.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.877+0000 7f7194800700 1 -- 172.21.15.42:0/908780010 >> 172.21.15.42:0/908780010 conn(0x7f718c0fedb0 msgr2=0x7f718c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:16.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.878+0000 7f7194800700 1 -- 172.21.15.42:0/908780010 shutdown_connections 2024-05-27T10:26:16.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.878+0000 7f7194800700 1 -- 172.21.15.42:0/908780010 wait complete. 2024-05-27T10:26:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.880+0000 7f7194800700 1 Processor -- start 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 -- start start 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c1030d0 0x7f718c10b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c10b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c10ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f718c116580 con 0x7f718c110c80 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f718c116400 con 0x7f718c103a20 2024-05-27T10:26:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.881+0000 7f7194800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f718c116700 con 0x7f718c1030d0 2024-05-27T10:26:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.882+0000 7f7191400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c10b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.882+0000 7f7191400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c10b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33408/0 (socket says 172.21.15.42:33408) 2024-05-27T10:26:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.882+0000 7f7191400700 1 -- 172.21.15.42:0/895778644 learned_addr learned my addr 172.21.15.42:0/895778644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.882+0000 7f7191e00700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c1030d0 0x7f718c10b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.882+0000 7f7192800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c10ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7191400700 1 -- 172.21.15.42:0/895778644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c1030d0 msgr2=0x7f718c10b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7191400700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c1030d0 0x7f718c10b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7191400700 1 -- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 msgr2=0x7f718c10ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7191400700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c10ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7191400700 1 -- 172.21.15.42:0/895778644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c110640 con 0x7f718c103a20 2024-05-27T10:26:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7192800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.883+0000 7f7191400700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c10b6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f717c00ec70 tx=0x7f717c00ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.884+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c00cd40 con 0x7f718c103a20 2024-05-27T10:26:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.884+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c110930 con 0x7f718c103a20 2024-05-27T10:26:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.884+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f717c0044b0 con 0x7f718c103a20 2024-05-27T10:26:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.884+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c010430 con 0x7f718c103a20 2024-05-27T10:26:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.884+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1055c0 con 0x7f718c103a20 2024-05-27T10:26:16.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.886+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c06be80 con 0x7f718c103a20 2024-05-27T10:26:16.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.892+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f717c003710 con 0x7f718c103a20 2024-05-27T10:26:16.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.896+0000 7f7186a00700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71740b2aa0 0x7f71740b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:16.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.896+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f717c0e2970 con 0x7f718c103a20 2024-05-27T10:26:16.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.897+0000 7f7191e00700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71740b2aa0 0x7f71740b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:16.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.898+0000 7f7191e00700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71740b2aa0 0x7f71740b4fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7180009c60 tx=0x7f7180009260 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:16.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:16.908+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f717c0aa3a0 con 0x7f718c103a20 2024-05-27T10:26:17.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:16 smithi143 ceph-mon[35545]: pgmap v230: 97 pgs: 97 active+clean; 331 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 491 KiB/s wr, 51 op/s 2024-05-27T10:26:17.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:16 smithi143 ceph-mon[35545]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T10:26:17.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:16 smithi042 ceph-mon[31661]: pgmap v230: 97 pgs: 97 active+clean; 331 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 491 KiB/s wr, 51 op/s 2024-05-27T10:26:17.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:16 smithi042 ceph-mon[31661]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T10:26:17.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:16 smithi104 ceph-mon[34812]: pgmap v230: 97 pgs: 97 active+clean; 331 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 491 KiB/s wr, 51 op/s 2024-05-27T10:26:17.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:16 smithi104 ceph-mon[34812]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T10:26:17.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.400+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f718c0512b0 con 0x7f718c103a20 2024-05-27T10:26:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:26:17.403 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:24:00.498568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:26:17.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.403+0000 7f7186a00700 1 -- 172.21.15.42:0/895778644 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11881 (secure 0 0 0) 0x7f717c0a9ca0 con 0x7f718c103a20 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71740b2aa0 msgr2=0x7f71740b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71740b2aa0 0x7f71740b4fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7180009c60 tx=0x7f7180009260 comp rx=0 tx=0).stop 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 msgr2=0x7f718c10b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c10b6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f717c00ec70 tx=0x7f717c00ef80 comp rx=0 tx=0).stop 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 shutdown_connections 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f71740b2aa0 0x7f71740b4fb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f718c110c80 0x7f718c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c1030d0 0x7f718c10b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 --2- 172.21.15.42:0/895778644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718c103a20 0x7f718c10b6e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 >> 172.21.15.42:0/895778644 conn(0x7f718c0fedb0 msgr2=0x7f718c112640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.407+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 shutdown_connections 2024-05-27T10:26:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:17.408+0000 7f7194800700 1 -- 172.21.15.42:0/895778644 wait complete. 2024-05-27T10:26:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 33 2024-05-27T10:26:18.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:17 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:26:18.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:17 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/895778644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:18.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:17 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:26:18.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:17 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/895778644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:18.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:17 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:26:18.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:17 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/895778644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:18.359 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 138.95 --await-for 45.0' 2024-05-27T10:26:18.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 138.95 --await-for 45.0 2024-05-27T10:26:18.717 INFO:tasks.workunit.client.0.smithi042.stdout: 46 3295 115 2500 82 965 141 2024-05-27T10:26:18.917 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:26:19.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:18 smithi042 ceph-mon[31661]: pgmap v232: 97 pgs: 97 active+clean; 331 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 309 KiB/s wr, 20 op/s 2024-05-27T10:26:19.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:18 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:26:19.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:18 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:26:19.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:18 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:26:19.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:18 smithi104 ceph-mon[34812]: pgmap v232: 97 pgs: 97 active+clean; 331 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 309 KiB/s wr, 20 op/s 2024-05-27T10:26:19.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:18 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:26:19.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:18 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:26:19.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:18 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:26:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:18 smithi143 ceph-mon[35545]: pgmap v232: 97 pgs: 97 active+clean; 331 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 309 KiB/s wr, 20 op/s 2024-05-27T10:26:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:18 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:26:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:18 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:26:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:18 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:26:20.820 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:20 smithi042 ceph-mon[31661]: pgmap v233: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-05-27T10:26:21.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:20 smithi104 ceph-mon[34812]: pgmap v233: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-05-27T10:26:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:20 smithi143 ceph-mon[35545]: pgmap v233: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-05-27T10:26:21.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 -- 172.21.15.42:0/617479996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106560 msgr2=0x7f74f0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:21.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 --2- 172.21.15.42:0/617479996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106560 0x7f74f0106970 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009990 tx=0x7f74dc009ca0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 -- 172.21.15.42:0/617479996 shutdown_connections 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 --2- 172.21.15.42:0/617479996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f0106eb0 0x7f74f0113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 --2- 172.21.15.42:0/617479996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f010c5a0 0x7f74f010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 --2- 172.21.15.42:0/617479996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106560 0x7f74f0106970 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.673+0000 7f74f7800700 1 -- 172.21.15.42:0/617479996 >> 172.21.15.42:0/617479996 conn(0x7f74f0102040 msgr2=0x7f74f0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.675+0000 7f74f7800700 1 -- 172.21.15.42:0/617479996 shutdown_connections 2024-05-27T10:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.675+0000 7f74f7800700 1 -- 172.21.15.42:0/617479996 wait complete. 2024-05-27T10:26:21.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.676+0000 7f74f7800700 1 Processor -- start 2024-05-27T10:26:21.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 -- start start 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f0106560 0x7f74f0110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106eb0 0x7f74f0110c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f010c5a0 0x7f74f01111d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01165d0 con 0x7f74f0106eb0 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0116450 con 0x7f74f010c5a0 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.677+0000 7f74f7800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0116750 con 0x7f74f0106560 2024-05-27T10:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.678+0000 7f74efe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106eb0 0x7f74f0110c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.678+0000 7f74f5800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f010c5a0 0x7f74f01111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.678+0000 7f74f5800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f010c5a0 0x7f74f01111d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33418/0 (socket says 172.21.15.42:33418) 2024-05-27T10:26:21.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.678+0000 7f74f5800700 1 -- 172.21.15.42:0/3850300530 learned_addr learned my addr 172.21.15.42:0/3850300530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:21.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.678+0000 7f74f4e00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f0106560 0x7f74f0110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74efe00700 1 -- 172.21.15.42:0/3850300530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f0106560 msgr2=0x7f74f0110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74efe00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f0106560 0x7f74f0110750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74efe00700 1 -- 172.21.15.42:0/3850300530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f010c5a0 msgr2=0x7f74f01111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74efe00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f010c5a0 0x7f74f01111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74efe00700 1 -- 172.21.15.42:0/3850300530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f01ad4a0 con 0x7f74f0106eb0 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74f4e00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f0106560 0x7f74f0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.679+0000 7f74efe00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106eb0 0x7f74f0110c90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f74dc000c00 tx=0x7f74dc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:21.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.680+0000 7f74ed600700 1 -- 172.21.15.42:0/3850300530 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc026070 con 0x7f74f0106eb0 2024-05-27T10:26:21.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.680+0000 7f74ed600700 1 -- 172.21.15.42:0/3850300530 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74dc00fe20 con 0x7f74f0106eb0 2024-05-27T10:26:21.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.680+0000 7f74ed600700 1 -- 172.21.15.42:0/3850300530 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc021dc0 con 0x7f74f0106eb0 2024-05-27T10:26:21.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.680+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f01ad730 con 0x7f74f0106eb0 2024-05-27T10:26:21.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.680+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01adb30 con 0x7f74f0106eb0 2024-05-27T10:26:21.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.685+0000 7f74ed600700 1 -- 172.21.15.42:0/3850300530 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f74dc009d10 con 0x7f74f0106eb0 2024-05-27T10:26:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.687+0000 7f74ed600700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f74d80b2aa0 0x7f74d80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.687+0000 7f7506419b80 1 Processor -- start 2024-05-27T10:26:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.687+0000 7f74ed600700 1 -- 172.21.15.42:0/3850300530 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f74dc0e2ef0 con 0x7f74f0106eb0 2024-05-27T10:26:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.687+0000 7f7506419b80 1 -- start start 2024-05-27T10:26:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.688+0000 7f74f4e00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f74d80b2aa0 0x7f74d80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.688+0000 7f7506419b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x5572334855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.688+0000 7f7506419b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x557233485b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.688+0000 7f7506419b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x55723347fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.688+0000 7f7506419b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5572333acfe0 con 0x5572332fffe0 2024-05-27T10:26:21.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f7506419b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55723339cb80 con 0x5572333ebb90 2024-05-27T10:26:21.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f7506419b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5572333a0bc0 con 0x55723347f680 2024-05-27T10:26:21.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f74efe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x557233485b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f74efe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x557233485b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44472/0 (socket says 172.21.15.42:44472) 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f74efe00700 1 -- 172.21.15.42:0/4074105775 learned_addr learned my addr 172.21.15.42:0/4074105775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f74f5800700 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x55723347fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f74f4e00700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f74d80b2aa0 0x7f74d80b4fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74f01079d0 tx=0x7f74e4009260 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.689+0000 7f74f4e00700 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x5572334855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74efe00700 1 -- 172.21.15.42:0/4074105775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 msgr2=0x55723347fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74efe00700 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x55723347fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74efe00700 1 -- 172.21.15.42:0/4074105775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 msgr2=0x5572334855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74efe00700 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x5572334855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74efe00700 1 -- 172.21.15.42:0/4074105775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572333655c0 con 0x5572332fffe0 2024-05-27T10:26:21.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74efe00700 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x557233485b30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0aa1b0 tx=0x7f74dc0a9e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508630df71cdd84d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:21.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74e2000700 1 -- 172.21.15.42:0/4074105775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0ae5d0 con 0x5572332fffe0 2024-05-27T10:26:21.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74e2000700 1 -- 172.21.15.42:0/4074105775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74dc0b2ff0 con 0x5572332fffe0 2024-05-27T10:26:21.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.690+0000 7f74e2000700 1 -- 172.21.15.42:0/4074105775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0b2dc0 con 0x5572332fffe0 2024-05-27T10:26:21.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 -- 172.21.15.42:0/4074105775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 msgr2=0x557233485b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:21.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x557233485b30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0aa1b0 tx=0x7f74dc0a9e80 comp rx=0 tx=0).stop 2024-05-27T10:26:21.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 -- 172.21.15.42:0/4074105775 shutdown_connections 2024-05-27T10:26:21.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x55723347fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x5572334855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 --2- 172.21.15.42:0/4074105775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x557233485b30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.692+0000 7f7506419b80 1 -- 172.21.15.42:0/4074105775 >> 172.21.15.42:0/4074105775 conn(0x557233399530 msgr2=0x55723347a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:21.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.693+0000 7f7506419b80 1 -- 172.21.15.42:0/4074105775 shutdown_connections 2024-05-27T10:26:21.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.693+0000 7f7506419b80 1 -- 172.21.15.42:0/4074105775 wait complete. 2024-05-27T10:26:21.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.693+0000 7f7506419b80 1 -- start start 2024-05-27T10:26:21.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.694+0000 7f7506419b80 1 Processor -- start 2024-05-27T10:26:21.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x55723348ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x55723348f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x557233489230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5572332a3140 con 0x5572332fffe0 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55723333d900 con 0x5572333ebb90 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557233480230 con 0x55723347f680 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f7506419b80 10 client.0 _ll_register_callbacks cb 0x557233346e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:21.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f74f4e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x55723348ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f74f4e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x55723348ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44482/0 (socket says 172.21.15.42:44482) 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.695+0000 7f74f4e00700 1 -- 172.21.15.42:0/1617268356 learned_addr learned my addr 172.21.15.42:0/1617268356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74efe00700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x55723348f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74f5800700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x557233489230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74f4e00700 1 -- 172.21.15.42:0/1617268356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 msgr2=0x557233489230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74f4e00700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x557233489230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74f4e00700 1 -- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 msgr2=0x55723348f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74f4e00700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x55723348f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.696+0000 7f74f4e00700 1 -- 172.21.15.42:0/1617268356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572332eca70 con 0x5572332fffe0 2024-05-27T10:26:21.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.697+0000 7f74f4e00700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x55723348ee30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f74e4003a00 tx=0x7f74e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1583838c87beb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:21.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.697+0000 7f74d6a00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:21.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.697+0000 7f74d6a00700 1 -- 172.21.15.42:0/1617268356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e400ec50 con 0x5572332fffe0 2024-05-27T10:26:21.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.698+0000 7f74d6a00700 1 -- 172.21.15.42:0/1617268356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f74e400edf0 con 0x5572332fffe0 2024-05-27T10:26:21.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.698+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74dc0b2ff0 con 0x5572332fffe0 2024-05-27T10:26:21.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.698+0000 7f74d6a00700 1 -- 172.21.15.42:0/1617268356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74e4028470 con 0x5572332fffe0 2024-05-27T10:26:21.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.698+0000 7f7506419b80 10 client.16641 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:21.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.698+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55723336d200 con 0x5572332fffe0 2024-05-27T10:26:21.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f74d6a00700 1 -- 172.21.15.42:0/1617268356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74dc0b2ff0 con 0x5572332fffe0 2024-05-27T10:26:21.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f74d6a00700 1 -- 172.21.15.42:0/1617268356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f74e401f400 con 0x5572332fffe0 2024-05-27T10:26:21.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f7506419b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:21.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f7506419b80 10 client.16641 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:26:21.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f7506419b80 20 client.16641 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:21.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f7506419b80 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233498f20 0x55723349b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f7506419b80 4 client.16641 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:21.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.699+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572333aef30 con 0x557233498f20 2024-05-27T10:26:21.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.700+0000 7f74efe00700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233498f20 0x55723349b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.701+0000 7f74efe00700 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233498f20 0x55723349b380 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2615c9aa296c6a server_cookie=abae2790be991e8f in_seq=0 out_seq=0 2024-05-27T10:26:21.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.701+0000 7f74d6a00700 10 client.16641 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:21.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.703+0000 7f74d6a00700 1 -- 172.21.15.42:0/1617268356 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5572333aef30 con 0x557233498f20 2024-05-27T10:26:21.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.703+0000 7f74d6a00700 10 client.16641 handle_command_reply: tid=0 2024-05-27T10:26:21.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.703+0000 7f7506419b80 1 client.16641 shutdown 2024-05-27T10:26:21.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.703+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 msgr2=0x55723348ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:21.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.703+0000 7f7506419b80 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x55723348ee30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f74e4003a00 tx=0x7f74e40047c0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 shutdown_connections 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233498f20 0x55723349b380 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x557233489230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572332fffe0 0x55723348ee30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 --2- 172.21.15.42:0/1617268356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572333ebb90 0x55723348f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 >> 172.21.15.42:0/1617268356 conn(0x557233399530 msgr2=0x55723347e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f74d6a00700 0 client.16641 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 shutdown_connections 2024-05-27T10:26:21.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 1 -- 172.21.15.42:0/1617268356 wait complete. 2024-05-27T10:26:21.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.704+0000 7f7506419b80 20 client.16641 trim_cache size 0 max 16384 2024-05-27T10:26:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.740+0000 7f7506419b80 1 Processor -- start 2024-05-27T10:26:21.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.740+0000 7f7506419b80 1 -- start start 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f7506419b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233489720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f7506419b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5572333ebb90 0x55723347a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f7506419b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 0x55723347a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f7506419b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01b7b60 con 0x55723347f680 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f7506419b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01b5390 con 0x5572332fffe0 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f7506419b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01b3200 con 0x5572333ebb90 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f74f5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 0x55723347a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f74f5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 0x55723347a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44488/0 (socket says 172.21.15.42:44488) 2024-05-27T10:26:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f74f5800700 1 -- 172.21.15.42:0/3696891576 learned_addr learned my addr 172.21.15.42:0/3696891576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.741+0000 7f74efe00700 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5572333ebb90 0x55723347a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74f4e00700 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233489720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74efe00700 1 -- 172.21.15.42:0/3696891576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 msgr2=0x557233489720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74efe00700 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233489720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74efe00700 1 -- 172.21.15.42:0/3696891576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 msgr2=0x55723347a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74efe00700 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 0x55723347a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74efe00700 1 -- 172.21.15.42:0/3696891576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723347b550 con 0x5572333ebb90 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.742+0000 7f74f5800700 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 0x55723347a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.743+0000 7f74efe00700 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5572333ebb90 0x55723347a050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557233480040 tx=0x7f74dc0a9b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca7c3f0a1c6697e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.743+0000 7f74e1600700 1 -- 172.21.15.42:0/3696891576 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0b2ff0 con 0x5572333ebb90 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.744+0000 7f74e1600700 1 -- 172.21.15.42:0/3696891576 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74dc0215c0 con 0x5572333ebb90 2024-05-27T10:26:21.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.744+0000 7f74e1600700 1 -- 172.21.15.42:0/3696891576 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0ae120 con 0x5572333ebb90 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 -- 172.21.15.42:0/3696891576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5572333ebb90 msgr2=0x55723347a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5572333ebb90 0x55723347a050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557233480040 tx=0x7f74dc0a9b70 comp rx=0 tx=0).stop 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 -- 172.21.15.42:0/3696891576 shutdown_connections 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55723347f680 0x55723347a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233489720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 --2- 172.21.15.42:0/3696891576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5572333ebb90 0x55723347a050 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 -- 172.21.15.42:0/3696891576 >> 172.21.15.42:0/3696891576 conn(0x557233399530 msgr2=0x557233489010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 -- 172.21.15.42:0/3696891576 shutdown_connections 2024-05-27T10:26:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.745+0000 7f7506419b80 1 -- 172.21.15.42:0/3696891576 wait complete. 2024-05-27T10:26:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.746+0000 7f7506419b80 1 -- start start 2024-05-27T10:26:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.746+0000 7f7506419b80 1 Processor -- start 2024-05-27T10:26:21.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233493d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572333ebb90 0x5572334942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x5572334947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01b3850 con 0x5572333ebb90 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01b4cc0 con 0x5572332fffe0 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01b3f20 con 0x55723347f680 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.747+0000 7f7506419b80 10 client.0 _ll_register_callbacks cb 0x55723349b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233493d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233493d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:33464/0 (socket says 172.21.15.42:33464) 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 -- 172.21.15.42:0/2344892429 learned_addr learned my addr 172.21.15.42:0/2344892429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:21.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f5800700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x5572334947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74efe00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572333ebb90 0x5572334942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 -- 172.21.15.42:0/2344892429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 msgr2=0x5572334947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x5572334947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 -- 172.21.15.42:0/2344892429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572333ebb90 msgr2=0x5572334942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:21.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572333ebb90 0x5572334942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74f4e00700 1 -- 172.21.15.42:0/2344892429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557233503a30 con 0x5572332fffe0 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.748+0000 7f74efe00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572333ebb90 0x5572334942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.749+0000 7f74f4e00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233493d70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f74e4022040 tx=0x7f74e401f760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4bce10f85c75746 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.749+0000 7f74d6a00700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.749+0000 7f74d6a00700 1 -- 172.21.15.42:0/2344892429 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e401fd60 con 0x5572332fffe0 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.749+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74dc0215c0 con 0x5572332fffe0 2024-05-27T10:26:21.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.749+0000 7f74d6a00700 1 -- 172.21.15.42:0/2344892429 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f74e400ea50 con 0x5572332fffe0 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.750+0000 7f74d6a00700 1 -- 172.21.15.42:0/2344892429 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74e400ec50 con 0x5572332fffe0 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.750+0000 7f7506419b80 10 client.36170 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.750+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557233504b30 con 0x5572332fffe0 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.750+0000 7f74d6a00700 1 -- 172.21.15.42:0/2344892429 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74dc0215c0 con 0x5572332fffe0 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f74d6a00700 1 -- 172.21.15.42:0/2344892429 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f74e4032420 con 0x5572332fffe0 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f7506419b80 10 client.36170 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f7506419b80 10 client.36170 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f7506419b80 20 client.36170 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f7506419b80 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233505280 0x5572335076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f7506419b80 4 client.36170 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.95, "await_for": 45.0}] 2024-05-27T10:26:21.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.751+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.95, "await_for": 45.0}) -- 0x5572334728b0 con 0x557233505280 2024-05-27T10:26:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.752+0000 7f74efe00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233505280 0x5572335076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.753+0000 7f74efe00700 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233505280 0x5572335076a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d72d85007e43974 server_cookie=617cb61eff4383b in_seq=0 out_seq=0 2024-05-27T10:26:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:21.753+0000 7f74d6a00700 10 client.36170 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:23.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:22 smithi042 ceph-mon[31661]: pgmap v234: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-05-27T10:26:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:22 smithi104 ceph-mon[34812]: pgmap v234: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-05-27T10:26:23.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:22 smithi143 ceph-mon[35545]: pgmap v234: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-05-27T10:26:25.094 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:24 smithi042 ceph-mon[31661]: pgmap v235: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-05-27T10:26:25.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:24 smithi104 ceph-mon[34812]: pgmap v235: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-05-27T10:26:25.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:24 smithi143 ceph-mon[35545]: pgmap v235: 97 pgs: 97 active+clean; 328 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-05-27T10:26:25.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.662+0000 7f74d6a00700 1 -- 172.21.15.42:0/2344892429 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5572334728b0 con 0x557233505280 2024-05-27T10:26:25.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.662+0000 7f74d6a00700 10 client.36170 handle_command_reply: tid=0 2024-05-27T10:26:25.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.662+0000 7f7506419b80 1 client.36170 shutdown 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 msgr2=0x557233493d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233493d70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f74e4022040 tx=0x7f74e401f760 comp rx=0 tx=0).stop 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 shutdown_connections 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557233505280 0x5572335076a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55723347f680 0x5572334947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572332fffe0 0x557233493d70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.663+0000 7f7506419b80 1 --2- 172.21.15.42:0/2344892429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572333ebb90 0x5572334942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.664+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 >> 172.21.15.42:0/2344892429 conn(0x557233399530 msgr2=0x55723347e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:25.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.664+0000 7f74d6a00700 0 client.36170 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:25.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.664+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 shutdown_connections 2024-05-27T10:26:25.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.664+0000 7f7506419b80 1 -- 172.21.15.42:0/2344892429 wait complete. 2024-05-27T10:26:25.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.665+0000 7f7506419b80 20 client.36170 trim_cache size 0 max 16384 2024-05-27T10:26:25.664 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:26:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 33, 2024-05-27T10:26:25.664 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 22, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "27e9bff1": { 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 22, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 138.9, 2024-05-27T10:26:25.665 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:25.666 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f74d80b2aa0 msgr2=0x7f74d80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f74d80b2aa0 0x7f74d80b4fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74f01079d0 tx=0x7f74e4009260 comp rx=0 tx=0).stop 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106eb0 msgr2=0x7f74f0110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106eb0 0x7f74f0110c90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f74dc000c00 tx=0x7f74dc0057c0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 shutdown_connections 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f74d80b2aa0 0x7f74d80b4fb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74f010c5a0 0x7f74f01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74f0106560 0x7f74f0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.667+0000 7f74f7800700 1 --2- 172.21.15.42:0/3850300530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74f0106eb0 0x7f74f0110c90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:25.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.668+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 >> 172.21.15.42:0/3850300530 conn(0x7f74f0102040 msgr2=0x7f74f01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:25.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.669+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 shutdown_connections 2024-05-27T10:26:25.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:25.669+0000 7f74f7800700 1 -- 172.21.15.42:0/3850300530 wait complete. 2024-05-27T10:26:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.838+0000 7f952e400640 1 -- 172.21.15.42:0/2268348411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9528108b90 msgr2=0x7f9528102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.838+0000 7f952e400640 1 --2- 172.21.15.42:0/2268348411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9528108b90 0x7f9528102730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9518009900 tx=0x7f951802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 -- 172.21.15.42:0/2268348411 shutdown_connections 2024-05-27T10:26:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 --2- 172.21.15.42:0/2268348411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95281032e0 0x7f952810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 --2- 172.21.15.42:0/2268348411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528109030 0x7f9528102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 --2- 172.21.15.42:0/2268348411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9528108b90 0x7f9528102730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 -- 172.21.15.42:0/2268348411 >> 172.21.15.42:0/2268348411 conn(0x7f95280782d0 msgr2=0x7f9528078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 -- 172.21.15.42:0/2268348411 shutdown_connections 2024-05-27T10:26:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.839+0000 7f952e400640 1 -- 172.21.15.42:0/2268348411 wait complete. 2024-05-27T10:26:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.841+0000 7f952e400640 1 Processor -- start 2024-05-27T10:26:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.841+0000 7f952e400640 1 -- start start 2024-05-27T10:26:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f952e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 0x7f9528198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f952e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9528108b90 0x7f9528198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f952e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528109030 0x7f952819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f952e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111b70 con 0x7f9528108b90 2024-05-27T10:26:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f952e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95281119f0 con 0x7f95281032e0 2024-05-27T10:26:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f952e400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111cf0 con 0x7f9528109030 2024-05-27T10:26:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f9527400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 0x7f9528198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f9527400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 0x7f9528198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:54006/0 (socket says 172.21.15.42:54006) 2024-05-27T10:26:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f9527400640 1 -- 172.21.15.42:0/3602254983 learned_addr learned my addr 172.21.15.42:0/3602254983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.842+0000 7f9526a00640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9528108b90 0x7f9528198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.843+0000 7f9527e00640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528109030 0x7f952819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.843+0000 7f9527400640 1 -- 172.21.15.42:0/3602254983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528109030 msgr2=0x7f952819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.843+0000 7f9527400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528109030 0x7f952819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.843+0000 7f9527400640 1 -- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9528108b90 msgr2=0x7f9528198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.843+0000 7f9527400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9528108b90 0x7f9528198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.843+0000 7f9527400640 1 -- 172.21.15.42:0/3602254983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952819d4b0 con 0x7f95281032e0 2024-05-27T10:26:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.844+0000 7f9527400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 0x7f9528198650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f951802a910 tx=0x7f9518003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.847+0000 7f951fe00640 1 -- 172.21.15.42:0/3602254983 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951803f070 con 0x7f95281032e0 2024-05-27T10:26:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.847+0000 7f951fe00640 1 -- 172.21.15.42:0/3602254983 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f951803baf0 con 0x7f95281032e0 2024-05-27T10:26:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.847+0000 7f951fe00640 1 -- 172.21.15.42:0/3602254983 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518035030 con 0x7f95281032e0 2024-05-27T10:26:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.847+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952819d740 con 0x7f95281032e0 2024-05-27T10:26:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.847+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952819dbe0 con 0x7f95281032e0 2024-05-27T10:26:26.847 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27e9bff1', quiesce took 13.2 seconds. Will release after: 5.75 2024-05-27T10:26:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.851+0000 7f95353af740 1 Processor -- start 2024-05-27T10:26:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 -- start start 2024-05-27T10:26:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366029860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x565366029da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x56536602a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565365f9ab90 con 0x565365c664d0 2024-05-27T10:26:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565365f8a730 con 0x565365c871b0 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.852+0000 7f95353af740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x565365f8e770 con 0x565365c5e2f0 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.853+0000 7f9526a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x56536602a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.853+0000 7f9526a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x56536602a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44532/0 (socket says 172.21.15.42:44532) 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.853+0000 7f9526a00640 1 -- 172.21.15.42:0/3278676724 learned_addr learned my addr 172.21.15.42:0/3278676724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.853+0000 7f9527e00640 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366029860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f9526a00640 1 -- 172.21.15.42:0/3278676724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 msgr2=0x565366029da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f9526a00640 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x565366029da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f9526a00640 1 -- 172.21.15.42:0/3278676724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 msgr2=0x565366029860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f9526a00640 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366029860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f9526a00640 1 -- 172.21.15.42:0/3278676724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565365f238b0 con 0x565365c664d0 2024-05-27T10:26:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f951fe00640 1 -- 172.21.15.42:0/3602254983 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9518042070 con 0x7f95281032e0 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.854+0000 7f9526a00640 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x56536602a2e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9514009b30 tx=0x7f95140113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e3eae653173be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.855+0000 7f951cc00640 1 -- 172.21.15.42:0/3278676724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514011e10 con 0x565365c664d0 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.855+0000 7f951cc00640 1 -- 172.21.15.42:0/3278676724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9514016070 con 0x565365c664d0 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.855+0000 7f951cc00640 1 -- 172.21.15.42:0/3278676724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951401b470 con 0x565365c664d0 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 -- 172.21.15.42:0/3278676724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 msgr2=0x56536602a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x56536602a2e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9514009b30 tx=0x7f95140113e0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f951fe00640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f950c0b2a40 0x7f950c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f951fe00640 1 -- 172.21.15.42:0/3602254983 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f9518102d00 con 0x7f95281032e0 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 -- 172.21.15.42:0/3278676724 shutdown_connections 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x56536602a2e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x565366029da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 --2- 172.21.15.42:0/3278676724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366029860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 -- 172.21.15.42:0/3278676724 >> 172.21.15.42:0/3278676724 conn(0x565365c7b300 msgr2=0x565366026010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f9527e00640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f950c0b2a40 0x7f950c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 -- 172.21.15.42:0/3278676724 shutdown_connections 2024-05-27T10:26:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.856+0000 7f95353af740 1 -- 172.21.15.42:0/3278676724 wait complete. 2024-05-27T10:26:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.857+0000 7f9527e00640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f950c0b2a40 0x7f950c0b4f50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x5653660269d0 tx=0x7f9520009500 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.857+0000 7f95353af740 1 -- start start 2024-05-27T10:26:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 Processor -- start 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x565366033f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x56536603fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565365c70790 con 0x565365c664d0 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565365f32bc0 con 0x565365c871b0 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x565366036990 con 0x565365c5e2f0 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.858+0000 7f95353af740 10 client.0 _ll_register_callbacks cb 0x565365f10400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.859+0000 7f9526a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x565366033f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.859+0000 7f9526a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x565366033f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44548/0 (socket says 172.21.15.42:44548) 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.859+0000 7f9526a00640 1 -- 172.21.15.42:0/3936126047 learned_addr learned my addr 172.21.15.42:0/3936126047 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.859+0000 7f9527400640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.860+0000 7f9527e00640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x56536603fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.860+0000 7f9526a00640 1 -- 172.21.15.42:0/3936126047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 msgr2=0x5653660339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.860+0000 7f9526a00640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.860+0000 7f9526a00640 1 -- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 msgr2=0x56536603fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.860+0000 7f9526a00640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x56536603fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.860+0000 7f9526a00640 1 -- 172.21.15.42:0/3936126047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565365c9f210 con 0x565365c664d0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.861+0000 7f9526a00640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x565366033f10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9514009b00 tx=0x7f9514003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc1154d9bba98f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.861+0000 7f9511600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.861+0000 7f9511600640 1 -- 172.21.15.42:0/3936126047 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514011ea0 con 0x565365c664d0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.861+0000 7f9511600640 1 -- 172.21.15.42:0/3936126047 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f9514004250 con 0x565365c664d0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.862+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9514016070 con 0x565365c664d0 2024-05-27T10:26:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.862+0000 7f9511600640 1 -- 172.21.15.42:0/3936126047 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9514004450 con 0x565365c664d0 2024-05-27T10:26:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.862+0000 7f95353af740 10 client.16659 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.862+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565365d6b620 con 0x565365c664d0 2024-05-27T10:26:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f9511600640 1 -- 172.21.15.42:0/3936126047 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9514016070 con 0x565365c664d0 2024-05-27T10:26:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f9511600640 1 -- 172.21.15.42:0/3936126047 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9514004c10 con 0x565365c664d0 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f95353af740 10 client.16659 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f95353af740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f95353af740 20 client.16659 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f95353af740 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c45d80 0x565366045b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f95353af740 4 client.16659 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.863+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565365f9cae0 con 0x565365c45d80 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.864+0000 7f9527400640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c45d80 0x565366045b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.865+0000 7f9527400640 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c45d80 0x565366045b30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce2424dc155843d server_cookie=7ed5ab4400691800 in_seq=0 out_seq=0 2024-05-27T10:26:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.865+0000 7f9511600640 10 client.16659 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f9511600640 1 -- 172.21.15.42:0/3936126047 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565365f9cae0 con 0x565365c45d80 2024-05-27T10:26:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f9511600640 10 client.16659 handle_command_reply: tid=0 2024-05-27T10:26:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 client.16659 shutdown 2024-05-27T10:26:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 msgr2=0x565366033f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x565366033f10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9514009b00 tx=0x7f9514003f30 comp rx=0 tx=0).stop 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 shutdown_connections 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x56536603fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c664d0 0x565366033f10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.867+0000 7f95353af740 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c45d80 0x565366045b30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.868+0000 7f9511600640 0 client.16659 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.868+0000 7f95353af740 1 --2- 172.21.15.42:0/3936126047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.868+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 >> 172.21.15.42:0/3936126047 conn(0x565365c7b300 msgr2=0x56536602b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.868+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 shutdown_connections 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.868+0000 7f95353af740 1 -- 172.21.15.42:0/3936126047 wait complete. 2024-05-27T10:26:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.868+0000 7f95353af740 20 client.16659 trim_cache size 0 max 16384 2024-05-27T10:26:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.895+0000 7f95353af740 1 Processor -- start 2024-05-27T10:26:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.895+0000 7f95353af740 1 -- start start 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.895+0000 7f95353af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366035bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.895+0000 7f95353af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c5e2f0 0x56536602f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.895+0000 7f95353af740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c871b0 0x56536602a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f95353af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b8a30 con 0x565365c45d80 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f95353af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b62a0 con 0x565365c5e2f0 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f95353af740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b40d0 con 0x565365c871b0 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f9527400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366035bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f9527400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366035bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44562/0 (socket says 172.21.15.42:44562) 2024-05-27T10:26:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f9527400640 1 -- 172.21.15.42:0/3552926322 learned_addr learned my addr 172.21.15.42:0/3552926322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f9527e00640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c871b0 0x56536602a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.896+0000 7f9526a00640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c5e2f0 0x56536602f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527400640 1 -- 172.21.15.42:0/3552926322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c871b0 msgr2=0x56536602a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527400640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c871b0 0x56536602a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527400640 1 -- 172.21.15.42:0/3552926322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c5e2f0 msgr2=0x56536602f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527400640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c5e2f0 0x56536602f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527400640 1 -- 172.21.15.42:0/3552926322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565366031f20 con 0x565365c45d80 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527e00640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c871b0 0x56536602a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9526a00640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c5e2f0 0x56536602f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9527400640 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366035bc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9518100470 tx=0x7f9518003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae58c205d12e5fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9513e00640 1 -- 172.21.15.42:0/3552926322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518049040 con 0x565365c45d80 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9513e00640 1 -- 172.21.15.42:0/3552926322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95180c9fa0 con 0x565365c45d80 2024-05-27T10:26:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.897+0000 7f9513e00640 1 -- 172.21.15.42:0/3552926322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518100690 con 0x565365c45d80 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 -- 172.21.15.42:0/3552926322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 msgr2=0x565366035bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366035bc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9518100470 tx=0x7f9518003f90 comp rx=0 tx=0).stop 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 -- 172.21.15.42:0/3552926322 shutdown_connections 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c871b0 0x56536602a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c5e2f0 0x56536602f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 --2- 172.21.15.42:0/3552926322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366035bc0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 -- 172.21.15.42:0/3552926322 >> 172.21.15.42:0/3552926322 conn(0x565365c7b300 msgr2=0x565366033940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 -- 172.21.15.42:0/3552926322 shutdown_connections 2024-05-27T10:26:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.899+0000 7f95353af740 1 -- 172.21.15.42:0/3552926322 wait complete. 2024-05-27T10:26:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.900+0000 7f95353af740 1 -- start start 2024-05-27T10:26:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.900+0000 7f95353af740 1 Processor -- start 2024-05-27T10:26:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366049070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366044900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b4760 con 0x565365c45d80 2024-05-27T10:26:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b5bd0 con 0x565365c871b0 2024-05-27T10:26:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b4e30 con 0x565365c5e2f0 2024-05-27T10:26:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f95353af740 10 client.0 _ll_register_callbacks cb 0x7f9520009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f9527400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366049070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f9527400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366049070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44570/0 (socket says 172.21.15.42:44570) 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f9527400640 1 -- 172.21.15.42:0/6890000 learned_addr learned my addr 172.21.15.42:0/6890000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f9526a00640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.901+0000 7f9527e00640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366044900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527400640 1 -- 172.21.15.42:0/6890000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 msgr2=0x5653660443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527400640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527400640 1 -- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 msgr2=0x565366044900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527400640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366044900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527400640 1 -- 172.21.15.42:0/6890000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565366044e40 con 0x565365c45d80 2024-05-27T10:26:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527e00640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366044900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9527400640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366049070 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f95180c9ca0 tx=0x7f95181003e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2100a4985e6a0116 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.902+0000 7f9511600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f9511600640 1 -- 172.21.15.42:0/6890000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95180a9060 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f9511600640 1 -- 172.21.15.42:0/6890000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f951803bca0 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95180c9fa0 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f9511600640 1 -- 172.21.15.42:0/6890000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9518049070 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f95353af740 10 client.16665 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5653660adc70 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.903+0000 7f9511600640 1 -- 172.21.15.42:0/6890000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95180c9fa0 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f9511600640 1 -- 172.21.15.42:0/6890000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f95180d2a00 con 0x565365c45d80 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f95353af740 10 client.16665 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f95353af740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f95353af740 20 client.16665 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f95353af740 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c1b1c0 0x5653660b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f95353af740 4 client.16665 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:26:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565365f962c0 con 0x565365c1b1c0 2024-05-27T10:26:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.904+0000 7f9526a00640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c1b1c0 0x5653660b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.906+0000 7f9526a00640 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c1b1c0 0x5653660b0330 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487cb126c2b33994 server_cookie=ac01a5e79062290b in_seq=0 out_seq=0 2024-05-27T10:26:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.906+0000 7f9511600640 10 client.16665 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f9511600640 1 -- 172.21.15.42:0/6890000 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565365f962c0 con 0x565365c1b1c0 2024-05-27T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f9511600640 10 client.16665 handle_command_reply: tid=0 2024-05-27T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 client.16665 shutdown 2024-05-27T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 msgr2=0x565366049070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366049070 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f95180c9ca0 tx=0x7f95181003e0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 shutdown_connections 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565365c871b0 0x565366044900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565365c1b1c0 0x5653660b0330 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565365c5e2f0 0x5653660443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 --2- 172.21.15.42:0/6890000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565365c45d80 0x565366049070 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 >> 172.21.15.42:0/6890000 conn(0x565365c7b300 msgr2=0x565366032f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.956+0000 7f9511600640 0 client.16665 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.957+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 shutdown_connections 2024-05-27T10:26:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.957+0000 7f95353af740 1 -- 172.21.15.42:0/6890000 wait complete. 2024-05-27T10:26:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.957+0000 7f95353af740 20 client.16665 trim_cache size 0 max 16384 2024-05-27T10:26:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:26:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:26:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.958+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f950c0b2a40 msgr2=0x7f950c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.958+0000 7f952e400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f950c0b2a40 0x7f950c0b4f50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x5653660269d0 tx=0x7f9520009500 comp rx=0 tx=0).stop 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.958+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 msgr2=0x7f9528198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.958+0000 7f952e400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 0x7f9528198650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f951802a910 tx=0x7f9518003b90 comp rx=0 tx=0).stop 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.958+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 shutdown_connections 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.958+0000 7f952e400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f950c0b2a40 0x7f950c0b4f50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.959+0000 7f952e400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528109030 0x7f952819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.959+0000 7f952e400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9528108b90 0x7f9528198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.959+0000 7f952e400640 1 --2- 172.21.15.42:0/3602254983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95281032e0 0x7f9528198650 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.959+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 >> 172.21.15.42:0/3602254983 conn(0x7f95280782d0 msgr2=0x7f9528104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.959+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 shutdown_connections 2024-05-27T10:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:26.959+0000 7f952e400640 1 -- 172.21.15.42:0/3602254983 wait complete. 2024-05-27T10:26:27.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:26 smithi042 ceph-mon[31661]: pgmap v236: 97 pgs: 97 active+clean; 338 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 229 op/s 2024-05-27T10:26:27.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:26 smithi104 ceph-mon[34812]: pgmap v236: 97 pgs: 97 active+clean; 338 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 229 op/s 2024-05-27T10:26:27.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:26 smithi143 ceph-mon[35545]: pgmap v236: 97 pgs: 97 active+clean; 338 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 229 op/s 2024-05-27T10:26:28.717 INFO:tasks.workunit.client.0.smithi042.stdout: 47 3786 8 3039 134 843 93 2024-05-27T10:26:29.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:28 smithi042 ceph-mon[31661]: pgmap v237: 97 pgs: 97 active+clean; 338 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-05-27T10:26:29.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:28 smithi104 ceph-mon[34812]: pgmap v237: 97 pgs: 97 active+clean; 338 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-05-27T10:26:29.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:28 smithi143 ceph-mon[35545]: pgmap v237: 97 pgs: 97 active+clean; 338 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-05-27T10:26:31.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:30 smithi104 ceph-mon[34812]: pgmap v238: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-27T10:26:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:30 smithi143 ceph-mon[35545]: pgmap v238: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-27T10:26:31.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:30 smithi042 ceph-mon[31661]: pgmap v238: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-27T10:26:32.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:26:33.089 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:26:33.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:32 smithi042 ceph-mon[31661]: pgmap v239: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-27T10:26:33.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:32 smithi104 ceph-mon[34812]: pgmap v239: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-27T10:26:33.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:32 smithi143 ceph-mon[35545]: pgmap v239: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-27T10:26:35.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:34 smithi042 ceph-mon[31661]: pgmap v240: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-27T10:26:35.110 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:34 smithi143 ceph-mon[35545]: pgmap v240: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-27T10:26:35.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:34 smithi104 ceph-mon[34812]: pgmap v240: 97 pgs: 97 active+clean; 342 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-27T10:26:37.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:36 smithi104 ceph-mon[34812]: pgmap v241: 97 pgs: 97 active+clean; 328 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.4 MiB/s wr, 199 op/s 2024-05-27T10:26:37.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:36 smithi143 ceph-mon[35545]: pgmap v241: 97 pgs: 97 active+clean; 328 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.4 MiB/s wr, 199 op/s 2024-05-27T10:26:37.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:36 smithi042 ceph-mon[31661]: pgmap v241: 97 pgs: 97 active+clean; 328 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.4 MiB/s wr, 199 op/s 2024-05-27T10:26:38.717 INFO:tasks.workunit.client.0.smithi042.stdout: 47 1591 0 1323 0 631 0 2024-05-27T10:26:39.031 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:38 smithi042 ceph-mon[31661]: pgmap v242: 97 pgs: 97 active+clean; 328 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 568 KiB/s wr, 109 op/s 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 -- 172.21.15.42:0/2772534588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc1045e0 msgr2=0x7f89fc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 --2- 172.21.15.42:0/2772534588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc1045e0 0x7f89fc1049f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f89f00099c0 tx=0x7f89f0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 -- 172.21.15.42:0/2772534588 shutdown_connections 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 --2- 172.21.15.42:0/2772534588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc109860 0x7f89fc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 --2- 172.21.15.42:0/2772534588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc1045e0 0x7f89fc1049f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 --2- 172.21.15.42:0/2772534588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89fc108f10 0x7f89fc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 -- 172.21.15.42:0/2772534588 >> 172.21.15.42:0/2772534588 conn(0x7f89fc100030 msgr2=0x7f89fc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 -- 172.21.15.42:0/2772534588 shutdown_connections 2024-05-27T10:26:39.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.031+0000 7f8a04000700 1 -- 172.21.15.42:0/2772534588 wait complete. 2024-05-27T10:26:39.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.033+0000 7f8a04000700 1 Processor -- start 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.033+0000 7f8a04000700 1 -- start start 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.033+0000 7f8a04000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc1045e0 0x7f89fc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.033+0000 7f8a04000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 0x7f89fc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a04000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89fc109860 0x7f89fc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a04000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc114390 con 0x7f89fc108f10 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a04000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc114210 con 0x7f89fc109860 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a04000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc114510 con 0x7f89fc1045e0 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a00c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 0x7f89fc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a00c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 0x7f89fc19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48806/0 (socket says 172.21.15.42:48806) 2024-05-27T10:26:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a00c00700 1 -- 172.21.15.42:0/2472619249 learned_addr learned my addr 172.21.15.42:0/2472619249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:39.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a01600700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc1045e0 0x7f89fc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:39.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.034+0000 7f8a02000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89fc109860 0x7f89fc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a00c00700 1 -- 172.21.15.42:0/2472619249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc1045e0 msgr2=0x7f89fc19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a00c00700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc1045e0 0x7f89fc19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a00c00700 1 -- 172.21.15.42:0/2472619249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89fc109860 msgr2=0x7f89fc19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a00c00700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89fc109860 0x7f89fc19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a00c00700 1 -- 172.21.15.42:0/2472619249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc19fc70 con 0x7f89fc108f10 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a01600700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc1045e0 0x7f89fc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a00c00700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 0x7f89fc19b340 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f89e800d980 tx=0x7f89e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8017070 con 0x7f89fc108f10 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc19ff60 con 0x7f89fc108f10 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89e8010460 con 0x7f89fc108f10 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8020060 con 0x7f89fc108f10 2024-05-27T10:26:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.035+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a7880 con 0x7f89fc108f10 2024-05-27T10:26:39.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.037+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89fc0512b0 con 0x7f89fc108f10 2024-05-27T10:26:39.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.039+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f89e8010610 con 0x7f89fc108f10 2024-05-27T10:26:39.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.042+0000 7f89fa000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89e40b29f0 0x7f89e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:39.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.042+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f89e80e2250 con 0x7f89fc108f10 2024-05-27T10:26:39.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.042+0000 7f8a01600700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89e40b29f0 0x7f89e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:39.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.043+0000 7f8a01600700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89e40b29f0 0x7f89e40b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89f0000c00 tx=0x7f89f0005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:39.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.047+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89e80a9bf0 con 0x7f89fc108f10 2024-05-27T10:26:39.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:38 smithi104 ceph-mon[34812]: pgmap v242: 97 pgs: 97 active+clean; 328 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 568 KiB/s wr, 109 op/s 2024-05-27T10:26:39.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:38 smithi143 ceph-mon[35545]: pgmap v242: 97 pgs: 97 active+clean; 328 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 568 KiB/s wr, 109 op/s 2024-05-27T10:26:39.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.333+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89fc19c100 con 0x7f89fc108f10 2024-05-27T10:26:39.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.337+0000 7f89fa000700 1 -- 172.21.15.42:0/2472619249 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11881 (secure 0 0 0) 0x7f89e80a94f0 con 0x7f89fc108f10 2024-05-27T10:26:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:26:39.337 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:24:00.498568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:26:39.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89e40b29f0 msgr2=0x7f89e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89e40b29f0 0x7f89e40b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89f0000c00 tx=0x7f89f0005fd0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 msgr2=0x7f89fc19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 0x7f89fc19b340 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f89e800d980 tx=0x7f89e800dd40 comp rx=0 tx=0).stop 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 shutdown_connections 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f89e40b29f0 0x7f89e40b4f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89fc109860 0x7f89fc19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89fc1045e0 0x7f89fc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 --2- 172.21.15.42:0/2472619249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89fc108f10 0x7f89fc19b340 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:39.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 >> 172.21.15.42:0/2472619249 conn(0x7f89fc100030 msgr2=0x7f89fc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:39.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 shutdown_connections 2024-05-27T10:26:39.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:39.340+0000 7f8a04000700 1 -- 172.21.15.42:0/2472619249 wait complete. 2024-05-27T10:26:39.341 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 33 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 -- 172.21.15.42:0/872358400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 msgr2=0x7f75d0103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 --2- 172.21.15.42:0/872358400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0103b90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009900 tx=0x7f75b802a410 comp rx=0 tx=0).stop 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 -- 172.21.15.42:0/872358400 shutdown_connections 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 --2- 172.21.15.42:0/872358400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 0x7f75d01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 --2- 172.21.15.42:0/872358400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0103b90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 --2- 172.21.15.42:0/872358400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d0109ab0 0x7f75d0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 -- 172.21.15.42:0/872358400 >> 172.21.15.42:0/872358400 conn(0x7f75d0079140 msgr2=0x7f75d0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 -- 172.21.15.42:0/872358400 shutdown_connections 2024-05-27T10:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.197+0000 7f75d6400640 1 -- 172.21.15.42:0/872358400 wait complete. 2024-05-27T10:26:40.198 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:39 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2472619249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.198+0000 7f75d6400640 1 Processor -- start 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.198+0000 7f75d6400640 1 -- start start 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.198+0000 7f75d6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 0x7f75d0074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.198+0000 7f75d6400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d0109ab0 0x7f75d0075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.198+0000 7f75d6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d6400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0113ba0 con 0x7f75d0109f50 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0113a20 con 0x7f75d0104200 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d6400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0113d20 con 0x7f75d0109ab0 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0075a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48828/0 (socket says 172.21.15.42:48828) 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 -- 172.21.15.42:0/3259672227 learned_addr learned my addr 172.21.15.42:0/3259672227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5000640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 0x7f75d0074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75cfe00640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d0109ab0 0x7f75d0075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 -- 172.21.15.42:0/3259672227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d0109ab0 msgr2=0x7f75d0075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d0109ab0 0x7f75d0075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 -- 172.21.15.42:0/3259672227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 msgr2=0x7f75d0074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 0x7f75d0074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5a00640 1 -- 172.21.15.42:0/3259672227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a89b0 con 0x7f75d0109f50 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.199+0000 7f75d5000640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 0x7f75d0074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.200+0000 7f75d5a00640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0075a40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f75c800eaa0 tx=0x7f75c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.200+0000 7f75cd600640 1 -- 172.21.15.42:0/3259672227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c800cdb0 con 0x7f75d0109f50 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.200+0000 7f75cd600640 1 -- 172.21.15.42:0/3259672227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c8005020 con 0x7f75d0109f50 2024-05-27T10:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.200+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a8ca0 con 0x7f75d0109f50 2024-05-27T10:26:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.200+0000 7f75cd600640 1 -- 172.21.15.42:0/3259672227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80116d0 con 0x7f75d0109f50 2024-05-27T10:26:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.200+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a9170 con 0x7f75d0109f50 2024-05-27T10:26:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.204+0000 7f75de971740 1 Processor -- start 2024-05-27T10:26:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.204+0000 7f75cd600640 1 -- 172.21.15.42:0/3259672227 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f75c80051d0 con 0x7f75d0109f50 2024-05-27T10:26:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.204+0000 7f75de971740 1 -- start start 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75de971740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e69857a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75de971740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 0x557e69857fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75de971740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694944d0 0x557e698584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75de971740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557e697c8b90 con 0x557e6948c2f0 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75de971740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557e697b8730 con 0x557e694944d0 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75de971740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e697bc770 con 0x557e694b51b0 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75d5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 0x557e69857fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75d5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 0x557e69857fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48832/0 (socket says 172.21.15.42:48832) 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75d5000640 1 -- 172.21.15.42:0/2022315939 learned_addr learned my addr 172.21.15.42:0/2022315939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75cfe00640 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e69857a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.205+0000 7f75cfe00640 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694944d0 0x557e698584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75d5000640 1 -- 172.21.15.42:0/2022315939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 msgr2=0x557e69857a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75d5000640 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e69857a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75d5000640 1 -- 172.21.15.42:0/2022315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694944d0 msgr2=0x557e698584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75d5000640 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694944d0 0x557e698584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75d5000640 1 -- 172.21.15.42:0/2022315939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e697518b0 con 0x557e6948c2f0 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75cd600640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f75b40b2950 0x7f75b40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75d5000640 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 0x557e69857fa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f75c000bcb0 tx=0x7f75c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145ab20c4b5433a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75cd600640 1 -- 172.21.15.42:0/3259672227 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f75c80e2900 con 0x7f75d0109f50 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75c6000640 1 -- 172.21.15.42:0/2022315939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0015070 con 0x557e6948c2f0 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.206+0000 7f75cfe00640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f75b40b2950 0x7f75b40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.207+0000 7f75c6000640 1 -- 172.21.15.42:0/2022315939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c00054b0 con 0x557e6948c2f0 2024-05-27T10:26:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.207+0000 7f75c6000640 1 -- 172.21.15.42:0/2022315939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0011440 con 0x557e6948c2f0 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.207+0000 7f75de971740 1 -- 172.21.15.42:0/2022315939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 msgr2=0x557e69857fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.207+0000 7f75cfe00640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f75b40b2950 0x7f75b40b4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75b802a930 tx=0x7f75b8000bc0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.207+0000 7f75de971740 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 0x557e69857fa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f75c000bcb0 tx=0x7f75c000f730 comp rx=0 tx=0).stop 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 -- 172.21.15.42:0/2022315939 shutdown_connections 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694944d0 0x557e698584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e6948c2f0 0x557e69857fa0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 --2- 172.21.15.42:0/2022315939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e69857a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 -- 172.21.15.42:0/2022315939 >> 172.21.15.42:0/2022315939 conn(0x557e694a9300 msgr2=0x557e69854210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 -- 172.21.15.42:0/2022315939 shutdown_connections 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 -- 172.21.15.42:0/2022315939 wait complete. 2024-05-27T10:26:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.208+0000 7f75de971740 1 -- start start 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 Processor -- start 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e69861be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e69862120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e6986def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557e6949e790 con 0x557e694944d0 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557e69760bc0 con 0x557e6948c2f0 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e69864b90 con 0x557e694b51b0 2024-05-27T10:26:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.209+0000 7f75de971740 10 client.0 _ll_register_callbacks cb 0x557e6973e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e69862120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e69862120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48848/0 (socket says 172.21.15.42:48848) 2024-05-27T10:26:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 -- 172.21.15.42:0/3745741862 learned_addr learned my addr 172.21.15.42:0/3745741862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75d5000640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e69861be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75d5a00640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e6986def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 -- 172.21.15.42:0/3745741862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 msgr2=0x557e6986def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e6986def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 -- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 msgr2=0x557e69861be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e69861be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.210+0000 7f75cfe00640 1 -- 172.21.15.42:0/3745741862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e694cd210 con 0x557e694944d0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75cfe00640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e69862120 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005760 tx=0x7f75b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b67d5a96e7db3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75bea00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75bea00640 1 -- 172.21.15.42:0/3745741862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b804f070 con 0x557e694944d0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75bea00640 1 -- 172.21.15.42:0/3745741862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f75b804b890 con 0x557e694944d0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c00054b0 con 0x557e694944d0 2024-05-27T10:26:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75bea00640 1 -- 172.21.15.42:0/3745741862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f75b805b050 con 0x557e694944d0 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75de971740 10 client.16683 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.211+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e69599620 con 0x557e694944d0 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75bea00640 1 -- 172.21.15.42:0/3745741862 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c00054b0 con 0x557e694944d0 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75bea00640 1 -- 172.21.15.42:0/3745741862 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f75b8048050 con 0x557e694944d0 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75de971740 10 client.16683 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75de971740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75de971740 20 client.16683 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75de971740 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e69473d80 0x557e69873e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75de971740 4 client.16683 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.212+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e697caae0 con 0x557e69473d80 2024-05-27T10:26:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.213+0000 7f75d5000640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e69473d80 0x557e69873e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.214+0000 7f75d5000640 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e69473d80 0x557e69873e90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b1795f826949f5 server_cookie=56306daf3221e23d in_seq=0 out_seq=0 2024-05-27T10:26:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.214+0000 7f75bea00640 10 client.16683 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75bea00640 1 -- 172.21.15.42:0/3745741862 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557e697caae0 con 0x557e69473d80 2024-05-27T10:26:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75bea00640 10 client.16683 handle_command_reply: tid=0 2024-05-27T10:26:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 client.16683 shutdown 2024-05-27T10:26:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 msgr2=0x557e69862120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e69862120 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005760 tx=0x7f75b8009260 comp rx=0 tx=0).stop 2024-05-27T10:26:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 shutdown_connections 2024-05-27T10:26:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e69473d80 0x557e69873e90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694b51b0 0x557e6986def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e69862120 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 --2- 172.21.15.42:0/3745741862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e69861be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75bea00640 0 client.16683 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 >> 172.21.15.42:0/3745741862 conn(0x557e694a9300 msgr2=0x557e69857890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 shutdown_connections 2024-05-27T10:26:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 1 -- 172.21.15.42:0/3745741862 wait complete. 2024-05-27T10:26:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.216+0000 7f75de971740 20 client.16683 trim_cache size 0 max 16384 2024-05-27T10:26:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.233+0000 7f75de971740 1 Processor -- start 2024-05-27T10:26:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.233+0000 7f75de971740 1 -- start start 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75de971740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6948c2f0 0x557e69864310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75de971740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e6986dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75de971740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694b51b0 0x557e6986e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75de971740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01b42b0 con 0x557e694944d0 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75de971740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01b1b20 con 0x557e694b51b0 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75de971740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01af990 con 0x557e6948c2f0 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e6986dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e6986dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48860/0 (socket says 172.21.15.42:48860) 2024-05-27T10:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.234+0000 7f75cfe00640 1 -- 172.21.15.42:0/395727858 learned_addr learned my addr 172.21.15.42:0/395727858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75d5a00640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694b51b0 0x557e6986e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75d5000640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6948c2f0 0x557e69864310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75cfe00640 1 -- 172.21.15.42:0/395727858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6948c2f0 msgr2=0x557e69864310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75cfe00640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6948c2f0 0x557e69864310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75cfe00640 1 -- 172.21.15.42:0/395727858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694b51b0 msgr2=0x557e6986e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75cfe00640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694b51b0 0x557e6986e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75cfe00640 1 -- 172.21.15.42:0/395727858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e69854e00 con 0x557e694944d0 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75d5a00640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694b51b0 0x557e6986e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75d5000640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6948c2f0 0x557e69864310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75cfe00640 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e6986dc60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f75b803db70 tx=0x7f75b802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6cd69d9f4ea42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.235+0000 7f75c5600640 1 -- 172.21.15.42:0/395727858 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b804f040 con 0x557e694944d0 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.236+0000 7f75c5600640 1 -- 172.21.15.42:0/395727858 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75b803d8e0 con 0x557e694944d0 2024-05-27T10:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.236+0000 7f75c5600640 1 -- 172.21.15.42:0/395727858 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8035040 con 0x557e694944d0 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.236+0000 7f75de971740 1 -- 172.21.15.42:0/395727858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 msgr2=0x557e6986dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.236+0000 7f75de971740 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e6986dc60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f75b803db70 tx=0x7f75b802ab10 comp rx=0 tx=0).stop 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 -- 172.21.15.42:0/395727858 shutdown_connections 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e694b51b0 0x557e6986e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694944d0 0x557e6986dc60 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 --2- 172.21.15.42:0/395727858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6948c2f0 0x557e69864310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 -- 172.21.15.42:0/395727858 >> 172.21.15.42:0/395727858 conn(0x557e694a9300 msgr2=0x557e69861ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 -- 172.21.15.42:0/395727858 shutdown_connections 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 -- 172.21.15.42:0/395727858 wait complete. 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 -- start start 2024-05-27T10:26:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.237+0000 7f75de971740 1 Processor -- start 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e6986a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 0x557e6986a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694b51b0 0x557e698654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01affe0 con 0x557e694b51b0 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01b1450 con 0x557e6948c2f0 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01b06b0 con 0x557e694944d0 2024-05-27T10:26:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.238+0000 7f75de971740 10 client.0 _ll_register_callbacks cb 0x7f75b80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75d5000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e6986a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 0x557e6986a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 0x557e6986a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:50784/0 (socket says 172.21.15.42:50784) 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 -- 172.21.15.42:0/302640481 learned_addr learned my addr 172.21.15.42:0/302640481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75d5a00640 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694b51b0 0x557e698654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 -- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 msgr2=0x557e6986a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e6986a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 -- 172.21.15.42:0/302640481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694b51b0 msgr2=0x557e698654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694b51b0 0x557e698654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.239+0000 7f75cfe00640 1 -- 172.21.15.42:0/302640481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e69865c60 con 0x557e694944d0 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.240+0000 7f75cfe00640 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 0x557e6986a640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006010 tx=0x7f75b80040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7541f74ea92620 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.240+0000 7f75bea00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:26:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.240+0000 7f75bea00640 1 -- 172.21.15.42:0/302640481 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b806b020 con 0x557e694944d0 2024-05-27T10:26:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.240+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e697c42c0 con 0x557e694944d0 2024-05-27T10:26:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.241+0000 7f75bea00640 1 -- 172.21.15.42:0/302640481 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f75b804b890 con 0x557e694944d0 2024-05-27T10:26:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.241+0000 7f75bea00640 1 -- 172.21.15.42:0/302640481 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f75b8076050 con 0x557e694944d0 2024-05-27T10:26:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.241+0000 7f75de971740 10 client.26230 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.241+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e698dbed0 con 0x557e694944d0 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75bea00640 1 -- 172.21.15.42:0/302640481 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557e697c42c0 con 0x557e694944d0 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75bea00640 1 -- 172.21.15.42:0/302640481 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f75b8048040 con 0x557e694944d0 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75de971740 10 client.26230 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75de971740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75de971740 20 client.26230 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75de971740 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e694491c0 0x557e698de540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75de971740 4 client.26230 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.242+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e6973e4b0 con 0x557e694491c0 2024-05-27T10:26:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.243+0000 7f75d5000640 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e694491c0 0x557e698de540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.244+0000 7f75d5000640 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e694491c0 0x557e698de540 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e79947d1e212f867 server_cookie=d3c2bd545d3061a1 in_seq=0 out_seq=0 2024-05-27T10:26:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.244+0000 7f75bea00640 10 client.26230 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.270+0000 7f75bea00640 1 -- 172.21.15.42:0/302640481 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557e6973e4b0 con 0x557e694491c0 2024-05-27T10:26:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.270+0000 7f75bea00640 10 client.26230 handle_command_reply: tid=0 2024-05-27T10:26:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.270+0000 7f75de971740 1 client.26230 shutdown 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.270+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 msgr2=0x557e6986a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.270+0000 7f75de971740 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 0x557e6986a640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006010 tx=0x7f75b80040d0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 shutdown_connections 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557e694b51b0 0x557e698654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x557e694491c0 0x557e698de540 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e694944d0 0x557e6986a640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 --2- 172.21.15.42:0/302640481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557e6948c2f0 0x557e6986a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75bea00640 0 client.26230 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 >> 172.21.15.42:0/302640481 conn(0x557e694a9300 msgr2=0x557e6985c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 shutdown_connections 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 1 -- 172.21.15.42:0/302640481 wait complete. 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.271+0000 7f75de971740 20 client.26230 trim_cache size 0 max 16384 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:26:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f75b40b2950 msgr2=0x7f75b40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f75b40b2950 0x7f75b40b4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75b802a930 tx=0x7f75b8000bc0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 msgr2=0x7f75d0075a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0075a40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f75c800eaa0 tx=0x7f75c800ef60 comp rx=0 tx=0).stop 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 shutdown_connections 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f75b40b2950 0x7f75b40b4e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75d0109f50 0x7f75d0075a40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d0109ab0 0x7f75d0075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 --2- 172.21.15.42:0/3259672227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d0104200 0x7f75d0074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.272+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 >> 172.21.15.42:0/3259672227 conn(0x7f75d0079140 msgr2=0x7f75d0102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.273+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 shutdown_connections 2024-05-27T10:26:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:40.273+0000 7f75d6400640 1 -- 172.21.15.42:0/3259672227 wait complete. 2024-05-27T10:26:40.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:39 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2472619249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:40.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:39 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2472619249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:26:40.431 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id 27e9bff1 --release --await' 2024-05-27T10:26:40.431 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id 27e9bff1 --release --await 2024-05-27T10:26:40.945 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:26:41.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:40 smithi104 ceph-mon[34812]: pgmap v243: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 604 KiB/s wr, 112 op/s 2024-05-27T10:26:41.212 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:40 smithi042 ceph-mon[31661]: pgmap v243: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 604 KiB/s wr, 112 op/s 2024-05-27T10:26:41.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:40 smithi143 ceph-mon[35545]: pgmap v243: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 604 KiB/s wr, 112 op/s 2024-05-27T10:26:43.093 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:42 smithi042 ceph-mon[31661]: pgmap v244: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 43 op/s 2024-05-27T10:26:43.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:42 smithi104 ceph-mon[34812]: pgmap v244: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 43 op/s 2024-05-27T10:26:43.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:42 smithi143 ceph-mon[35545]: pgmap v244: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 43 op/s 2024-05-27T10:26:43.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.767+0000 7f1b81c00700 1 -- 172.21.15.42:0/1421635159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c10b860 msgr2=0x7f1b7c1131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.767+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1421635159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1131c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009920 tx=0x7f1b74009c30 comp rx=0 tx=0).stop 2024-05-27T10:26:43.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 -- 172.21.15.42:0/1421635159 shutdown_connections 2024-05-27T10:26:43.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1421635159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1131c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1421635159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7c106630 0x7f1b7c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1421635159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10af10 0x7f1b7c10b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 -- 172.21.15.42:0/1421635159 >> 172.21.15.42:0/1421635159 conn(0x7f1b7c102020 msgr2=0x7f1b7c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:43.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 -- 172.21.15.42:0/1421635159 shutdown_connections 2024-05-27T10:26:43.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.768+0000 7f1b81c00700 1 -- 172.21.15.42:0/1421635159 wait complete. 2024-05-27T10:26:43.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.769+0000 7f1b81c00700 1 Processor -- start 2024-05-27T10:26:43.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.769+0000 7f1b81c00700 1 -- start start 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 0x7f1b7c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7c10af10 0x7f1b7c19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c116340 con 0x7f1b7c106630 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1161c0 con 0x7f1b7c10b860 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1164c0 con 0x7f1b7c10af10 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b7be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 0x7f1b7c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b7be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 0x7f1b7c19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56896/0 (socket says 172.21.15.42:56896) 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b7be00700 1 -- 172.21.15.42:0/1774573895 learned_addr learned my addr 172.21.15.42:0/1774573895 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b73400700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7c10af10 0x7f1b7c19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.770+0000 7f1b81200700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b7be00700 1 -- 172.21.15.42:0/1774573895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7c10af10 msgr2=0x7f1b7c19d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7c10af10 0x7f1b7c19d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b7be00700 1 -- 172.21.15.42:0/1774573895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10b860 msgr2=0x7f1b7c1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b7be00700 1 -- 172.21.15.42:0/1774573895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c1a1c40 con 0x7f1b7c106630 2024-05-27T10:26:43.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b81200700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:43.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.771+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 0x7f1b7c19cd70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c0c0 tx=0x7f1b6c00c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:43.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.772+0000 7f1b79600700 1 -- 172.21.15.42:0/1774573895 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00cd10 con 0x7f1b7c106630 2024-05-27T10:26:43.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.772+0000 7f1b79600700 1 -- 172.21.15.42:0/1774573895 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c0075c0 con 0x7f1b7c106630 2024-05-27T10:26:43.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.772+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c1a1ed0 con 0x7f1b7c106630 2024-05-27T10:26:43.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.772+0000 7f1b79600700 1 -- 172.21.15.42:0/1774573895 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0146e0 con 0x7f1b7c106630 2024-05-27T10:26:43.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.772+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1a9850 con 0x7f1b7c106630 2024-05-27T10:26:43.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.776+0000 7f1b79600700 1 -- 172.21.15.42:0/1774573895 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1b6c01b020 con 0x7f1b7c106630 2024-05-27T10:26:43.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b91e36b80 1 Processor -- start 2024-05-27T10:26:43.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b91e36b80 1 -- start start 2024-05-27T10:26:43.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x56118129cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b79600700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1b600b2a70 0x7f1b600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56118111df40 0x56118129d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611812a7840 0x5611812a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.777+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611811caf40 con 0x561181209af0 2024-05-27T10:26:43.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611811baae0 con 0x56118111df40 2024-05-27T10:26:43.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5611811beb20 con 0x5611812a7840 2024-05-27T10:26:43.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b81200700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1b600b2a70 0x7f1b600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b79600700 1 -- 172.21.15.42:0/1774573895 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f1b6c0e2440 con 0x7f1b7c106630 2024-05-27T10:26:43.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b73400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56118111df40 0x56118129d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b7be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x56118129cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b7be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x56118129cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56910/0 (socket says 172.21.15.42:56910) 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b7be00700 1 -- 172.21.15.42:0/1891463883 learned_addr learned my addr 172.21.15.42:0/1891463883 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.778+0000 7f1b81200700 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611812a7840 0x5611812a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b7be00700 1 -- 172.21.15.42:0/1891463883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611812a7840 msgr2=0x5611812a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611812a7840 0x5611812a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b7be00700 1 -- 172.21.15.42:0/1891463883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56118111df40 msgr2=0x56118129d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56118111df40 0x56118129d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b7be00700 1 -- 172.21.15.42:0/1891463883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561181183520 con 0x561181209af0 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b81200700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1b600b2a70 0x7f1b600b4f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b740059d0 tx=0x7f1b7400f460 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.779+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x56118129cc40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0dfbb0 tx=0x7f1b6c0a90b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb30857470591f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.780+0000 7f1b71600700 1 -- 172.21.15.42:0/1891463883 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0a91f0 con 0x561181209af0 2024-05-27T10:26:43.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.780+0000 7f1b71600700 1 -- 172.21.15.42:0/1891463883 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c0a9de0 con 0x561181209af0 2024-05-27T10:26:43.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.780+0000 7f1b71600700 1 -- 172.21.15.42:0/1891463883 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0a7630 con 0x561181209af0 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1891463883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 msgr2=0x56118129cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x56118129cc40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0dfbb0 tx=0x7f1b6c0a90b0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1891463883 shutdown_connections 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611812a7840 0x5611812a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x56118129cc40 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1891463883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56118111df40 0x56118129d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1891463883 >> 172.21.15.42:0/1891463883 conn(0x5611811b7490 msgr2=0x561181298b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:43.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1891463883 shutdown_connections 2024-05-27T10:26:43.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.781+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1891463883 wait complete. 2024-05-27T10:26:43.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.782+0000 7f1b91e36b80 1 -- start start 2024-05-27T10:26:43.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.782+0000 7f1b91e36b80 1 Processor -- start 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.782+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611810c10a0 con 0x561181209af0 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56118115b860 con 0x5611812a7840 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5611812aa2f0 con 0x56118111df40 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b91e36b80 10 client.0 _ll_register_callbacks cb 0x561181164d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:43.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56916/0 (socket says 172.21.15.42:56916) 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 -- 172.21.15.42:0/26849471 learned_addr learned my addr 172.21.15.42:0/26849471 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b7be00700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b81200700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 -- 172.21.15.42:0/26849471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 msgr2=0x5611812a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 -- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 msgr2=0x5611812a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b73400700 1 -- 172.21.15.42:0/26849471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118110a9d0 con 0x561181209af0 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.783+0000 7f1b81200700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.784+0000 7f1b73400700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a4d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1b640126b0 tx=0x7f1b64012a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7998cf6d5852c320 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:43.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.784+0000 7f1b6a000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.784+0000 7f1b6a000700 1 -- 172.21.15.42:0/26849471 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400f4e0 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.784+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c0a9de0 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.784+0000 7f1b6a000700 1 -- 172.21.15.42:0/26849471 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f1b6400f680 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.784+0000 7f1b6a000700 1 -- 172.21.15.42:0/26849471 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1b6400f880 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 10 client.16701 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56118118b160 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b6a000700 1 -- 172.21.15.42:0/26849471 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c0a9de0 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b6a000700 1 -- 172.21.15.42:0/26849471 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1b64023070 con 0x561181209af0 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 10 client.16701 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 20 client.16701 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611812b7250 0x5611812b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 4 client.16701 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:43.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.785+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611811cce90 con 0x5611812b7250 2024-05-27T10:26:43.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.786+0000 7f1b7be00700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611812b7250 0x5611812b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.787+0000 7f1b7be00700 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611812b7250 0x5611812b9670 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b738f5991c27125b server_cookie=f33256a80ee8d028 in_seq=0 out_seq=0 2024-05-27T10:26:43.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.787+0000 7f1b6a000700 10 client.16701 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:43.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b6a000700 1 -- 172.21.15.42:0/26849471 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5611811cce90 con 0x5611812b7250 2024-05-27T10:26:43.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b6a000700 10 client.16701 handle_command_reply: tid=0 2024-05-27T10:26:43.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 client.16701 shutdown 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 msgr2=0x5611812a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a4d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1b640126b0 tx=0x7f1b64012a70 comp rx=0 tx=0).stop 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 shutdown_connections 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611812b7250 0x5611812b9670 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/26849471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a4d30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b6a000700 0 client.16701 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.789+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 >> 172.21.15.42:0/26849471 conn(0x5611811b7490 msgr2=0x56118129ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.790+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 shutdown_connections 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.790+0000 7f1b91e36b80 1 -- 172.21.15.42:0/26849471 wait complete. 2024-05-27T10:26:43.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.790+0000 7f1b91e36b80 20 client.16701 trim_cache size 0 max 16384 2024-05-27T10:26:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.812+0000 7f1b91e36b80 1 Processor -- start 2024-05-27T10:26:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.812+0000 7f1b91e36b80 1 -- start start 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b3760 con 0x561181209af0 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b0f90 con 0x5611812a7840 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1aee00 con 0x56118111df40 2024-05-27T10:26:43.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b73400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b73400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56922/0 (socket says 172.21.15.42:56922) 2024-05-27T10:26:43.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.813+0000 7f1b73400700 1 -- 172.21.15.42:0/779218772 learned_addr learned my addr 172.21.15.42:0/779218772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b7be00700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b81200700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b73400700 1 -- 172.21.15.42:0/779218772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 msgr2=0x5611812a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b73400700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b73400700 1 -- 172.21.15.42:0/779218772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 msgr2=0x5611812a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b73400700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b73400700 1 -- 172.21.15.42:0/779218772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118129e190 con 0x561181209af0 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b81200700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b7be00700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.814+0000 7f1b73400700 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a8c60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009340 tx=0x7f1b64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7475fd9c0b1b054 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:43.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.815+0000 7f1b70c00700 1 -- 172.21.15.42:0/779218772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64029030 con 0x561181209af0 2024-05-27T10:26:43.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.815+0000 7f1b70c00700 1 -- 172.21.15.42:0/779218772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6400f620 con 0x561181209af0 2024-05-27T10:26:43.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.815+0000 7f1b70c00700 1 -- 172.21.15.42:0/779218772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640050d0 con 0x561181209af0 2024-05-27T10:26:43.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 -- 172.21.15.42:0/779218772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 msgr2=0x5611812a8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a8c60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009340 tx=0x7f1b64012000 comp rx=0 tx=0).stop 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 -- 172.21.15.42:0/779218772 shutdown_connections 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/779218772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812a8c60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 -- 172.21.15.42:0/779218772 >> 172.21.15.42:0/779218772 conn(0x5611811b7490 msgr2=0x5611812a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 -- 172.21.15.42:0/779218772 shutdown_connections 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 -- 172.21.15.42:0/779218772 wait complete. 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.816+0000 7f1b91e36b80 1 -- start start 2024-05-27T10:26:43.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 Processor -- start 2024-05-27T10:26:43.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1af450 con 0x561181209af0 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b08c0 con 0x5611812a7840 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1afb20 con 0x56118111df40 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.817+0000 7f1b91e36b80 10 client.0 _ll_register_callbacks cb 0x561181164d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56926/0 (socket says 172.21.15.42:56926) 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 -- 172.21.15.42:0/1024414634 learned_addr learned my addr 172.21.15.42:0/1024414634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b81200700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 -- 172.21.15.42:0/1024414634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 msgr2=0x5611812afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 -- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 msgr2=0x5611812ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 -- 172.21.15.42:0/1024414634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118132a520 con 0x561181209af0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b81200700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b73400700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812ac1a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1b64005040 tx=0x7f1b6400fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c3f07b16b4cd69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.818+0000 7f1b6a000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b6a000700 1 -- 172.21.15.42:0/1024414634 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64029040 con 0x561181209af0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b6a000700 1 -- 172.21.15.42:0/1024414634 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f1b64010570 con 0x561181209af0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6400f620 con 0x561181209af0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b6a000700 1 -- 172.21.15.42:0/1024414634 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1b64010770 con 0x561181209af0 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b91e36b80 10 client.16707 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:43.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56118132b620 con 0x561181209af0 2024-05-27T10:26:43.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.819+0000 7f1b6a000700 1 -- 172.21.15.42:0/1024414634 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6400f620 con 0x561181209af0 2024-05-27T10:26:43.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b6a000700 1 -- 172.21.15.42:0/1024414634 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1b64034030 con 0x561181209af0 2024-05-27T10:26:43.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b91e36b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:43.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b91e36b80 10 client.16707 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b91e36b80 20 client.16707 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56118132bd40 0x56118132e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b91e36b80 4 client.16707 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27e9bff1", "release": true, "await": true}] 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.820+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27e9bff1", "release": true, "await": true}) -- 0x5611811c6670 con 0x56118132bd40 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.821+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56118132bd40 0x56118132e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.822+0000 7f1b7be00700 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56118132bd40 0x56118132e160 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91fa512149e1f977 server_cookie=35b146733c4c3243 in_seq=0 out_seq=0 2024-05-27T10:26:43.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.822+0000 7f1b6a000700 10 client.16707 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.865+0000 7f1b6a000700 1 -- 172.21.15.42:0/1024414634 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5611811c6670 con 0x56118132bd40 2024-05-27T10:26:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.865+0000 7f1b6a000700 10 client.16707 handle_command_reply: tid=0 2024-05-27T10:26:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.865+0000 7f1b91e36b80 1 client.16707 shutdown 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.865+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 msgr2=0x5611812ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.865+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812ac1a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1b64005040 tx=0x7f1b6400fee0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 shutdown_connections 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56118132bd40 0x56118132e160 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611812a7840 0x5611812ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56118111df40 0x5611812afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 --2- 172.21.15.42:0/1024414634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561181209af0 0x5611812ac1a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b6a000700 0 client.16707 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 >> 172.21.15.42:0/1024414634 conn(0x5611811b7490 msgr2=0x5611812a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 shutdown_connections 2024-05-27T10:26:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.866+0000 7f1b91e36b80 1 -- 172.21.15.42:0/1024414634 wait complete. 2024-05-27T10:26:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.867+0000 7f1b91e36b80 20 client.16707 trim_cache size 0 max 16384 2024-05-27T10:26:43.867 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 33, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 24, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "27e9bff1": { 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 24, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:26:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 138.9, 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:26:43.869 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1b600b2a70 msgr2=0x7f1b600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1b600b2a70 0x7f1b600b4f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b740059d0 tx=0x7f1b7400f460 comp rx=0 tx=0).stop 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 msgr2=0x7f1b7c19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 0x7f1b7c19cd70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c0c0 tx=0x7f1b6c00c3d0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 shutdown_connections 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1b600b2a70 0x7f1b600b4f80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c10b860 0x7f1b7c1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c106630 0x7f1b7c19cd70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 --2- 172.21.15.42:0/1774573895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7c10af10 0x7f1b7c19d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.868+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 >> 172.21.15.42:0/1774573895 conn(0x7f1b7c102020 msgr2=0x7f1b7c10c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.869+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 shutdown_connections 2024-05-27T10:26:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:26:43.869+0000 7f1b81c00700 1 -- 172.21.15.42:0/1774573895 wait complete. 2024-05-27T10:26:45.033 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27e9bff1', total seconds elapsed: 31.4 2024-05-27T10:26:45.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:44 smithi104 ceph-mon[34812]: pgmap v245: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 43 op/s 2024-05-27T10:26:45.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:44 smithi143 ceph-mon[35545]: pgmap v245: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 43 op/s 2024-05-27T10:26:45.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:44 smithi042 ceph-mon[31661]: pgmap v245: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 43 op/s 2024-05-27T10:26:47.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:46 smithi104 ceph-mon[34812]: pgmap v246: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 241 KiB/s wr, 48 op/s 2024-05-27T10:26:47.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:46 smithi143 ceph-mon[35545]: pgmap v246: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 241 KiB/s wr, 48 op/s 2024-05-27T10:26:47.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:46 smithi042 ceph-mon[31661]: pgmap v246: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 241 KiB/s wr, 48 op/s 2024-05-27T10:26:48.717 INFO:tasks.workunit.client.0.smithi042.stdout: 49 1831 86 1209 93 387 142 2024-05-27T10:26:49.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:48 smithi104 ceph-mon[34812]: pgmap v247: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 7 op/s 2024-05-27T10:26:49.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:48 smithi143 ceph-mon[35545]: pgmap v247: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 7 op/s 2024-05-27T10:26:49.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:48 smithi042 ceph-mon[31661]: pgmap v247: 97 pgs: 97 active+clean; 329 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 7 op/s 2024-05-27T10:26:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:50 smithi104 ceph-mon[34812]: pgmap v248: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-05-27T10:26:51.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:50 smithi143 ceph-mon[35545]: pgmap v248: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-05-27T10:26:51.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:50 smithi042 ceph-mon[31661]: pgmap v248: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-05-27T10:26:53.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:52 smithi104 ceph-mon[34812]: pgmap v249: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-27T10:26:53.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:52 smithi143 ceph-mon[35545]: pgmap v249: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-27T10:26:53.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:52 smithi042 ceph-mon[31661]: pgmap v249: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-27T10:26:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 -- 172.21.15.42:0/2857554807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac104220 msgr2=0x7f85ac1105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2857554807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac104220 0x7f85ac1105f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009ca0 tx=0x7f85a402a8d0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 -- 172.21.15.42:0/2857554807 shutdown_connections 2024-05-27T10:26:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2857554807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac104220 0x7f85ac1105f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2857554807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85ac109f70 0x7f85ac103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2857554807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85ac109ad0 0x7f85ac103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 -- 172.21.15.42:0/2857554807 >> 172.21.15.42:0/2857554807 conn(0x7f85ac079100 msgr2=0x7f85ac079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.583+0000 7f85b0c00640 1 -- 172.21.15.42:0/2857554807 shutdown_connections 2024-05-27T10:26:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.584+0000 7f85b0c00640 1 -- 172.21.15.42:0/2857554807 wait complete. 2024-05-27T10:26:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.585+0000 7f85b0c00640 1 Processor -- start 2024-05-27T10:26:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.585+0000 7f85b0c00640 1 -- start start 2024-05-27T10:26:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85b0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85ac104220 0x7f85ac074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85b0c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85ac109ad0 0x7f85ac075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85b0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 0x7f85ac075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85b0c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac113bf0 con 0x7f85ac109f70 2024-05-27T10:26:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85b0c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac113a70 con 0x7f85ac109ad0 2024-05-27T10:26:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85b0c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac113d70 con 0x7f85ac104220 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 0x7f85ac075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 0x7f85ac075a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45474/0 (socket says 172.21.15.42:45474) 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85abe00640 1 -- 172.21.15.42:0/2934289508 learned_addr learned my addr 172.21.15.42:0/2934289508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85ab400640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85ac104220 0x7f85ac074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.586+0000 7f85aaa00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85ac109ad0 0x7f85ac075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.587+0000 7f85abe00640 1 -- 172.21.15.42:0/2934289508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85ac104220 msgr2=0x7f85ac074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.587+0000 7f85abe00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85ac104220 0x7f85ac074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.587+0000 7f85abe00640 1 -- 172.21.15.42:0/2934289508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85ac109ad0 msgr2=0x7f85ac075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.587+0000 7f85abe00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85ac109ad0 0x7f85ac075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.587+0000 7f85abe00640 1 -- 172.21.15.42:0/2934289508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac1a8930 con 0x7f85ac109f70 2024-05-27T10:26:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.587+0000 7f85abe00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 0x7f85ac075a40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f85a4032c00 tx=0x7f85a4032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.588+0000 7f85a3e00640 1 -- 172.21.15.42:0/2934289508 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4004040 con 0x7f85ac109f70 2024-05-27T10:26:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.588+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac1a8bc0 con 0x7f85ac109f70 2024-05-27T10:26:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.588+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1a9160 con 0x7f85ac109f70 2024-05-27T10:26:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.588+0000 7f85a3e00640 1 -- 172.21.15.42:0/2934289508 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a4004670 con 0x7f85ac109f70 2024-05-27T10:26:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.588+0000 7f85a3e00640 1 -- 172.21.15.42:0/2934289508 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4030d60 con 0x7f85ac109f70 2024-05-27T10:26:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.592+0000 7f85b91d2740 1 Processor -- start 2024-05-27T10:26:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.593+0000 7f85a3e00640 1 -- 172.21.15.42:0/2934289508 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f85a404e440 con 0x7f85ac109f70 2024-05-27T10:26:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.593+0000 7f85b91d2740 1 -- start start 2024-05-27T10:26:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.594+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85bc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.594+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.594+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.594+0000 7f85b91d2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dcf852db90 con 0x55dcf81f94d0 2024-05-27T10:26:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.594+0000 7f85b91d2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dcf851d730 con 0x55dcf81f12f0 2024-05-27T10:26:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.594+0000 7f85b91d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8521770 con 0x55dcf821a1b0 2024-05-27T10:26:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.595+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.595+0000 7f85ab400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.595+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85bd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45476/0 (socket says 172.21.15.42:45476) 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.595+0000 7f85aaa00640 1 -- 172.21.15.42:0/3411991913 learned_addr learned my addr 172.21.15.42:0/3411991913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.595+0000 7f85aaa00640 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85aaa00640 1 -- 172.21.15.42:0/3411991913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 msgr2=0x55dcf85bc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85aaa00640 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85bc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85aaa00640 1 -- 172.21.15.42:0/3411991913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 msgr2=0x55dcf85bce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85aaa00640 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85bce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85aaa00640 1 -- 172.21.15.42:0/3411991913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf84b68b0 con 0x55dcf81f94d0 2024-05-27T10:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85a3e00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f85900b29d0 0x7f85900b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.596+0000 7f85aaa00640 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85bd3d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8598009fd0 tx=0x7f85980094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f314175990ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.597+0000 7f85a3e00640 1 -- 172.21.15.42:0/2934289508 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f85a41039e0 con 0x7f85ac109f70 2024-05-27T10:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.597+0000 7f8597e00640 1 -- 172.21.15.42:0/3411991913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598017e60 con 0x55dcf81f94d0 2024-05-27T10:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.597+0000 7f8597e00640 1 -- 172.21.15.42:0/3411991913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f859801d070 con 0x55dcf81f94d0 2024-05-27T10:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.597+0000 7f8597e00640 1 -- 172.21.15.42:0/3411991913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598018870 con 0x55dcf81f94d0 2024-05-27T10:26:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.597+0000 7f85ab400640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f85900b29d0 0x7f85900b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85ab400640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f85900b29d0 0x7f85900b4ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55dcf85ba550 tx=0x7f859c00a2d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 -- 172.21.15.42:0/3411991913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 msgr2=0x55dcf85bd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85bd3d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8598009fd0 tx=0x7f85980094d0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 -- 172.21.15.42:0/3411991913 shutdown_connections 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85bd3d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85bce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 --2- 172.21.15.42:0/3411991913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85bc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.599+0000 7f85b91d2740 1 -- 172.21.15.42:0/3411991913 >> 172.21.15.42:0/3411991913 conn(0x55dcf820e300 msgr2=0x55dcf85b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.600+0000 7f85b91d2740 1 -- 172.21.15.42:0/3411991913 shutdown_connections 2024-05-27T10:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.600+0000 7f85b91d2740 1 -- 172.21.15.42:0/3411991913 wait complete. 2024-05-27T10:26:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.600+0000 7f85b91d2740 1 -- start start 2024-05-27T10:26:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.601+0000 7f85b91d2740 1 Processor -- start 2024-05-27T10:26:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.601+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81f12f0 0x55dcf85cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85b91d2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8203790 con 0x55dcf81f94d0 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85b91d2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dcf84c5bc0 con 0x55dcf821a1b0 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85b91d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55dcf85d2340 con 0x55dcf81f12f0 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85b91d2740 10 client.0 _ll_register_callbacks cb 0x55dcf84a3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45482/0 (socket says 172.21.15.42:45482) 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85aaa00640 1 -- 172.21.15.42:0/987789932 learned_addr learned my addr 172.21.15.42:0/987789932 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.602+0000 7f85abe00640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85aaa00640 1 -- 172.21.15.42:0/987789932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81f12f0 msgr2=0x55dcf85cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85aaa00640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81f12f0 0x55dcf85cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85aaa00640 1 -- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 msgr2=0x55dcf85db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85aaa00640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85aaa00640 1 -- 172.21.15.42:0/987789932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf8232210 con 0x55dcf81f94d0 2024-05-27T10:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85abe00640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f85aaa00640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85cfc70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8598015990 tx=0x7f85980159c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90469f0bd80c43b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f8594c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f8594c00640 1 -- 172.21.15.42:0/987789932 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598025070 con 0x55dcf81f94d0 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.603+0000 7f8594c00640 1 -- 172.21.15.42:0/987789932 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f85980185f0 con 0x55dcf81f94d0 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.604+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859801d070 con 0x55dcf81f94d0 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.604+0000 7f8594c00640 1 -- 172.21.15.42:0/987789932 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f859802f050 con 0x55dcf81f94d0 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.604+0000 7f85b91d2740 10 client.16719 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.604+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dcf82fe620 con 0x55dcf81f94d0 2024-05-27T10:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.604+0000 7f8594c00640 1 -- 172.21.15.42:0/987789932 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f859801d070 con 0x55dcf81f94d0 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f8594c00640 1 -- 172.21.15.42:0/987789932 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f859802a440 con 0x55dcf81f94d0 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f85b91d2740 10 client.16719 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f85b91d2740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f85b91d2740 20 client.16719 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f85b91d2740 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf81d8d80 0x55dcf85f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f85b91d2740 4 client.16719 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:26:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.605+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf852fae0 con 0x55dcf81d8d80 2024-05-27T10:26:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.606+0000 7f85ab400640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf81d8d80 0x55dcf85f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.607+0000 7f85ab400640 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf81d8d80 0x55dcf85f25d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3717375ace0a4d1d server_cookie=d4747f8586f19781 in_seq=0 out_seq=0 2024-05-27T10:26:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.607+0000 7f8594c00640 10 client.16719 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.608+0000 7f8594c00640 1 -- 172.21.15.42:0/987789932 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dcf852fae0 con 0x55dcf81d8d80 2024-05-27T10:26:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.608+0000 7f8594c00640 10 client.16719 handle_command_reply: tid=0 2024-05-27T10:26:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.608+0000 7f85b91d2740 1 client.16719 shutdown 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 msgr2=0x55dcf85cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85cfc70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8598015990 tx=0x7f85980159c0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 shutdown_connections 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f94d0 0x55dcf85cfc70 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf81d8d80 0x55dcf85f25d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 --2- 172.21.15.42:0/987789932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81f12f0 0x55dcf85cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 >> 172.21.15.42:0/987789932 conn(0x55dcf820e300 msgr2=0x55dcf85be710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f8594c00640 0 client.16719 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 shutdown_connections 2024-05-27T10:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 1 -- 172.21.15.42:0/987789932 wait complete. 2024-05-27T10:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.609+0000 7f85b91d2740 20 client.16719 trim_cache size 0 max 16384 2024-05-27T10:26:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.631+0000 7f85b91d2740 1 Processor -- start 2024-05-27T10:26:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.631+0000 7f85b91d2740 1 -- start start 2024-05-27T10:26:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 0x55dcf85db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81ae1c0 0x55dcf85c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85b91d2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac0731c0 con 0x55dcf81f12f0 2024-05-27T10:26:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85b91d2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac070a30 con 0x55dcf821a1b0 2024-05-27T10:26:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85b91d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac06e8a0 con 0x55dcf81ae1c0 2024-05-27T10:26:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 0x55dcf85db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 0x55dcf85db5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45488/0 (socket says 172.21.15.42:45488) 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.632+0000 7f85ab400640 1 -- 172.21.15.42:0/1064701198 learned_addr learned my addr 172.21.15.42:0/1064701198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85aaa00640 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85abe00640 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81ae1c0 0x55dcf85c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85ab400640 1 -- 172.21.15.42:0/1064701198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81ae1c0 msgr2=0x55dcf85c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85ab400640 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81ae1c0 0x55dcf85c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85ab400640 1 -- 172.21.15.42:0/1064701198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 msgr2=0x55dcf85bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85ab400640 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85bcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.633+0000 7f85ab400640 1 -- 172.21.15.42:0/1064701198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf85d0610 con 0x55dcf81f12f0 2024-05-27T10:26:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.634+0000 7f85ab400640 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 0x55dcf85db5e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f859c011010 tx=0x7f859c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cad564b5d45f02b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.634+0000 7f8597400640 1 -- 172.21.15.42:0/1064701198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c02b070 con 0x55dcf81f12f0 2024-05-27T10:26:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.634+0000 7f8597400640 1 -- 172.21.15.42:0/1064701198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f859c027420 con 0x55dcf81f12f0 2024-05-27T10:26:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.634+0000 7f8597400640 1 -- 172.21.15.42:0/1064701198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c00f680 con 0x55dcf81f12f0 2024-05-27T10:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 -- 172.21.15.42:0/1064701198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 msgr2=0x55dcf85db5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 0x55dcf85db5e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f859c011010 tx=0x7f859c015000 comp rx=0 tx=0).stop 2024-05-27T10:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 -- 172.21.15.42:0/1064701198 shutdown_connections 2024-05-27T10:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf81ae1c0 0x55dcf85c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf821a1b0 0x55dcf85bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1064701198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81f12f0 0x55dcf85db5e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.635+0000 7f85b91d2740 1 -- 172.21.15.42:0/1064701198 >> 172.21.15.42:0/1064701198 conn(0x55dcf820e300 msgr2=0x55dcf85bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.636+0000 7f85b91d2740 1 -- 172.21.15.42:0/1064701198 shutdown_connections 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.636+0000 7f85b91d2740 1 -- 172.21.15.42:0/1064701198 wait complete. 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.636+0000 7f85b91d2740 1 -- start start 2024-05-27T10:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.637+0000 7f85b91d2740 1 Processor -- start 2024-05-27T10:26:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81ae1c0 0x55dcf85f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac06eef0 con 0x55dcf81ae1c0 2024-05-27T10:26:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac070360 con 0x55dcf81f12f0 2024-05-27T10:26:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac06f5c0 con 0x55dcf821a1b0 2024-05-27T10:26:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85b91d2740 10 client.0 _ll_register_callbacks cb 0x7f859c00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:26:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:55092/0 (socket says 172.21.15.42:55092) 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85aaa00640 1 -- 172.21.15.42:0/1051004059 learned_addr learned my addr 172.21.15.42:0/1051004059 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85ab400640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81ae1c0 0x55dcf85f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.638+0000 7f85abe00640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.639+0000 7f85aaa00640 1 -- 172.21.15.42:0/1051004059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 msgr2=0x55dcf85f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.639+0000 7f85aaa00640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.639+0000 7f85aaa00640 1 -- 172.21.15.42:0/1051004059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81ae1c0 msgr2=0x55dcf85f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.639+0000 7f85aaa00640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81ae1c0 0x55dcf85f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.639+0000 7f85aaa00640 1 -- 172.21.15.42:0/1051004059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf8658c10 con 0x55dcf81f12f0 2024-05-27T10:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.639+0000 7f85ab400640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81ae1c0 0x55dcf85f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.640+0000 7f85aaa00640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85f2b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8598015960 tx=0x7f8598015ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b7ec892736281b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.640+0000 7f8594c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.640+0000 7f8594c00640 1 -- 172.21.15.42:0/1051004059 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598038070 con 0x55dcf81f12f0 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.640+0000 7f8594c00640 1 -- 172.21.15.42:0/1051004059 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8484+0+0 (secure 0 0 0) 0x7f8598009110 con 0x55dcf81f12f0 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.641+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859c027420 con 0x55dcf81f12f0 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.641+0000 7f8594c00640 1 -- 172.21.15.42:0/1051004059 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8598044050 con 0x55dcf81f12f0 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.641+0000 7f85b91d2740 10 client.36236 fetch_fsmap learned FSMap version 33 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.641+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dcf8659e80 con 0x55dcf81f12f0 2024-05-27T10:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.642+0000 7f8594c00640 1 -- 172.21.15.42:0/1051004059 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f859c027420 con 0x55dcf81f12f0 2024-05-27T10:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f8594c00640 1 -- 172.21.15.42:0/1051004059 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f85980188e0 con 0x55dcf81f12f0 2024-05-27T10:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f85b91d2740 10 client.36236 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f85b91d2740 10 client.36236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f85b91d2740 20 client.36236 populate_metadata read hostname 'smithi042' 2024-05-27T10:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf865a4e0 0x55dcf865c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f85b91d2740 4 client.36236 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.643+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcf85292c0 con 0x55dcf865a4e0 2024-05-27T10:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.644+0000 7f85ab400640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf865a4e0 0x55dcf865c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.645+0000 7f85ab400640 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf865a4e0 0x55dcf865c900 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b922728224b2445 server_cookie=18fae7819cfeccdb in_seq=0 out_seq=0 2024-05-27T10:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.645+0000 7f8594c00640 10 client.36236 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.704+0000 7f8594c00640 1 -- 172.21.15.42:0/1051004059 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dcf85292c0 con 0x55dcf865a4e0 2024-05-27T10:26:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.704+0000 7f8594c00640 10 client.36236 handle_command_reply: tid=0 2024-05-27T10:26:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.704+0000 7f85b91d2740 1 client.36236 shutdown 2024-05-27T10:26:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.704+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 msgr2=0x55dcf85f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.704+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85f2b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8598015960 tx=0x7f8598015ab0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 shutdown_connections 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55dcf865a4e0 0x55dcf865c900 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dcf821a1b0 0x55dcf85f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf81f12f0 0x55dcf85f2b80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 --2- 172.21.15.42:0/1051004059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dcf81ae1c0 0x55dcf85f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f8594c00640 0 client.36236 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:26:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 >> 172.21.15.42:0/1051004059 conn(0x55dcf820e300 msgr2=0x55dcf85d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 shutdown_connections 2024-05-27T10:26:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.705+0000 7f85b91d2740 1 -- 172.21.15.42:0/1051004059 wait complete. 2024-05-27T10:26:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.706+0000 7f85b91d2740 20 client.36236 trim_cache size 0 max 16384 2024-05-27T10:26:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:26:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:26:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:26:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f85900b29d0 msgr2=0x7f85900b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f85900b29d0 0x7f85900b4ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55dcf85ba550 tx=0x7f859c00a2d0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 msgr2=0x7f85ac075a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:26:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 0x7f85ac075a40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f85a4032c00 tx=0x7f85a4032c30 comp rx=0 tx=0).stop 2024-05-27T10:26:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 shutdown_connections 2024-05-27T10:26:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f85900b29d0 0x7f85900b4ee0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ac109f70 0x7f85ac075a40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85ac109ad0 0x7f85ac075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 --2- 172.21.15.42:0/2934289508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85ac104220 0x7f85ac074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:26:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 >> 172.21.15.42:0/2934289508 conn(0x7f85ac079100 msgr2=0x7f85ac101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:26:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.707+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 shutdown_connections 2024-05-27T10:26:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:26:53.708+0000 7f85b0c00640 1 -- 172.21.15.42:0/2934289508 wait complete. 2024-05-27T10:26:55.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:54 smithi104 ceph-mon[34812]: pgmap v250: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-27T10:26:55.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:54 smithi143 ceph-mon[35545]: pgmap v250: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-27T10:26:55.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:54 smithi042 ceph-mon[31661]: pgmap v250: 97 pgs: 97 active+clean; 341 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-27T10:26:56.938 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:56 smithi042 ceph-mon[31661]: pgmap v251: 97 pgs: 97 active+clean; 354 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-05-27T10:26:57.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:56 smithi104 ceph-mon[34812]: pgmap v251: 97 pgs: 97 active+clean; 354 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-05-27T10:26:57.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:56 smithi143 ceph-mon[35545]: pgmap v251: 97 pgs: 97 active+clean; 354 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-05-27T10:26:58.717 INFO:tasks.workunit.client.0.smithi042.stdout: 54 5611 164 4396 217 1440 299 2024-05-27T10:26:59.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:26:58 smithi104 ceph-mon[34812]: pgmap v252: 97 pgs: 97 active+clean; 354 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-05-27T10:26:59.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:26:58 smithi143 ceph-mon[35545]: pgmap v252: 97 pgs: 97 active+clean; 354 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-05-27T10:26:59.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:26:58 smithi042 ceph-mon[31661]: pgmap v252: 97 pgs: 97 active+clean; 354 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-05-27T10:27:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:00 smithi143 ceph-mon[35545]: pgmap v253: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-05-27T10:27:01.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:00 smithi042 ceph-mon[31661]: pgmap v253: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-05-27T10:27:01.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:00 smithi104 ceph-mon[34812]: pgmap v253: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-05-27T10:27:03.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:02 smithi143 ceph-mon[35545]: pgmap v254: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-05-27T10:27:03.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:02 smithi042 ceph-mon[31661]: pgmap v254: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-05-27T10:27:03.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:02 smithi104 ceph-mon[34812]: pgmap v254: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-05-27T10:27:05.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:04 smithi143 ceph-mon[35545]: pgmap v255: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-05-27T10:27:05.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:04 smithi042 ceph-mon[31661]: pgmap v255: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-05-27T10:27:05.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:04 smithi104 ceph-mon[34812]: pgmap v255: 97 pgs: 97 active+clean; 354 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-05-27T10:27:06.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:05 smithi143 ceph-mon[35545]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T10:27:06.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:05 smithi143 ceph-mon[35545]: pgmap v257: 97 pgs: 97 active+clean; 359 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-05-27T10:27:06.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:05 smithi042 ceph-mon[31661]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T10:27:06.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:05 smithi042 ceph-mon[31661]: pgmap v257: 97 pgs: 97 active+clean; 359 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-05-27T10:27:06.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:05 smithi104 ceph-mon[34812]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T10:27:06.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:05 smithi104 ceph-mon[34812]: pgmap v257: 97 pgs: 97 active+clean; 359 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-05-27T10:27:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 -- 172.21.15.42:0/404598044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc440110540 msgr2=0x7fc440112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 --2- 172.21.15.42:0/404598044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc440110540 0x7fc440112960 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc438009be0 tx=0x7fc43802a6c0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 -- 172.21.15.42:0/404598044 shutdown_connections 2024-05-27T10:27:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 --2- 172.21.15.42:0/404598044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc440110540 0x7fc440112960 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 --2- 172.21.15.42:0/404598044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc44010dbe0 0x7fc440110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 --2- 172.21.15.42:0/404598044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4401029a0 0x7fc440102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.009+0000 7fc446200640 1 -- 172.21.15.42:0/404598044 >> 172.21.15.42:0/404598044 conn(0x7fc4400782d0 msgr2=0x7fc440078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.010+0000 7fc446200640 1 -- 172.21.15.42:0/404598044 shutdown_connections 2024-05-27T10:27:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.010+0000 7fc446200640 1 -- 172.21.15.42:0/404598044 wait complete. 2024-05-27T10:27:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.011+0000 7fc446200640 1 Processor -- start 2024-05-27T10:27:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.011+0000 7fc446200640 1 -- start start 2024-05-27T10:27:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc446200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 0x7fc4401a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc446200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44010dbe0 0x7fc4401a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc446200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc440110540 0x7fc44019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc446200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc440115ee0 con 0x7fc4401029a0 2024-05-27T10:27:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc446200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc440115d60 con 0x7fc440110540 2024-05-27T10:27:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc446200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc440116060 con 0x7fc44010dbe0 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 0x7fc4401a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 0x7fc4401a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54414/0 (socket says 172.21.15.42:54414) 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc43f400640 1 -- 172.21.15.42:0/1755837864 learned_addr learned my addr 172.21.15.42:0/1755837864 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.012+0000 7fc43ea00640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44010dbe0 0x7fc4401a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.013+0000 7fc43fe00640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc440110540 0x7fc44019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.013+0000 7fc43f400640 1 -- 172.21.15.42:0/1755837864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44010dbe0 msgr2=0x7fc4401a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.013+0000 7fc43f400640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44010dbe0 0x7fc4401a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.013+0000 7fc43f400640 1 -- 172.21.15.42:0/1755837864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc440110540 msgr2=0x7fc44019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.013+0000 7fc43f400640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc440110540 0x7fc44019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.013+0000 7fc43f400640 1 -- 172.21.15.42:0/1755837864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44019ce80 con 0x7fc4401029a0 2024-05-27T10:27:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.014+0000 7fc43f400640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 0x7fc4401a2570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc43000ea60 tx=0x7fc43000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.014+0000 7fc437e00640 1 -- 172.21.15.42:0/1755837864 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43000cd20 con 0x7fc4401029a0 2024-05-27T10:27:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.014+0000 7fc437e00640 1 -- 172.21.15.42:0/1755837864 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4300035f0 con 0x7fc4401029a0 2024-05-27T10:27:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.014+0000 7fc437e00640 1 -- 172.21.15.42:0/1755837864 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430010690 con 0x7fc4401029a0 2024-05-27T10:27:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.014+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44019d170 con 0x7fc4401029a0 2024-05-27T10:27:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.015+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4401a93c0 con 0x7fc4401029a0 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.019+0000 7fc44d216740 1 Processor -- start 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 -- start start 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e4a5150b90 con 0x55e4a4e3d1b0 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4a5140730 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.020+0000 7fc44d216740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e4a5144770 con 0x55e4a4e142f0 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.021+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.021+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54416/0 (socket says 172.21.15.42:54416) 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.021+0000 7fc43fe00640 1 -- 172.21.15.42:0/2601976756 learned_addr learned my addr 172.21.15.42:0/2601976756 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.021+0000 7fc437e00640 1 -- 172.21.15.42:0/1755837864 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc430004000 con 0x7fc4401029a0 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.021+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.021+0000 7fc43f400640 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.022+0000 7fc43ea00640 1 -- 172.21.15.42:0/2601976756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 msgr2=0x55e4a51e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.022+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.022+0000 7fc43ea00640 1 -- 172.21.15.42:0/2601976756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e3d1b0 msgr2=0x55e4a51e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.022+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.022+0000 7fc43ea00640 1 -- 172.21.15.42:0/2601976756 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a50d98b0 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.022+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51e8af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc440112110 tx=0x7fc42c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b9b78b5a9d32da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.023+0000 7fc437e00640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc4240b2a50 0x7fc4240b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.023+0000 7fc42be00640 1 -- 172.21.15.42:0/2601976756 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c004050 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.023+0000 7fc437e00640 1 -- 172.21.15.42:0/1755837864 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fc4300e28d0 con 0x7fc4401029a0 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.023+0000 7fc43fe00640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc4240b2a50 0x7fc4240b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.023+0000 7fc42be00640 1 -- 172.21.15.42:0/2601976756 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42c0035f0 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.024+0000 7fc42be00640 1 -- 172.21.15.42:0/2601976756 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c019690 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc43fe00640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc4240b2a50 0x7fc4240b4f60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4380099a0 tx=0x7fc438005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 -- 172.21.15.42:0/2601976756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e1c4d0 msgr2=0x55e4a51e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51e8af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc440112110 tx=0x7fc42c011810 comp rx=0 tx=0).stop 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 -- 172.21.15.42:0/2601976756 shutdown_connections 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51e8af0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 --2- 172.21.15.42:0/2601976756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 -- 172.21.15.42:0/2601976756 >> 172.21.15.42:0/2601976756 conn(0x55e4a4e31300 msgr2=0x55e4a51e4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 -- 172.21.15.42:0/2601976756 shutdown_connections 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.025+0000 7fc44d216740 1 -- 172.21.15.42:0/2601976756 wait complete. 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.026+0000 7fc44d216740 1 -- start start 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.026+0000 7fc44d216740 1 Processor -- start 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e4a4e26790 con 0x55e4a4e142f0 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4a50e8bc0 con 0x55e4a4e3d1b0 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e4a51f51a0 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.027+0000 7fc44d216740 10 client.0 _ll_register_callbacks cb 0x55e4a50c6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54432/0 (socket says 172.21.15.42:54432) 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 -- 172.21.15.42:0/1037004498 learned_addr learned my addr 172.21.15.42:0/1037004498 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43fe00640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43ea00640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 -- 172.21.15.42:0/1037004498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e1c4d0 msgr2=0x55e4a51f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 -- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 msgr2=0x55e4a51fe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51fe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.028+0000 7fc43f400640 1 -- 172.21.15.42:0/1037004498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a4e55210 con 0x55e4a4e142f0 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.029+0000 7fc43ea00640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:27:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.029+0000 7fc43fe00640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.029+0000 7fc43f400640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51f2230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc43000c6a0 tx=0x7fc43000cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2f9097fac2621 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.029+0000 7fc428c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.029+0000 7fc428c00640 1 -- 172.21.15.42:0/1037004498 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4300a94c0 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.029+0000 7fc428c00640 1 -- 172.21.15.42:0/1037004498 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fc4300e0260 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.030+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc42c0035f0 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.030+0000 7fc428c00640 1 -- 172.21.15.42:0/1037004498 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc4300c3050 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.030+0000 7fc44d216740 10 client.16737 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.030+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e4a4f21620 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.030+0000 7fc428c00640 1 -- 172.21.15.42:0/1037004498 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42c0035f0 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc428c00640 1 -- 172.21.15.42:0/1037004498 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4300a5e00 con 0x55e4a4e142f0 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc44d216740 10 client.16737 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc44d216740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc44d216740 20 client.16737 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc44d216740 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a4dfbd80 0x55e4a5204830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc44d216740 4 client.16737 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:27:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.031+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a5152ae0 con 0x55e4a4dfbd80 2024-05-27T10:27:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.032+0000 7fc43ea00640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a4dfbd80 0x55e4a5204830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.033+0000 7fc43ea00640 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a4dfbd80 0x55e4a5204830 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f47eeb15a0a1c1 server_cookie=416371b0c9ed7830 in_seq=0 out_seq=0 2024-05-27T10:27:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.033+0000 7fc428c00640 10 client.16737 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc428c00640 1 -- 172.21.15.42:0/1037004498 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e4a5152ae0 con 0x55e4a4dfbd80 2024-05-27T10:27:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc428c00640 10 client.16737 handle_command_reply: tid=0 2024-05-27T10:27:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 client.16737 shutdown 2024-05-27T10:27:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 msgr2=0x55e4a51f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51f2230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc43000c6a0 tx=0x7fc43000cfc0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 shutdown_connections 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a4dfbd80 0x55e4a5204830 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 --2- 172.21.15.42:0/1037004498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51f2230 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc428c00640 0 client.16737 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.035+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 >> 172.21.15.42:0/1037004498 conn(0x55e4a4e31300 msgr2=0x55e4a51e9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.036+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 shutdown_connections 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.036+0000 7fc44d216740 1 -- 172.21.15.42:0/1037004498 wait complete. 2024-05-27T10:27:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.036+0000 7fc44d216740 20 client.16737 trim_cache size 0 max 16384 2024-05-27T10:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 Processor -- start 2024-05-27T10:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 -- start start 2024-05-27T10:27:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b02f0 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401adb60 con 0x55e4a4e3d1b0 2024-05-27T10:27:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.063+0000 7fc44d216740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401ab9f0 con 0x55e4a4e142f0 2024-05-27T10:27:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.064+0000 7fc43ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.064+0000 7fc43f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.064+0000 7fc43f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51eda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:37840/0 (socket says 172.21.15.42:37840) 2024-05-27T10:27:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.064+0000 7fc43f400640 1 -- 172.21.15.42:0/2733608112 learned_addr learned my addr 172.21.15.42:0/2733608112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.064+0000 7fc43fe00640 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43ea00640 1 -- 172.21.15.42:0/2733608112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 msgr2=0x55e4a51eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43ea00640 1 -- 172.21.15.42:0/2733608112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 msgr2=0x55e4a51e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43ea00640 1 -- 172.21.15.42:0/2733608112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a51f4d00 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43f400640 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:27:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.065+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f4380 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009230 tx=0x7fc42c009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fd5a04d64f093a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.066+0000 7fc42b400640 1 -- 172.21.15.42:0/2733608112 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c003a00 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.066+0000 7fc42b400640 1 -- 172.21.15.42:0/2733608112 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42c029070 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.066+0000 7fc42b400640 1 -- 172.21.15.42:0/2733608112 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c019890 con 0x55e4a4e1c4d0 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.067+0000 7fc44d216740 1 -- 172.21.15.42:0/2733608112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e1c4d0 msgr2=0x55e4a51f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.067+0000 7fc44d216740 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f4380 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009230 tx=0x7fc42c009970 comp rx=0 tx=0).stop 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 -- 172.21.15.42:0/2733608112 shutdown_connections 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e1c4d0 0x55e4a51f4380 secure :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009230 tx=0x7fc42c009970 comp rx=0 tx=0).stop 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 --2- 172.21.15.42:0/2733608112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e142f0 0x55e4a51eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 -- 172.21.15.42:0/2733608112 >> 172.21.15.42:0/2733608112 conn(0x55e4a4e31300 msgr2=0x55e4a51f1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 -- 172.21.15.42:0/2733608112 shutdown_connections 2024-05-27T10:27:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.068+0000 7fc44d216740 1 -- 172.21.15.42:0/2733608112 wait complete. 2024-05-27T10:27:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 -- start start 2024-05-27T10:27:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 Processor -- start 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4dd11c0 0x55e4a51f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401ac040 con 0x55e4a4e142f0 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401ad490 con 0x55e4a4dd11c0 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.069+0000 7fc44d216740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401ac6f0 con 0x55e4a4e3d1b0 2024-05-27T10:27:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.070+0000 7fc44d216740 10 client.0 _ll_register_callbacks cb 0x7fc43000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.070+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.070+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51ec2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54438/0 (socket says 172.21.15.42:54438) 2024-05-27T10:27:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.070+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4dd11c0 0x55e4a51f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.070+0000 7fc43f400640 1 -- 172.21.15.42:0/2442324700 learned_addr learned my addr 172.21.15.42:0/2442324700 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.070+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43f400640 1 -- 172.21.15.42:0/2442324700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e3d1b0 msgr2=0x55e4a51ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43f400640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43f400640 1 -- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4dd11c0 msgr2=0x55e4a51f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43f400640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4dd11c0 0x55e4a51f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43f400640 1 -- 172.21.15.42:0/2442324700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a51ed0b0 con 0x55e4a4e142f0 2024-05-27T10:27:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc43f400640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51ec2d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc4300e2cd0 tx=0x7fc4300a6030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413e103e1e41b571 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc428c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:27:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc428c00640 1 -- 172.21.15.42:0/2442324700 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4300042f0 con 0x55e4a4e142f0 2024-05-27T10:27:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc428c00640 1 -- 172.21.15.42:0/2442324700 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8528+0+0 (secure 0 0 0) 0x7fc4300a8dd0 con 0x55e4a4e142f0 2024-05-27T10:27:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.071+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc42c029070 con 0x55e4a4e142f0 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.072+0000 7fc428c00640 1 -- 172.21.15.42:0/2442324700 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc43008f050 con 0x55e4a4e142f0 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.072+0000 7fc44d216740 10 client.16743 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.072+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e4a51fa9d0 con 0x55e4a4e142f0 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.072+0000 7fc428c00640 1 -- 172.21.15.42:0/2442324700 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42c029070 con 0x55e4a4e142f0 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc428c00640 1 -- 172.21.15.42:0/2442324700 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4300a8840 con 0x55e4a4e142f0 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc44d216740 10 client.16743 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc44d216740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc44d216740 20 client.16743 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc44d216740 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a526cf00 0x55e4a526f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc44d216740 4 client.16743 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:27:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4a514c2c0 con 0x55e4a526cf00 2024-05-27T10:27:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.073+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a526cf00 0x55e4a526f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.074+0000 7fc43ea00640 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a526cf00 0x55e4a526f320 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfd4bb9b4fa7d51 server_cookie=65b1380a94e94317 in_seq=0 out_seq=0 2024-05-27T10:27:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.074+0000 7fc428c00640 10 client.16743 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc428c00640 1 -- 172.21.15.42:0/2442324700 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4a514c2c0 con 0x55e4a526cf00 2024-05-27T10:27:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc428c00640 10 client.16743 handle_command_reply: tid=0 2024-05-27T10:27:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 client.16743 shutdown 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 msgr2=0x55e4a51ec2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51ec2d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc4300e2cd0 tx=0x7fc4300a6030 comp rx=0 tx=0).stop 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 shutdown_connections 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4a4dd11c0 0x55e4a51f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55e4a526cf00 0x55e4a526f320 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4a4e3d1b0 0x55e4a51ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 --2- 172.21.15.42:0/2442324700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e4a4e142f0 0x55e4a51ec2d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 >> 172.21.15.42:0/2442324700 conn(0x55e4a4e31300 msgr2=0x55e4a51f24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.156+0000 7fc428c00640 0 client.16743 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.157+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 shutdown_connections 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.157+0000 7fc44d216740 1 -- 172.21.15.42:0/2442324700 wait complete. 2024-05-27T10:27:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.157+0000 7fc44d216740 20 client.16743 trim_cache size 0 max 16384 2024-05-27T10:27:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:27:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:27:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc4240b2a50 msgr2=0x7fc4240b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc4240b2a50 0x7fc4240b4f60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4380099a0 tx=0x7fc438005fd0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 msgr2=0x7fc4401a2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 0x7fc4401a2570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc43000ea60 tx=0x7fc43000ee60 comp rx=0 tx=0).stop 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 shutdown_connections 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fc4240b2a50 0x7fc4240b4f60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc440110540 0x7fc44019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44010dbe0 0x7fc4401a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 --2- 172.21.15.42:0/1755837864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4401029a0 0x7fc4401a2570 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.158+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 >> 172.21.15.42:0/1755837864 conn(0x7fc4400782d0 msgr2=0x7fc4401012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.159+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 shutdown_connections 2024-05-27T10:27:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:07.159+0000 7fc446200640 1 -- 172.21.15.42:0/1755837864 wait complete. 2024-05-27T10:27:08.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:08 smithi104 ceph-mon[34812]: pgmap v258: 97 pgs: 97 active+clean; 359 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-05-27T10:27:08.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:08 smithi042 ceph-mon[31661]: pgmap v258: 97 pgs: 97 active+clean; 359 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-05-27T10:27:08.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:08 smithi143 ceph-mon[35545]: pgmap v258: 97 pgs: 97 active+clean; 359 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-05-27T10:27:08.717 INFO:tasks.workunit.client.0.smithi042.stdout: 57 4317 220 3396 118 1143 176 2024-05-27T10:27:10.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:10 smithi104 ceph-mon[34812]: pgmap v259: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 242 op/s 2024-05-27T10:27:10.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:10 smithi143 ceph-mon[35545]: pgmap v259: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 242 op/s 2024-05-27T10:27:10.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:10 smithi042 ceph-mon[31661]: pgmap v259: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 242 op/s 2024-05-27T10:27:12.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:11 smithi042 ceph-mon[31661]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T10:27:12.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:11 smithi104 ceph-mon[34812]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T10:27:12.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:11 smithi143 ceph-mon[35545]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T10:27:13.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:12 smithi042 ceph-mon[31661]: pgmap v261: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-27T10:27:13.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:12 smithi104 ceph-mon[34812]: pgmap v261: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-27T10:27:13.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:12 smithi143 ceph-mon[35545]: pgmap v261: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-27T10:27:15.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:14 smithi042 ceph-mon[31661]: pgmap v262: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-27T10:27:15.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:14 smithi104 ceph-mon[34812]: pgmap v262: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-27T10:27:15.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:14 smithi143 ceph-mon[35545]: pgmap v262: 97 pgs: 97 active+clean; 376 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-27T10:27:17.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:16 smithi104 ceph-mon[34812]: pgmap v263: 97 pgs: 97 active+clean; 374 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-05-27T10:27:17.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:16 smithi143 ceph-mon[35545]: pgmap v263: 97 pgs: 97 active+clean; 374 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-05-27T10:27:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:16 smithi042 ceph-mon[31661]: pgmap v263: 97 pgs: 97 active+clean; 374 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-05-27T10:27:18.717 INFO:tasks.workunit.client.0.smithi042.stdout: 61 6189 292 4427 183 1761 239 2024-05-27T10:27:19.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:18 smithi104 ceph-mon[34812]: pgmap v264: 97 pgs: 97 active+clean; 374 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-05-27T10:27:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:18 smithi143 ceph-mon[35545]: pgmap v264: 97 pgs: 97 active+clean; 374 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-05-27T10:27:19.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:18 smithi042 ceph-mon[31661]: pgmap v264: 97 pgs: 97 active+clean; 374 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-05-27T10:27:20.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:19 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:27:20.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:19 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:27:20.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:19 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:27:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 -- 172.21.15.42:0/2611786894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 msgr2=0x7f1848112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 --2- 172.21.15.42:0/2611786894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f1848112740 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1840009ca0 tx=0x7f184002a950 comp rx=0 tx=0).stop 2024-05-27T10:27:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 -- 172.21.15.42:0/2611786894 shutdown_connections 2024-05-27T10:27:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 --2- 172.21.15.42:0/2611786894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f1848112740 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 --2- 172.21.15.42:0/2611786894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f184810d9c0 0x7f184810fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 --2- 172.21.15.42:0/2611786894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1848075420 0x7f1848075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 -- 172.21.15.42:0/2611786894 >> 172.21.15.42:0/2611786894 conn(0x7f184807a1b0 msgr2=0x7f184807a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 -- 172.21.15.42:0/2611786894 shutdown_connections 2024-05-27T10:27:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.428+0000 7f1847200640 1 -- 172.21.15.42:0/2611786894 wait complete. 2024-05-27T10:27:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.429+0000 7f1847200640 1 Processor -- start 2024-05-27T10:27:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.429+0000 7f1847200640 1 -- start start 2024-05-27T10:27:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.430+0000 7f1847200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1848075420 0x7f18481a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.430+0000 7f1847200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f184810d9c0 0x7f18481a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.430+0000 7f1847200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f184819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.430+0000 7f1847200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1848115d90 con 0x7f1848110320 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.430+0000 7f1847200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1848115c10 con 0x7f1848075420 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.430+0000 7f1847200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1848115f10 con 0x7f184810d9c0 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f184819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f184819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58148/0 (socket says 172.21.15.42:58148) 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 -- 172.21.15.42:0/2076961545 learned_addr learned my addr 172.21.15.42:0/2076961545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1845e00640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1848075420 0x7f18481a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1845400640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f184810d9c0 0x7f18481a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 -- 172.21.15.42:0/2076961545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f184810d9c0 msgr2=0x7f18481a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f184810d9c0 0x7f18481a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 -- 172.21.15.42:0/2076961545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1848075420 msgr2=0x7f18481a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1848075420 0x7f18481a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1846800640 1 -- 172.21.15.42:0/2076961545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184819cc90 con 0x7f1848110320 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.431+0000 7f1845400640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f184810d9c0 0x7f18481a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.432+0000 7f1846800640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f184819c650 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f18400098d0 tx=0x7f1840003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.432+0000 7f183ea00640 1 -- 172.21.15.42:0/2076961545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840034040 con 0x7f1848110320 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.432+0000 7f183ea00640 1 -- 172.21.15.42:0/2076961545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840030880 con 0x7f1848110320 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.432+0000 7f183ea00640 1 -- 172.21.15.42:0/2076961545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840004730 con 0x7f1848110320 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.432+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184819cf20 con 0x7f1848110320 2024-05-27T10:27:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.432+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184819d460 con 0x7f1848110320 2024-05-27T10:27:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.436+0000 7f1853e29740 1 Processor -- start 2024-05-27T10:27:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.437+0000 7f183ea00640 1 -- 172.21.15.42:0/2076961545 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1840037020 con 0x7f1848110320 2024-05-27T10:27:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.437+0000 7f1853e29740 1 -- start start 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1853e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b6127a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1853e29740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6127fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1853e29740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b61284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1853e29740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6098b90 con 0x55f7b5d851b0 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1853e29740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6088730 con 0x55f7b5d644d0 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1853e29740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f7b608c770 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1845400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b6127a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1845400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b6127a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58152/0 (socket says 172.21.15.42:58152) 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1845400640 1 -- 172.21.15.42:0/2451048660 learned_addr learned my addr 172.21.15.42:0/2451048660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.438+0000 7f1845400640 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b61284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845e00640 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6127fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845400640 1 -- 172.21.15.42:0/2451048660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 msgr2=0x55f7b6127fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845400640 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6127fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845400640 1 -- 172.21.15.42:0/2451048660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 msgr2=0x55f7b61284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845400640 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b61284e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845400640 1 -- 172.21.15.42:0/2451048660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b60218b0 con 0x55f7b5d851b0 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845e00640 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6127fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1845400640 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b6127a60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1834013820 tx=0x7f1834008380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b6f59c4e92d33d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f183ea00640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f182c0b29d0 0x7f182c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.439+0000 7f1833400640 1 -- 172.21.15.42:0/2451048660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834013e60 con 0x55f7b5d851b0 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.440+0000 7f183ea00640 1 -- 172.21.15.42:0/2076961545 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f1840103690 con 0x7f1848110320 2024-05-27T10:27:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.440+0000 7f1845e00640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f182c0b29d0 0x7f182c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.440+0000 7f1833400640 1 -- 172.21.15.42:0/2451048660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18340047a0 con 0x55f7b5d851b0 2024-05-27T10:27:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.440+0000 7f1833400640 1 -- 172.21.15.42:0/2451048660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834005280 con 0x55f7b5d851b0 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 -- 172.21.15.42:0/2451048660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 msgr2=0x55f7b6127a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b6127a60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1834013820 tx=0x7f1834008380 comp rx=0 tx=0).stop 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1845e00640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f182c0b29d0 0x7f182c0b4ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55f7b6128e10 tx=0x7f183800a2d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 -- 172.21.15.42:0/2451048660 shutdown_connections 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b61284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6127fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 --2- 172.21.15.42:0/2451048660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b6127a60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 -- 172.21.15.42:0/2451048660 >> 172.21.15.42:0/2451048660 conn(0x55f7b5d79300 msgr2=0x55f7b6124210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 -- 172.21.15.42:0/2451048660 shutdown_connections 2024-05-27T10:27:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.441+0000 7f1853e29740 1 -- 172.21.15.42:0/2451048660 wait complete. 2024-05-27T10:27:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.442+0000 7f1853e29740 1 -- start start 2024-05-27T10:27:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.442+0000 7f1853e29740 1 Processor -- start 2024-05-27T10:27:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b6132180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b613de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f7b5d6e790 con 0x55f7b5d851b0 2024-05-27T10:27:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6030bc0 con 0x55f7b5d644d0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6134b30 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1853e29740 10 client.0 _ll_register_callbacks cb 0x55f7b600e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1845400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b6132180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1845400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b6132180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:44430/0 (socket says 172.21.15.42:44430) 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1845400640 1 -- 172.21.15.42:0/331562482 learned_addr learned my addr 172.21.15.42:0/331562482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1846800640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b613de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.443+0000 7f1845e00640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1845400640 1 -- 172.21.15.42:0/331562482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 msgr2=0x55f7b6131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1845400640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6131c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1845400640 1 -- 172.21.15.42:0/331562482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 msgr2=0x55f7b613de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1845400640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b613de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1845400640 1 -- 172.21.15.42:0/331562482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b5d9d210 con 0x55f7b5d644d0 2024-05-27T10:27:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1845400640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b6132180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18340085b0 tx=0x7f183400cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd36630c066b9b56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.444+0000 7f1827e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:27:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.445+0000 7f1827e00640 1 -- 172.21.15.42:0/331562482 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183402c060 con 0x55f7b5d644d0 2024-05-27T10:27:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.445+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18340047a0 con 0x55f7b5d644d0 2024-05-27T10:27:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.445+0000 7f1827e00640 1 -- 172.21.15.42:0/331562482 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f1834004f20 con 0x55f7b5d644d0 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.445+0000 7f1827e00640 1 -- 172.21.15.42:0/331562482 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1834013e60 con 0x55f7b5d644d0 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.445+0000 7f1853e29740 10 client.36263 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.445+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f7b5e69620 con 0x55f7b5d644d0 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1827e00640 1 -- 172.21.15.42:0/331562482 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18340047a0 con 0x55f7b5d644d0 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1827e00640 1 -- 172.21.15.42:0/331562482 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1834005320 con 0x55f7b5d644d0 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1853e29740 10 client.36263 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1853e29740 10 client.36263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1853e29740 20 client.36263 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1853e29740 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d43d80 0x55f7b6143d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1853e29740 4 client.36263 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:27:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.446+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b609aae0 con 0x55f7b5d43d80 2024-05-27T10:27:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.447+0000 7f1845e00640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d43d80 0x55f7b6143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.447+0000 7f1845e00640 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d43d80 0x55f7b6143d70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79511d6f9387694 server_cookie=830315bbf3058ca0 in_seq=0 out_seq=0 2024-05-27T10:27:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.447+0000 7f1827e00640 10 client.36263 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.448+0000 7f1827e00640 1 -- 172.21.15.42:0/331562482 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f7b609aae0 con 0x55f7b5d43d80 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.448+0000 7f1827e00640 10 client.36263 handle_command_reply: tid=0 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.448+0000 7f1853e29740 1 client.36263 shutdown 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 msgr2=0x55f7b6132180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b6132180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18340085b0 tx=0x7f183400cb10 comp rx=0 tx=0).stop 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 shutdown_connections 2024-05-27T10:27:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b613de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d644d0 0x55f7b6132180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d43d80 0x55f7b6143d70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 --2- 172.21.15.42:0/331562482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b6131c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 >> 172.21.15.42:0/331562482 conn(0x55f7b5d79300 msgr2=0x55f7b6127830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1827e00640 0 client.36263 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 shutdown_connections 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.449+0000 7f1853e29740 1 -- 172.21.15.42:0/331562482 wait complete. 2024-05-27T10:27:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.450+0000 7f1853e29740 20 client.36263 trim_cache size 0 max 16384 2024-05-27T10:27:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.478+0000 7f1853e29740 1 Processor -- start 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 -- start start 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d644d0 0x55f7b6134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 0x55f7b613de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b4260 con 0x55f7b5d644d0 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b1ad0 con 0x55f7b5d851b0 2024-05-27T10:27:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.479+0000 7f1853e29740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f18481af940 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.480+0000 7f1846800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 0x55f7b613de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.480+0000 7f1846800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 0x55f7b613de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:44440/0 (socket says 172.21.15.42:44440) 2024-05-27T10:27:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.480+0000 7f1846800640 1 -- 172.21.15.42:0/3281750725 learned_addr learned my addr 172.21.15.42:0/3281750725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.480+0000 7f1845400640 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d644d0 0x55f7b6134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.480+0000 7f1845e00640 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1846800640 1 -- 172.21.15.42:0/3281750725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 msgr2=0x55f7b61232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1846800640 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1846800640 1 -- 172.21.15.42:0/3281750725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d644d0 msgr2=0x55f7b6134370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1846800640 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d644d0 0x55f7b6134370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1846800640 1 -- 172.21.15.42:0/3281750725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b6130190 con 0x55f7b5d851b0 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1845400640 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d644d0 0x55f7b6134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:27:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.481+0000 7f1846800640 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 0x55f7b613de00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18400ca740 tx=0x7f18400ca450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e6bbca84cdb2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.482+0000 7f1832a00640 1 -- 172.21.15.42:0/3281750725 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400d3050 con 0x55f7b5d851b0 2024-05-27T10:27:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.482+0000 7f1832a00640 1 -- 172.21.15.42:0/3281750725 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18400ca550 con 0x55f7b5d851b0 2024-05-27T10:27:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.482+0000 7f1832a00640 1 -- 172.21.15.42:0/3281750725 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184002c040 con 0x55f7b5d851b0 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 -- 172.21.15.42:0/3281750725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 msgr2=0x55f7b613de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 0x55f7b613de00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18400ca740 tx=0x7f18400ca450 comp rx=0 tx=0).stop 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 -- 172.21.15.42:0/3281750725 shutdown_connections 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d851b0 0x55f7b613de00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d644d0 0x55f7b6134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 --2- 172.21.15.42:0/3281750725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 -- 172.21.15.42:0/3281750725 >> 172.21.15.42:0/3281750725 conn(0x55f7b5d79300 msgr2=0x55f7b6128220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 -- 172.21.15.42:0/3281750725 shutdown_connections 2024-05-27T10:27:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.483+0000 7f1853e29740 1 -- 172.21.15.42:0/3281750725 wait complete. 2024-05-27T10:27:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.484+0000 7f1853e29740 1 -- start start 2024-05-27T10:27:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.484+0000 7f1853e29740 1 Processor -- start 2024-05-27T10:27:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d644d0 0x55f7b61a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b61442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18481aff90 con 0x55f7b5d851b0 2024-05-27T10:27:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b1400 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b0660 con 0x55f7b5d644d0 2024-05-27T10:27:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.485+0000 7f1853e29740 10 client.0 _ll_register_callbacks cb 0x7f1834008de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61a34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:44446/0 (socket says 172.21.15.42:44446) 2024-05-27T10:27:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 -- 172.21.15.42:0/3946481668 learned_addr learned my addr 172.21.15.42:0/3946481668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845400640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d644d0 0x55f7b61a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1846800640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b61442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 -- 172.21.15.42:0/3946481668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d644d0 msgr2=0x55f7b61a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d644d0 0x55f7b61a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 -- 172.21.15.42:0/3946481668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 msgr2=0x55f7b61442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.486+0000 7f1845e00640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b61442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.487+0000 7f1845e00640 1 -- 172.21.15.42:0/3946481668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b6144a00 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.487+0000 7f1845e00640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61a34d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1838012fd0 tx=0x7f18380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ff4baaab86e49b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.487+0000 7f1827e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:27:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.488+0000 7f1827e00640 1 -- 172.21.15.42:0/3946481668 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838004220 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.488+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18400ca550 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.488+0000 7f1827e00640 1 -- 172.21.15.42:0/3946481668 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f1838004020 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.489+0000 7f1827e00640 1 -- 172.21.15.42:0/3946481668 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1838004220 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.489+0000 7f1853e29740 10 client.36269 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.489+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f7b61abc40 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1827e00640 1 -- 172.21.15.42:0/3946481668 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18400ca550 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1827e00640 1 -- 172.21.15.42:0/3946481668 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1838003970 con 0x55f7b5d5c2f0 2024-05-27T10:27:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1853e29740 10 client.36269 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1853e29740 10 client.36269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1853e29740 20 client.36269 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1853e29740 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d191c0 0x55f7b61ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1853e29740 4 client.36269 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7b60942c0 con 0x55f7b5d191c0 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.490+0000 7f1845400640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d191c0 0x55f7b61ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.491+0000 7f1827e00640 10 client.36269 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.491+0000 7f1845400640 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d191c0 0x55f7b61ae390 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7a10cfe7e4b08e server_cookie=27781a8cde4e3de5 in_seq=0 out_seq=0 2024-05-27T10:27:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1827e00640 1 -- 172.21.15.42:0/3946481668 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f7b60942c0 con 0x55f7b5d191c0 2024-05-27T10:27:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1827e00640 10 client.36269 handle_command_reply: tid=0 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 client.36269 shutdown 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 msgr2=0x55f7b61a34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61a34d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1838012fd0 tx=0x7f18380046c0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 shutdown_connections 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b5d851b0 0x55f7b61442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55f7b5d191c0 0x55f7b61ae390 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f7b5d644d0 0x55f7b61a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 --2- 172.21.15.42:0/3946481668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7b5d5c2f0 0x55f7b61a34d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 >> 172.21.15.42:0/3946481668 conn(0x55f7b5d79300 msgr2=0x55f7b6127f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.551+0000 7f1827e00640 0 client.36269 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.552+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 shutdown_connections 2024-05-27T10:27:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.552+0000 7f1853e29740 1 -- 172.21.15.42:0/3946481668 wait complete. 2024-05-27T10:27:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.552+0000 7f1853e29740 20 client.36269 trim_cache size 0 max 16384 2024-05-27T10:27:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:27:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:27:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:27:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.553+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f182c0b29d0 msgr2=0x7f182c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.553+0000 7f1847200640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f182c0b29d0 0x7f182c0b4ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55f7b6128e10 tx=0x7f183800a2d0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.553+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 msgr2=0x7f184819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.553+0000 7f1847200640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f184819c650 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f18400098d0 tx=0x7f1840003f80 comp rx=0 tx=0).stop 2024-05-27T10:27:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 shutdown_connections 2024-05-27T10:27:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f182c0b29d0 0x7f182c0b4ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1848110320 0x7f184819c650 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f184810d9c0 0x7f18481a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 --2- 172.21.15.42:0/2076961545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1848075420 0x7f18481a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 >> 172.21.15.42:0/2076961545 conn(0x7f184807a1b0 msgr2=0x7f18481021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 shutdown_connections 2024-05-27T10:27:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:20.554+0000 7f1847200640 1 -- 172.21.15.42:0/2076961545 wait complete. 2024-05-27T10:27:21.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:20 smithi104 ceph-mon[34812]: pgmap v265: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-05-27T10:27:21.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:20 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:27:21.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:20 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:27:21.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:20 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:27:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:20 smithi143 ceph-mon[35545]: pgmap v265: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-05-27T10:27:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:20 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:27:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:20 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:27:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:20 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:27:21.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:20 smithi042 ceph-mon[31661]: pgmap v265: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-05-27T10:27:21.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:20 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:27:21.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:20 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:27:21.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:20 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:27:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:22 smithi104 ceph-mon[34812]: pgmap v266: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 285 op/s 2024-05-27T10:27:23.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:22 smithi143 ceph-mon[35545]: pgmap v266: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 285 op/s 2024-05-27T10:27:23.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:22 smithi042 ceph-mon[31661]: pgmap v266: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 285 op/s 2024-05-27T10:27:25.146 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:24 smithi042 ceph-mon[31661]: pgmap v267: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-05-27T10:27:25.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:24 smithi104 ceph-mon[34812]: pgmap v267: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-05-27T10:27:25.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:24 smithi143 ceph-mon[35545]: pgmap v267: 97 pgs: 97 active+clean; 390 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-05-27T10:27:27.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:26 smithi104 ceph-mon[34812]: pgmap v268: 97 pgs: 97 active+clean; 375 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-05-27T10:27:27.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:26 smithi143 ceph-mon[35545]: pgmap v268: 97 pgs: 97 active+clean; 375 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-05-27T10:27:27.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:26 smithi042 ceph-mon[31661]: pgmap v268: 97 pgs: 97 active+clean; 375 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-05-27T10:27:29.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:28 smithi104 ceph-mon[34812]: pgmap v269: 97 pgs: 97 active+clean; 375 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-05-27T10:27:29.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:28 smithi143 ceph-mon[35545]: pgmap v269: 97 pgs: 97 active+clean; 375 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-05-27T10:27:29.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:28 smithi042 ceph-mon[31661]: pgmap v269: 97 pgs: 97 active+clean; 375 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-05-27T10:27:31.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:30 smithi104 ceph-mon[34812]: pgmap v270: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-27T10:27:31.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:30 smithi143 ceph-mon[35545]: pgmap v270: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-27T10:27:31.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:30 smithi042 ceph-mon[31661]: pgmap v270: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-27T10:27:31.749 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-05-27T10:27:31.749 INFO:tasks.workunit.client.0.smithi042.stdout:Final score for writes: 61 2024-05-27T10:27:31.749 INFO:tasks.workunit.client.0.smithi042.stdout:Final score for reads : 3657 2024-05-27T10:27:31.749 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-05-27T10:27:31.752 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T10:27:31.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T10:27:33.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:32 smithi104 ceph-mon[34812]: pgmap v271: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 914 KiB/s wr, 81 op/s 2024-05-27T10:27:33.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:32 smithi143 ceph-mon[35545]: pgmap v271: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 914 KiB/s wr, 81 op/s 2024-05-27T10:27:33.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:32 smithi042 ceph-mon[31661]: pgmap v271: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 914 KiB/s wr, 81 op/s 2024-05-27T10:27:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 -- 172.21.15.42:0/435228397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410a340 msgr2=0x7f525410a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 --2- 172.21.15.42:0/435228397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410a340 0x7f525410a750 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5244009900 tx=0x7f524402a410 comp rx=0 tx=0).stop 2024-05-27T10:27:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 -- 172.21.15.42:0/435228397 shutdown_connections 2024-05-27T10:27:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 --2- 172.21.15.42:0/435228397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f525410f390 0x7f5254116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 --2- 172.21.15.42:0/435228397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410e8d0 0x7f525410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 --2- 172.21.15.42:0/435228397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410a340 0x7f525410a750 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 -- 172.21.15.42:0/435228397 >> 172.21.15.42:0/435228397 conn(0x7f5254076cd0 msgr2=0x7f5254077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 -- 172.21.15.42:0/435228397 shutdown_connections 2024-05-27T10:27:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.796+0000 7f5253e00640 1 -- 172.21.15.42:0/435228397 wait complete. 2024-05-27T10:27:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.797+0000 7f5253e00640 1 Processor -- start 2024-05-27T10:27:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.797+0000 7f5253e00640 1 -- start start 2024-05-27T10:27:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f5253e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410a340 0x7f52541a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f5253e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 0x7f52541a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f5253e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f525410f390 0x7f52541a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f5253e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f525411a170 con 0x7f525410e8d0 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f5253e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5254119ff0 con 0x7f525410f390 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f5253e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f525411a2f0 con 0x7f525410a340 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f524be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 0x7f52541a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f524be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 0x7f52541a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56660/0 (socket says 172.21.15.42:56660) 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.798+0000 7f524be00640 1 -- 172.21.15.42:0/3505557008 learned_addr learned my addr 172.21.15.42:0/3505557008 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f5252a00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410a340 0x7f52541a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f5253400640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f525410f390 0x7f52541a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524be00640 1 -- 172.21.15.42:0/3505557008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410a340 msgr2=0x7f52541a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524be00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410a340 0x7f52541a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524be00640 1 -- 172.21.15.42:0/3505557008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f525410f390 msgr2=0x7f52541a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524be00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f525410f390 0x7f52541a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524be00640 1 -- 172.21.15.42:0/3505557008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52541a1170 con 0x7f525410e8d0 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f5252a00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410a340 0x7f52541a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524be00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 0x7f52541a6db0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f523c00d740 tx=0x7f523c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.799+0000 7f524b400640 1 -- 172.21.15.42:0/3505557008 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c017070 con 0x7f525410e8d0 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.800+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52541a1460 con 0x7f525410e8d0 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.800+0000 7f524b400640 1 -- 172.21.15.42:0/3505557008 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f523c00de50 con 0x7f525410e8d0 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.800+0000 7f524b400640 1 -- 172.21.15.42:0/3505557008 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c004c60 con 0x7f525410e8d0 2024-05-27T10:27:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.800+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52541ad6f0 con 0x7f525410e8d0 2024-05-27T10:27:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.803+0000 7f52605ac740 1 Processor -- start 2024-05-27T10:27:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.803+0000 7f52605ac740 1 -- start start 2024-05-27T10:27:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f524b400640 1 -- 172.21.15.42:0/3505557008 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f523c004e00 con 0x7f525410e8d0 2024-05-27T10:27:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f52605ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f715ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f52605ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f715bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f52605ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d984d0 0x55c2f715c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f52605ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c2f70ccb90 con 0x55c2f6db91b0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f52605ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c2f70bc730 con 0x55c2f6d984d0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f52605ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c2f70c0770 con 0x55c2f6d902f0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f5253400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f715ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f5253400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f715ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56672/0 (socket says 172.21.15.42:56672) 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f5253400640 1 -- 172.21.15.42:0/2428041737 learned_addr learned my addr 172.21.15.42:0/2428041737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f5252a00640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f715bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.804+0000 7f524be00640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d984d0 0x55c2f715c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5253400640 1 -- 172.21.15.42:0/2428041737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 msgr2=0x55c2f715bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5253400640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f715bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5253400640 1 -- 172.21.15.42:0/2428041737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d984d0 msgr2=0x55c2f715c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5253400640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d984d0 0x55c2f715c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5253400640 1 -- 172.21.15.42:0/2428041737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f70558b0 con 0x55c2f6db91b0 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5252a00640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f715bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f524be00640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d984d0 0x55c2f715c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:27:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5253400640 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f715ba60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e60 tx=0x7f524c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3000f6475e4e7bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5243400640 1 -- 172.21.15.42:0/2428041737 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c016070 con 0x55c2f6db91b0 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f524b400640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52380b2950 0x7f52380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5243400640 1 -- 172.21.15.42:0/2428041737 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f524c004d20 con 0x55c2f6db91b0 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.805+0000 7f5243400640 1 -- 172.21.15.42:0/2428041737 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c0126b0 con 0x55c2f6db91b0 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.806+0000 7f524b400640 1 -- 172.21.15.42:0/3505557008 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f523c0e20d0 con 0x7f525410e8d0 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.806+0000 7f5252a00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52380b2950 0x7f52380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.806+0000 7f52605ac740 1 -- 172.21.15.42:0/2428041737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 msgr2=0x55c2f715ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.806+0000 7f52605ac740 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f715ba60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e60 tx=0x7f524c010740 comp rx=0 tx=0).stop 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 -- 172.21.15.42:0/2428041737 shutdown_connections 2024-05-27T10:27:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d984d0 0x55c2f715c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f715bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 --2- 172.21.15.42:0/2428041737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f715ba60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 -- 172.21.15.42:0/2428041737 >> 172.21.15.42:0/2428041737 conn(0x55c2f6dad300 msgr2=0x55c2f7158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 -- 172.21.15.42:0/2428041737 shutdown_connections 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f5252a00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52380b2950 0x7f52380b4e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5244006010 tx=0x7f52440058b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 -- 172.21.15.42:0/2428041737 wait complete. 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.807+0000 7f52605ac740 1 -- start start 2024-05-27T10:27:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 Processor -- start 2024-05-27T10:27:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 0x55c2f7165be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7166120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f7171ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c2f6da2790 con 0x55c2f6d902f0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7064bc0 con 0x55c2f6db91b0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7168b90 con 0x55c2f6d984d0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f52605ac740 10 client.0 _ll_register_callbacks cb 0x55c2f7042400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f5252a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 0x55c2f7165be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f5252a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 0x55c2f7165be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56674/0 (socket says 172.21.15.42:56674) 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.808+0000 7f5252a00640 1 -- 172.21.15.42:0/3553641560 learned_addr learned my addr 172.21.15.42:0/3553641560 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5253400640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f7171ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f524be00640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5252a00640 1 -- 172.21.15.42:0/3553641560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 msgr2=0x55c2f7166120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5252a00640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7166120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5252a00640 1 -- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 msgr2=0x55c2f7171ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5252a00640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f7171ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5252a00640 1 -- 172.21.15.42:0/3553641560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f6dd1210 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5252a00640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 0x55c2f7165be0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f52440096c0 tx=0x7f524402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3041e1d3299af4ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.809+0000 7f5237e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f5237e00640 1 -- 172.21.15.42:0/3553641560 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524404b070 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f5237e00640 1 -- 172.21.15.42:0/3553641560 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f5244003800 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524c004d20 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f5237e00640 1 -- 172.21.15.42:0/3553641560 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5244057050 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f52605ac740 10 client.16773 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c2f6e9d620 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.810+0000 7f5237e00640 1 -- 172.21.15.42:0/3553641560 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f524c004d20 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f5237e00640 1 -- 172.21.15.42:0/3553641560 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5244047730 con 0x55c2f6d902f0 2024-05-27T10:27:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f52605ac740 10 client.16773 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f52605ac740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f52605ac740 20 client.16773 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f52605ac740 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d77d80 0x55c2f7177d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f52605ac740 4 client.16773 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f70ceae0 con 0x55c2f6d77d80 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.811+0000 7f524be00640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d77d80 0x55c2f7177d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.812+0000 7f524be00640 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d77d80 0x55c2f7177d60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1deca0133c50a07c server_cookie=cf9ecad745b64e40 in_seq=0 out_seq=0 2024-05-27T10:27:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.812+0000 7f5237e00640 10 client.16773 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f5237e00640 1 -- 172.21.15.42:0/3553641560 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c2f70ceae0 con 0x55c2f6d77d80 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f5237e00640 10 client.16773 handle_command_reply: tid=0 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 client.16773 shutdown 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 msgr2=0x55c2f7165be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 0x55c2f7165be0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f52440096c0 tx=0x7f524402a410 comp rx=0 tx=0).stop 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 shutdown_connections 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f7171ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d77d80 0x55c2f7177d60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7166120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 --2- 172.21.15.42:0/3553641560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d902f0 0x55c2f7165be0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 >> 172.21.15.42:0/3553641560 conn(0x55c2f6dad300 msgr2=0x55c2f715b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f5237e00640 0 client.16773 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 shutdown_connections 2024-05-27T10:27:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 1 -- 172.21.15.42:0/3553641560 wait complete. 2024-05-27T10:27:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.815+0000 7f52605ac740 20 client.16773 trim_cache size 0 max 16384 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 Processor -- start 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 -- start start 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d902f0 0x55c2f71613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7171c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f7172170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52541b87f0 con 0x55c2f6db91b0 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52541b6060 con 0x55c2f6d902f0 2024-05-27T10:27:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.834+0000 7f52605ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f52541b3ed0 con 0x55c2f6d984d0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f7172170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5252a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d902f0 0x55c2f71613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f7172170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56688/0 (socket says 172.21.15.42:56688) 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 -- 172.21.15.42:0/2469596738 learned_addr learned my addr 172.21.15.42:0/2469596738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f524be00640 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7171c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 -- 172.21.15.42:0/2469596738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 msgr2=0x55c2f7171c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7171c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 -- 172.21.15.42:0/2469596738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d902f0 msgr2=0x55c2f71613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d902f0 0x55c2f71613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.835+0000 7f5253400640 1 -- 172.21.15.42:0/2469596738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f715c2c0 con 0x55c2f6db91b0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.836+0000 7f5253400640 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f7172170 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55c2f7158bd0 tx=0x7f524c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50afed995971943c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.836+0000 7f5242a00640 1 -- 172.21.15.42:0/2469596738 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c029070 con 0x55c2f6db91b0 2024-05-27T10:27:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.836+0000 7f5242a00640 1 -- 172.21.15.42:0/2469596738 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f524c012ba0 con 0x55c2f6db91b0 2024-05-27T10:27:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.836+0000 7f5242a00640 1 -- 172.21.15.42:0/2469596738 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c016040 con 0x55c2f6db91b0 2024-05-27T10:27:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 -- 172.21.15.42:0/2469596738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 msgr2=0x55c2f7172170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f7172170 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55c2f7158bd0 tx=0x7f524c010f10 comp rx=0 tx=0).stop 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 -- 172.21.15.42:0/2469596738 shutdown_connections 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6db91b0 0x55c2f7172170 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d984d0 0x55c2f7171c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 --2- 172.21.15.42:0/2469596738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6d902f0 0x55c2f71613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 -- 172.21.15.42:0/2469596738 >> 172.21.15.42:0/2469596738 conn(0x55c2f6dad300 msgr2=0x55c2f7165ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 -- 172.21.15.42:0/2469596738 shutdown_connections 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.837+0000 7f52605ac740 1 -- 172.21.15.42:0/2469596738 wait complete. 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.838+0000 7f52605ac740 1 -- start start 2024-05-27T10:27:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.838+0000 7f52605ac740 1 Processor -- start 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f716b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 0x55c2f716b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f71de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52541b4520 con 0x55c2f6d984d0 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52541b5990 con 0x55c2f6db91b0 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f52541b4bf0 con 0x55c2f6d902f0 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f52605ac740 10 client.0 _ll_register_callbacks cb 0x7f524403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f524be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 0x55c2f716b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f524be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 0x55c2f716b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56704/0 (socket says 172.21.15.42:56704) 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f524be00640 1 -- 172.21.15.42:0/1926279976 learned_addr learned my addr 172.21.15.42:0/1926279976 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f5252a00640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f716b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.839+0000 7f5253400640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f71de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f524be00640 1 -- 172.21.15.42:0/1926279976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 msgr2=0x55c2f716b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f524be00640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f716b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f524be00640 1 -- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 msgr2=0x55c2f71de430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f524be00640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f71de430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f524be00640 1 -- 172.21.15.42:0/1926279976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f71de970 con 0x55c2f6d984d0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f524be00640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 0x55c2f716b760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f523c0e4560 tx=0x7f523c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1670db863f9d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f5237e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f5237e00640 1 -- 172.21.15.42:0/1926279976 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c020040 con 0x55c2f6d984d0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f5237e00640 1 -- 172.21.15.42:0/1926279976 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f523c0dfa60 con 0x55c2f6d984d0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.840+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524c012ba0 con 0x55c2f6d984d0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f5237e00640 1 -- 172.21.15.42:0/1926279976 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f523c0dfc60 con 0x55c2f6d984d0 2024-05-27T10:27:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 10 client.16779 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c2f71dfab0 con 0x55c2f6d984d0 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f5237e00640 1 -- 172.21.15.42:0/1926279976 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f524c012ba0 con 0x55c2f6d984d0 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f5237e00640 1 -- 172.21.15.42:0/1926279976 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f523c0a4ce0 con 0x55c2f6d984d0 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 10 client.16779 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 20 client.16779 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d4d1c0 0x55c2f71e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 4 client.16779 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:27:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.841+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2f70c82c0 con 0x55c2f6d4d1c0 2024-05-27T10:27:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.842+0000 7f5252a00640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d4d1c0 0x55c2f71e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.843+0000 7f5252a00640 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d4d1c0 0x55c2f71e21e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3edb326e5662826 server_cookie=be66a06c8dc4f701 in_seq=0 out_seq=0 2024-05-27T10:27:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.843+0000 7f5237e00640 10 client.16779 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f5237e00640 1 -- 172.21.15.42:0/1926279976 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c2f70c82c0 con 0x55c2f6d4d1c0 2024-05-27T10:27:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f5237e00640 10 client.16779 handle_command_reply: tid=0 2024-05-27T10:27:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 client.16779 shutdown 2024-05-27T10:27:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 msgr2=0x55c2f716b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 0x55c2f716b760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f523c0e4560 tx=0x7f523c009e60 comp rx=0 tx=0).stop 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 shutdown_connections 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2f6db91b0 0x55c2f71de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2f6d984d0 0x55c2f716b760 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55c2f6d4d1c0 0x55c2f71e21e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.877+0000 7f52605ac740 1 --2- 172.21.15.42:0/1926279976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f6d902f0 0x55c2f716b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.878+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 >> 172.21.15.42:0/1926279976 conn(0x55c2f6dad300 msgr2=0x55c2f7167290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.878+0000 7f5237e00640 0 client.16779 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.878+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 shutdown_connections 2024-05-27T10:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.878+0000 7f52605ac740 1 -- 172.21.15.42:0/1926279976 wait complete. 2024-05-27T10:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.878+0000 7f52605ac740 20 client.16779 trim_cache size 0 max 16384 2024-05-27T10:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52380b2950 msgr2=0x7f52380b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52380b2950 0x7f52380b4e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5244006010 tx=0x7f52440058b0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 msgr2=0x7f52541a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 0x7f52541a6db0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f523c00d740 tx=0x7f523c00dc00 comp rx=0 tx=0).stop 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 shutdown_connections 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52380b2950 0x7f52380b4e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f525410f390 0x7f52541a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525410e8d0 0x7f52541a6db0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 --2- 172.21.15.42:0/3505557008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f525410a340 0x7f52541a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 >> 172.21.15.42:0/3505557008 conn(0x7f5254076cd0 msgr2=0x7f5254107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 shutdown_connections 2024-05-27T10:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:33.879+0000 7f5253e00640 1 -- 172.21.15.42:0/3505557008 wait complete. 2024-05-27T10:27:35.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:34 smithi104 ceph-mon[34812]: pgmap v272: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 914 KiB/s wr, 81 op/s 2024-05-27T10:27:35.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:34 smithi143 ceph-mon[35545]: pgmap v272: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 914 KiB/s wr, 81 op/s 2024-05-27T10:27:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:34 smithi042 ceph-mon[31661]: pgmap v272: 97 pgs: 97 active+clean; 376 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 914 KiB/s wr, 81 op/s 2024-05-27T10:27:37.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:36 smithi104 ceph-mon[34812]: pgmap v273: 97 pgs: 97 active+clean; 378 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-05-27T10:27:37.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:36 smithi143 ceph-mon[35545]: pgmap v273: 97 pgs: 97 active+clean; 378 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-05-27T10:27:37.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:36 smithi042 ceph-mon[31661]: pgmap v273: 97 pgs: 97 active+clean; 378 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-05-27T10:27:39.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:38 smithi104 ceph-mon[34812]: pgmap v274: 97 pgs: 97 active+clean; 378 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 253 KiB/s wr, 23 op/s 2024-05-27T10:27:39.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:38 smithi143 ceph-mon[35545]: pgmap v274: 97 pgs: 97 active+clean; 378 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 253 KiB/s wr, 23 op/s 2024-05-27T10:27:39.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:38 smithi042 ceph-mon[31661]: pgmap v274: 97 pgs: 97 active+clean; 378 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 253 KiB/s wr, 23 op/s 2024-05-27T10:27:41.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:40 smithi104 ceph-mon[34812]: pgmap v275: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 496 KiB/s wr, 77 op/s 2024-05-27T10:27:41.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:40 smithi143 ceph-mon[35545]: pgmap v275: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 496 KiB/s wr, 77 op/s 2024-05-27T10:27:41.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:40 smithi042 ceph-mon[31661]: pgmap v275: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 496 KiB/s wr, 77 op/s 2024-05-27T10:27:43.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:42 smithi104 ceph-mon[34812]: pgmap v276: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 384 KiB/s wr, 65 op/s 2024-05-27T10:27:43.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:42 smithi143 ceph-mon[35545]: pgmap v276: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 384 KiB/s wr, 65 op/s 2024-05-27T10:27:43.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:42 smithi042 ceph-mon[31661]: pgmap v276: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 384 KiB/s wr, 65 op/s 2024-05-27T10:27:45.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:44 smithi143 ceph-mon[35545]: pgmap v277: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 384 KiB/s wr, 65 op/s 2024-05-27T10:27:45.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:44 smithi042 ceph-mon[31661]: pgmap v277: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 384 KiB/s wr, 65 op/s 2024-05-27T10:27:45.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:44 smithi104 ceph-mon[34812]: pgmap v277: 97 pgs: 97 active+clean; 376 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 384 KiB/s wr, 65 op/s 2024-05-27T10:27:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 -- 172.21.15.42:0/2460014765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6958110110 msgr2=0x7f6958109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 --2- 172.21.15.42:0/2460014765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6958110110 0x7f6958109ca0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f694c009900 tx=0x7f694c02a480 comp rx=0 tx=0).stop 2024-05-27T10:27:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 -- 172.21.15.42:0/2460014765 shutdown_connections 2024-05-27T10:27:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 --2- 172.21.15.42:0/2460014765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695810a890 0x7f6958116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 --2- 172.21.15.42:0/2460014765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6958110520 0x7f695810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 --2- 172.21.15.42:0/2460014765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6958110110 0x7f6958109ca0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.112+0000 7f695ec00640 1 -- 172.21.15.42:0/2460014765 >> 172.21.15.42:0/2460014765 conn(0x7f6958076070 msgr2=0x7f6958078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.113+0000 7f695ec00640 1 -- 172.21.15.42:0/2460014765 shutdown_connections 2024-05-27T10:27:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.113+0000 7f695ec00640 1 -- 172.21.15.42:0/2460014765 wait complete. 2024-05-27T10:27:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.114+0000 7f695ec00640 1 Processor -- start 2024-05-27T10:27:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.114+0000 7f695ec00640 1 -- start start 2024-05-27T10:27:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695ec00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6958110110 0x7f69581a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695ec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6958110520 0x7f69581a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 0x7f69581a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f695811a180 con 0x7f69581a0690 2024-05-27T10:27:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695ec00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f695811a000 con 0x7f6958110110 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695ec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f695811a300 con 0x7f6958110520 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 0x7f69581a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 0x7f69581a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33654/0 (socket says 172.21.15.42:33654) 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.115+0000 7f695e200640 1 -- 172.21.15.42:0/1242817553 learned_addr learned my addr 172.21.15.42:0/1242817553 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695ce00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6958110520 0x7f69581a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695d800640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6958110110 0x7f69581a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695e200640 1 -- 172.21.15.42:0/1242817553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6958110520 msgr2=0x7f69581a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695e200640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6958110520 0x7f69581a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695e200640 1 -- 172.21.15.42:0/1242817553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6958110110 msgr2=0x7f69581a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695e200640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6958110110 0x7f69581a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695e200640 1 -- 172.21.15.42:0/1242817553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581a1180 con 0x7f69581a0690 2024-05-27T10:27:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.116+0000 7f695e200640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 0x7f69581a0aa0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f695000eaa0 tx=0x7f695000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.117+0000 7f6956000640 1 -- 172.21.15.42:0/1242817553 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695000cdb0 con 0x7f69581a0690 2024-05-27T10:27:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.117+0000 7f6956000640 1 -- 172.21.15.42:0/1242817553 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6950005020 con 0x7f69581a0690 2024-05-27T10:27:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.117+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581a1470 con 0x7f69581a0690 2024-05-27T10:27:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.117+0000 7f6956000640 1 -- 172.21.15.42:0/1242817553 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500116d0 con 0x7f69581a0690 2024-05-27T10:27:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.117+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581ad5f0 con 0x7f69581a0690 2024-05-27T10:27:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.120+0000 7f69671e4740 1 Processor -- start 2024-05-27T10:27:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f6956000640 1 -- 172.21.15.42:0/1242817553 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f69500051d0 con 0x7f69581a0690 2024-05-27T10:27:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 -- start start 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037235a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037235fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372364e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0371a6b90 con 0x55a036e6a2f0 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a037196730 con 0x55a036e931b0 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.121+0000 7f69671e4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a03719a770 con 0x55a036e724d0 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037235fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037235fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33670/0 (socket says 172.21.15.42:33670) 2024-05-27T10:27:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 -- 172.21.15.42:0/1418013857 learned_addr learned my addr 172.21.15.42:0/1418013857 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695ce00640 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037235a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 -- 172.21.15.42:0/1418013857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 msgr2=0x55a0372364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 -- 172.21.15.42:0/1418013857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 msgr2=0x55a037235a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037235a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.122+0000 7f695d800640 1 -- 172.21.15.42:0/1418013857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03712f8b0 con 0x55a036e6a2f0 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f6956000640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f69400b29d0 0x7f69400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f695d800640 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037235fa0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f694c000c00 tx=0x7f694c032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d37d4dc64338cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f6956000640 1 -- 172.21.15.42:0/1242817553 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f69500e2b70 con 0x7f69581a0690 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f695ce00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f69400b29d0 0x7f69400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f694aa00640 1 -- 172.21.15.42:0/1418013857 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c03f070 con 0x55a036e6a2f0 2024-05-27T10:27:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f694aa00640 1 -- 172.21.15.42:0/1418013857 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c004030 con 0x55a036e6a2f0 2024-05-27T10:27:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.123+0000 7f694aa00640 1 -- 172.21.15.42:0/1418013857 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c035070 con 0x55a036e6a2f0 2024-05-27T10:27:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 -- 172.21.15.42:0/1418013857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 msgr2=0x55a037235fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037235fa0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f694c000c00 tx=0x7f694c032bc0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f695ce00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f69400b29d0 0x7f69400b4ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6944005d80 tx=0x7f6944005d10 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 -- 172.21.15.42:0/1418013857 shutdown_connections 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037235fa0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 --2- 172.21.15.42:0/1418013857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037235a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.124+0000 7f69671e4740 1 -- 172.21.15.42:0/1418013857 >> 172.21.15.42:0/1418013857 conn(0x55a036e87300 msgr2=0x55a037232210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.125+0000 7f69671e4740 1 -- 172.21.15.42:0/1418013857 shutdown_connections 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.125+0000 7f69671e4740 1 -- 172.21.15.42:0/1418013857 wait complete. 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.125+0000 7f69671e4740 1 -- start start 2024-05-27T10:27:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 Processor -- start 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e6a2f0 0x55a03723fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 0x55a037240170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a036e7c790 con 0x55a036e724d0 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a03713ebc0 con 0x55a036e6a2f0 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a037242b30 con 0x55a036e931b0 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f69671e4740 10 client.0 _ll_register_callbacks cb 0x55a03711c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.126+0000 7f695ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 0x55a037240170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 0x55a037240170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33678/0 (socket says 172.21.15.42:33678) 2024-05-27T10:27:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 -- 172.21.15.42:0/3783597410 learned_addr learned my addr 172.21.15.42:0/3783597410 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695d800640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e6a2f0 0x55a03723fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695e200640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 -- 172.21.15.42:0/3783597410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 msgr2=0x55a03724be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 -- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e6a2f0 msgr2=0x55a03723fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e6a2f0 0x55a03723fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 -- 172.21.15.42:0/3783597410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a036eab210 con 0x55a036e724d0 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695d800640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e6a2f0 0x55a03723fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f695ce00640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 0x55a037240170 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6944012870 tx=0x7f69440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8cd5be3c19c8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.127+0000 7f693f400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:27:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f693f400640 1 -- 172.21.15.42:0/3783597410 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694400fb40 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f693f400640 1 -- 172.21.15.42:0/3783597410 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f694400fce0 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694c004030 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f693f400640 1 -- 172.21.15.42:0/3783597410 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6944005020 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f69671e4740 10 client.16791 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a036f77620 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.128+0000 7f693f400640 1 -- 172.21.15.42:0/3783597410 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c004030 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f693f400640 1 -- 172.21.15.42:0/3783597410 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f694402e020 con 0x55a036e724d0 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f69671e4740 10 client.16791 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f69671e4740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f69671e4740 20 client.16791 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f69671e4740 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e51d80 0x55a037251ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f69671e4740 4 client.16791 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:27:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.129+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0371a8ae0 con 0x55a036e51d80 2024-05-27T10:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.130+0000 7f695d800640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e51d80 0x55a037251ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.131+0000 7f695d800640 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e51d80 0x55a037251ec0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc19ac2602b0532 server_cookie=396f1bf3756c6d19 in_seq=0 out_seq=0 2024-05-27T10:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.131+0000 7f693f400640 10 client.16791 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f693f400640 1 -- 172.21.15.42:0/3783597410 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a0371a8ae0 con 0x55a036e51d80 2024-05-27T10:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f693f400640 10 client.16791 handle_command_reply: tid=0 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 client.16791 shutdown 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 msgr2=0x55a037240170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 0x55a037240170 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6944012870 tx=0x7f69440046c0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 shutdown_connections 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e51d80 0x55a037251ec0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e724d0 0x55a037240170 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 --2- 172.21.15.42:0/3783597410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e6a2f0 0x55a03723fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f693f400640 0 client.16791 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.134+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 >> 172.21.15.42:0/3783597410 conn(0x55a036e87300 msgr2=0x55a037237780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.135+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 shutdown_connections 2024-05-27T10:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.135+0000 7f69671e4740 1 -- 172.21.15.42:0/3783597410 wait complete. 2024-05-27T10:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.135+0000 7f69671e4740 20 client.16791 trim_cache size 0 max 16384 2024-05-27T10:27:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 Processor -- start 2024-05-27T10:27:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 -- start start 2024-05-27T10:27:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037242360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e724d0 0x55a03724bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69581b8850 con 0x55a036e6a2f0 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69581b60c0 con 0x55a036e724d0 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.153+0000 7f69671e4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f69581b3f30 con 0x55a036e931b0 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037242360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037242360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33690/0 (socket says 172.21.15.42:33690) 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 -- 172.21.15.42:0/36339605 learned_addr learned my addr 172.21.15.42:0/36339605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695ce00640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e724d0 0x55a03724bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695e200640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 -- 172.21.15.42:0/36339605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 msgr2=0x55a03724c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 -- 172.21.15.42:0/36339605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e724d0 msgr2=0x55a03724bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e724d0 0x55a03724bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695d800640 1 -- 172.21.15.42:0/36339605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a037232bd0 con 0x55a036e6a2f0 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695e200640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.154+0000 7f695ce00640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e724d0 0x55a03724bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.155+0000 7f695d800640 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037242360 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f694c000c00 tx=0x7f694c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a699d840e246dad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.155+0000 7f694a000640 1 -- 172.21.15.42:0/36339605 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c03f040 con 0x55a036e6a2f0 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.155+0000 7f694a000640 1 -- 172.21.15.42:0/36339605 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c005040 con 0x55a036e6a2f0 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.155+0000 7f694a000640 1 -- 172.21.15.42:0/36339605 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c035040 con 0x55a036e6a2f0 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 -- 172.21.15.42:0/36339605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 msgr2=0x55a037242360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037242360 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f694c000c00 tx=0x7f694c02a480 comp rx=0 tx=0).stop 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 -- 172.21.15.42:0/36339605 shutdown_connections 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e931b0 0x55a03724c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e724d0 0x55a03724bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 --2- 172.21.15.42:0/36339605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037242360 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 -- 172.21.15.42:0/36339605 >> 172.21.15.42:0/36339605 conn(0x55a036e87300 msgr2=0x55a03723fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 -- 172.21.15.42:0/36339605 shutdown_connections 2024-05-27T10:27:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.156+0000 7f69671e4740 1 -- 172.21.15.42:0/36339605 wait complete. 2024-05-27T10:27:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 -- start start 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 Processor -- start 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037248230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037243b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69581b4580 con 0x55a036e6a2f0 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69581b59f0 con 0x55a036e931b0 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.157+0000 7f69671e4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f69581b4c50 con 0x55a036e724d0 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f69671e4740 10 client.0 _ll_register_callbacks cb 0x7f694400c510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037248230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037248230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33694/0 (socket says 172.21.15.42:33694) 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 -- 172.21.15.42:0/882257819 learned_addr learned my addr 172.21.15.42:0/882257819 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695ce00640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695e200640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037243b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 -- 172.21.15.42:0/882257819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 msgr2=0x55a0372435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 -- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 msgr2=0x55a037243b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037243b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 -- 172.21.15.42:0/882257819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a037244050 con 0x55a036e6a2f0 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.158+0000 7f695d800640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037248230 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f694c031990 tx=0x7f694c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a83dcc264171da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f693f400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:27:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f693f400640 1 -- 172.21.15.42:0/882257819 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c05c020 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f693f400640 1 -- 172.21.15.42:0/882257819 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f694c0051f0 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694c005040 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f693f400640 1 -- 172.21.15.42:0/882257819 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f694c067050 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f69671e4740 10 client.16797 fetch_fsmap learned FSMap version 33 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a0372ba1a0 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.159+0000 7f693f400640 1 -- 172.21.15.42:0/882257819 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c005040 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f693f400640 1 -- 172.21.15.42:0/882257819 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f694c04f430 con 0x55a036e6a2f0 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f69671e4740 10 client.16797 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f69671e4740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f69671e4740 20 client.16797 populate_metadata read hostname 'smithi042' 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f69671e4740 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e271c0 0x55a0372bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f69671e4740 4 client.16797 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:27:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0371a22c0 con 0x55a036e271c0 2024-05-27T10:27:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.160+0000 7f695ce00640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e271c0 0x55a0372bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.162+0000 7f695ce00640 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e271c0 0x55a0372bc480 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3f8d2aa39ce4a5 server_cookie=ed24379994ef6441 in_seq=0 out_seq=0 2024-05-27T10:27:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.162+0000 7f693f400640 10 client.16797 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f693f400640 1 -- 172.21.15.42:0/882257819 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a0371a22c0 con 0x55a036e271c0 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f693f400640 10 client.16797 handle_command_reply: tid=0 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 client.16797 shutdown 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 msgr2=0x55a037248230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037248230 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f694c031990 tx=0x7f694c005d50 comp rx=0 tx=0).stop 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 shutdown_connections 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a036e931b0 0x55a037243b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a036e271c0 0x55a0372bc480 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a036e724d0 0x55a0372435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 --2- 172.21.15.42:0/882257819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a036e6a2f0 0x55a037248230 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 >> 172.21.15.42:0/882257819 conn(0x55a036e87300 msgr2=0x55a0372412e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f693f400640 0 client.16797 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:27:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.194+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 shutdown_connections 2024-05-27T10:27:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.195+0000 7f69671e4740 1 -- 172.21.15.42:0/882257819 wait complete. 2024-05-27T10:27:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.195+0000 7f69671e4740 20 client.16797 trim_cache size 0 max 16384 2024-05-27T10:27:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:27:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:27:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:27:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f69400b29d0 msgr2=0x7f69400b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f69400b29d0 0x7f69400b4ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6944005d80 tx=0x7f6944005d10 comp rx=0 tx=0).stop 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 msgr2=0x7f69581a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 0x7f69581a0aa0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f695000eaa0 tx=0x7f695000ef60 comp rx=0 tx=0).stop 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 shutdown_connections 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f69400b29d0 0x7f69400b4ee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581a0690 0x7f69581a0aa0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6958110520 0x7f69581a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 --2- 172.21.15.42:0/1242817553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6958110110 0x7f69581a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 >> 172.21.15.42:0/1242817553 conn(0x7f6958076070 msgr2=0x7f69581158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 shutdown_connections 2024-05-27T10:27:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:27:47.196+0000 7f695ec00640 1 -- 172.21.15.42:0/1242817553 wait complete. 2024-05-27T10:27:47.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:46 smithi143 ceph-mon[35545]: pgmap v278: 97 pgs: 97 active+clean; 376 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 668 KiB/s wr, 123 op/s 2024-05-27T10:27:47.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:46 smithi042 ceph-mon[31661]: pgmap v278: 97 pgs: 97 active+clean; 376 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 668 KiB/s wr, 123 op/s 2024-05-27T10:27:47.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:46 smithi104 ceph-mon[34812]: pgmap v278: 97 pgs: 97 active+clean; 376 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 668 KiB/s wr, 123 op/s 2024-05-27T10:27:49.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:48 smithi143 ceph-mon[35545]: pgmap v279: 97 pgs: 97 active+clean; 376 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 527 KiB/s wr, 111 op/s 2024-05-27T10:27:49.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:48 smithi042 ceph-mon[31661]: pgmap v279: 97 pgs: 97 active+clean; 376 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 527 KiB/s wr, 111 op/s 2024-05-27T10:27:49.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:48 smithi104 ceph-mon[34812]: pgmap v279: 97 pgs: 97 active+clean; 376 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 527 KiB/s wr, 111 op/s 2024-05-27T10:27:51.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:50 smithi143 ceph-mon[35545]: pgmap v280: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 894 KiB/s wr, 176 op/s 2024-05-27T10:27:51.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:50 smithi042 ceph-mon[31661]: pgmap v280: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 894 KiB/s wr, 176 op/s 2024-05-27T10:27:51.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:50 smithi104 ceph-mon[34812]: pgmap v280: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 894 KiB/s wr, 176 op/s 2024-05-27T10:27:53.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:52 smithi143 ceph-mon[35545]: pgmap v281: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 651 KiB/s wr, 122 op/s 2024-05-27T10:27:53.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:52 smithi042 ceph-mon[31661]: pgmap v281: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 651 KiB/s wr, 122 op/s 2024-05-27T10:27:53.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:52 smithi104 ceph-mon[34812]: pgmap v281: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 651 KiB/s wr, 122 op/s 2024-05-27T10:27:55.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:54 smithi143 ceph-mon[35545]: pgmap v282: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 651 KiB/s wr, 122 op/s 2024-05-27T10:27:55.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:54 smithi042 ceph-mon[31661]: pgmap v282: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 651 KiB/s wr, 122 op/s 2024-05-27T10:27:55.419 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-27T10:27:55.420 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.3 2024-05-27T10:27:55.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:27:55.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:54 smithi104 ceph-mon[34812]: pgmap v282: 97 pgs: 97 active+clean; 377 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 651 KiB/s wr, 122 op/s 2024-05-27T10:27:55.911 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:27:56.301 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:55 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:27:56.301 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:55 smithi042 ceph-mon[31661]: pgmap v283: 97 pgs: 97 active+clean; 379 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 885 KiB/s wr, 162 op/s 2024-05-27T10:27:56.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:55 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:27:56.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:55 smithi143 ceph-mon[35545]: pgmap v283: 97 pgs: 97 active+clean; 379 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 885 KiB/s wr, 162 op/s 2024-05-27T10:27:56.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:55 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:27:56.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:55 smithi104 ceph-mon[34812]: pgmap v283: 97 pgs: 97 active+clean; 379 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 885 KiB/s wr, 162 op/s 2024-05-27T10:27:58.329 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:58 smithi042 ceph-mon[31661]: pgmap v284: 97 pgs: 97 active+clean; 379 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 601 KiB/s wr, 105 op/s 2024-05-27T10:27:58.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:58 smithi104 ceph-mon[34812]: pgmap v284: 97 pgs: 97 active+clean; 379 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 601 KiB/s wr, 105 op/s 2024-05-27T10:27:58.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:58 smithi143 ceph-mon[35545]: pgmap v284: 97 pgs: 97 active+clean; 379 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 601 KiB/s wr, 105 op/s 2024-05-27T10:27:58.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 -- 172.21.15.42:0/2364979172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10b890 msgr2=0x7fb5dc1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:58.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/2364979172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10b890 0x7fb5dc1131f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400b0d0 tx=0x7fb5d400b3e0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 -- 172.21.15.42:0/2364979172 shutdown_connections 2024-05-27T10:27:58.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/2364979172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10b890 0x7fb5dc1131f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/2364979172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5dc106610 0x7fb5dc106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/2364979172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5dc10af40 0x7fb5dc10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 -- 172.21.15.42:0/2364979172 >> 172.21.15.42:0/2364979172 conn(0x7fb5dc102020 msgr2=0x7fb5dc104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:58.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 -- 172.21.15.42:0/2364979172 shutdown_connections 2024-05-27T10:27:58.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.589+0000 7fb5dbe00700 1 -- 172.21.15.42:0/2364979172 wait complete. 2024-05-27T10:27:58.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.591+0000 7fb5dbe00700 1 Processor -- start 2024-05-27T10:27:58.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.591+0000 7fb5dbe00700 1 -- start start 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5dbe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5dc106610 0x7fb5dc1105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5dbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 0x7fb5dc110b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5dbe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5dc10b890 0x7fb5dc111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5dbe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc116370 con 0x7fb5dc10af40 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5dbe00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1161f0 con 0x7fb5dc106610 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5dbe00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1164f0 con 0x7fb5dc10b890 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5da000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 0x7fb5dc110b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:58.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5da000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 0x7fb5dc110b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49996/0 (socket says 172.21.15.42:49996) 2024-05-27T10:27:58.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.592+0000 7fb5da000700 1 -- 172.21.15.42:0/4245117630 learned_addr learned my addr 172.21.15.42:0/4245117630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:27:58.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5daa00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5dc106610 0x7fb5dc1105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:58.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5db400700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5dc10b890 0x7fb5dc111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:58.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5da000700 1 -- 172.21.15.42:0/4245117630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5dc10b890 msgr2=0x7fb5dc111070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5da000700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5dc10b890 0x7fb5dc111070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5da000700 1 -- 172.21.15.42:0/4245117630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5dc106610 msgr2=0x7fb5dc1105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5da000700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5dc106610 0x7fb5dc1105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.593+0000 7fb5da000700 1 -- 172.21.15.42:0/4245117630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc1ad240 con 0x7fb5dc10af40 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.594+0000 7fb5da000700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 0x7fb5dc110b30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800d980 tx=0x7fb5c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.594+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8017070 con 0x7fb5dc10af40 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.594+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5c800fd20 con 0x7fb5dc10af40 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.594+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc1ad530 con 0x7fb5dc10af40 2024-05-27T10:27:58.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.594+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c800e3b0 con 0x7fb5dc10af40 2024-05-27T10:27:58.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.594+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc1adac0 con 0x7fb5dc10af40 2024-05-27T10:27:58.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.596+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc079370 con 0x7fb5dc10af40 2024-05-27T10:27:58.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.599+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb5c800f890 con 0x7fb5dc10af40 2024-05-27T10:27:58.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.601+0000 7fb5d3400700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5c00b2a70 0x7fb5c00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:27:58.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.601+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7fb5c80e1fd0 con 0x7fb5dc10af40 2024-05-27T10:27:58.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.601+0000 7fb5daa00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5c00b2a70 0x7fb5c00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:27:58.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.602+0000 7fb5daa00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5c00b2a70 0x7fb5c00b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc005950 tx=0x7fb5cc00b410 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:27:58.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.606+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5c80a9970 con 0x7fb5dc10af40 2024-05-27T10:27:58.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.929+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5dc111b20 con 0x7fb5dc10af40 2024-05-27T10:27:58.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.931+0000 7fb5d3400700 1 -- 172.21.15.42:0/4245117630 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11882 (secure 0 0 0) 0x7fb5c80a9270 con 0x7fb5dc10af40 2024-05-27T10:27:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:27:58.931 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:27:54.910348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34421":{"gid":34421,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/3150585498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3150585498},{"type":"v1","addr":"172.21.15.104:6833","nonce":3150585498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5c00b2a70 msgr2=0x7fb5c00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5c00b2a70 0x7fb5c00b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc005950 tx=0x7fb5cc00b410 comp rx=0 tx=0).stop 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 msgr2=0x7fb5dc110b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 0x7fb5dc110b30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800d980 tx=0x7fb5c800dc90 comp rx=0 tx=0).stop 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 shutdown_connections 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5c00b2a70 0x7fb5c00b4f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5dc10b890 0x7fb5dc111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5dc106610 0x7fb5dc1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 --2- 172.21.15.42:0/4245117630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc10af40 0x7fb5dc110b30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:27:58.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.934+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 >> 172.21.15.42:0/4245117630 conn(0x7fb5dc102020 msgr2=0x7fb5dc10a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:27:58.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.935+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 shutdown_connections 2024-05-27T10:27:58.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:27:58.935+0000 7fb5dbe00700 1 -- 172.21.15.42:0/4245117630 wait complete. 2024-05-27T10:27:58.935 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 34 2024-05-27T10:27:59.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:27:59 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4245117630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:27:59.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:27:59 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4245117630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:27:59.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:27:59 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4245117630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:27:59.887 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 142.3 --await-for 45.0' 2024-05-27T10:27:59.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 142.3 --await-for 45.0 2024-05-27T10:28:00.350 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.443+0000 7f6765600640 1 -- 172.21.15.42:0/3871141837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 msgr2=0x7f676010fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.443+0000 7f6765600640 1 --2- 172.21.15.42:0/3871141837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f676010fe20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c02a410 comp rx=0 tx=0).stop 2024-05-27T10:28:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 -- 172.21.15.42:0/3871141837 shutdown_connections 2024-05-27T10:28:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 --2- 172.21.15.42:0/3871141837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6760110360 0x7f6760112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 --2- 172.21.15.42:0/3871141837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f676010fe20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 --2- 172.21.15.42:0/3871141837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67601027c0 0x7f6760102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 -- 172.21.15.42:0/3871141837 >> 172.21.15.42:0/3871141837 conn(0x7f67600fe210 msgr2=0x7f6760100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 -- 172.21.15.42:0/3871141837 shutdown_connections 2024-05-27T10:28:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.444+0000 7f6765600640 1 -- 172.21.15.42:0/3871141837 wait complete. 2024-05-27T10:28:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.445+0000 7f6765600640 1 Processor -- start 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 -- start start 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67601027c0 0x7f67601a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f67601a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6760110360 0x7f676019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6760115d70 con 0x7f676010da00 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6760115bf0 con 0x7f6760110360 2024-05-27T10:28:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.446+0000 7f6765600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6760115ef0 con 0x7f67601027c0 2024-05-27T10:28:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f67601a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f67601a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50006/0 (socket says 172.21.15.42:50006) 2024-05-27T10:28:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 -- 172.21.15.42:0/2143383968 learned_addr learned my addr 172.21.15.42:0/2143383968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675fe00640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67601027c0 0x7f67601a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f6764c00640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6760110360 0x7f676019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:00 smithi104 ceph-mon[34812]: pgmap v285: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 939 KiB/s wr, 155 op/s 2024-05-27T10:28:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 -- 172.21.15.42:0/2143383968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67601027c0 msgr2=0x7f67601a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67601027c0 0x7f67601a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 -- 172.21.15.42:0/2143383968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6760110360 msgr2=0x7f676019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6760110360 0x7f676019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.447+0000 7f675f400640 1 -- 172.21.15.42:0/2143383968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676019cda0 con 0x7f676010da00 2024-05-27T10:28:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.448+0000 7f675f400640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f67601a2a30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f674c02a980 tx=0x7f674c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.449+0000 7f675cc00640 1 -- 172.21.15.42:0/2143383968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c03f070 con 0x7f676010da00 2024-05-27T10:28:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.449+0000 7f675cc00640 1 -- 172.21.15.42:0/2143383968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f674c03bd00 con 0x7f676010da00 2024-05-27T10:28:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.449+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676019cfd0 con 0x7f676010da00 2024-05-27T10:28:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.449+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67601a9290 con 0x7f676010da00 2024-05-27T10:28:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.449+0000 7f675cc00640 1 -- 172.21.15.42:0/2143383968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035030 con 0x7f676010da00 2024-05-27T10:28:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.453+0000 7f676dbbe740 1 Processor -- start 2024-05-27T10:28:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.454+0000 7f676dbbe740 1 -- start start 2024-05-27T10:28:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.454+0000 7f675cc00640 1 -- 172.21.15.42:0/2143383968 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f674c042070 con 0x7f676010da00 2024-05-27T10:28:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.454+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.454+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f676dbbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56346033eb90 con 0x56346002b1b0 2024-05-27T10:28:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f676dbbe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56346032e730 con 0x56346000a4d0 2024-05-27T10:28:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f676dbbe740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563460332770 con 0x5634600022f0 2024-05-27T10:28:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50008/0 (socket says 172.21.15.42:50008) 2024-05-27T10:28:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 -- 172.21.15.42:0/1327117080 learned_addr learned my addr 172.21.15.42:0/1327117080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f675fe00640 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 -- 172.21.15.42:0/1327117080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 msgr2=0x5634603cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 -- 172.21.15.42:0/1327117080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 msgr2=0x5634603ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f6764c00640 1 -- 172.21.15.42:0/1327117080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634602c78b0 con 0x56346002b1b0 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.455+0000 7f675fe00640 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.456+0000 7f6764c00640 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603cda60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6758009f20 tx=0x7f675800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221f9e150b09175f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.456+0000 7f675cc00640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f67440b2aa0 0x7f67440b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.456+0000 7f6755200640 1 -- 172.21.15.42:0/1327117080 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758015070 con 0x56346002b1b0 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.456+0000 7f675cc00640 1 -- 172.21.15.42:0/2143383968 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f674c102f80 con 0x7f676010da00 2024-05-27T10:28:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.456+0000 7f6755200640 1 -- 172.21.15.42:0/1327117080 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f675801aaf0 con 0x56346002b1b0 2024-05-27T10:28:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.456+0000 7f6755200640 1 -- 172.21.15.42:0/1327117080 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758011b10 con 0x56346002b1b0 2024-05-27T10:28:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.457+0000 7f675fe00640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f67440b2aa0 0x7f67440b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.458+0000 7f675fe00640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f67440b2aa0 0x7f67440b4fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5634603cee10 tx=0x7f6750005f50 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.458+0000 7f676dbbe740 1 -- 172.21.15.42:0/1327117080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 msgr2=0x5634603cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.458+0000 7f676dbbe740 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603cda60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6758009f20 tx=0x7f675800f7f0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.458+0000 7f676dbbe740 1 -- 172.21.15.42:0/1327117080 shutdown_connections 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 --2- 172.21.15.42:0/1327117080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603cda60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 -- 172.21.15.42:0/1327117080 >> 172.21.15.42:0/1327117080 conn(0x56346001f300 msgr2=0x5634603ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 -- 172.21.15.42:0/1327117080 shutdown_connections 2024-05-27T10:28:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 -- 172.21.15.42:0/1327117080 wait complete. 2024-05-27T10:28:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.459+0000 7f676dbbe740 1 -- start start 2024-05-27T10:28:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.460+0000 7f676dbbe740 1 Processor -- start 2024-05-27T10:28:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.461+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634600022f0 0x5634603d7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.461+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 0x5634603d8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.461+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346002b1b0 0x5634603e3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.461+0000 7f676dbbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563460014790 con 0x56346000a4d0 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f676dbbe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5634602d6bc0 con 0x5634600022f0 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 0x5634603d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634600022f0 0x5634603d7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 0x5634603d8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50018/0 (socket says 172.21.15.42:50018) 2024-05-27T10:28:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 -- 172.21.15.42:0/575264636 learned_addr learned my addr 172.21.15.42:0/575264636 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634603dab90 con 0x56346002b1b0 2024-05-27T10:28:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f676dbbe740 10 client.0 _ll_register_callbacks cb 0x5634602b4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f6764c00640 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346002b1b0 0x5634603e3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 -- 172.21.15.42:0/575264636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346002b1b0 msgr2=0x5634603e3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346002b1b0 0x5634603e3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 -- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634600022f0 msgr2=0x5634603d7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634600022f0 0x5634603d7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.462+0000 7f675f400640 1 -- 172.21.15.42:0/575264636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563460043210 con 0x56346000a4d0 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.463+0000 7f675f400640 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 0x5634603d8440 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f674c0c9f20 tx=0x7f674c0c9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93c7fc380e1cacd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.463+0000 7f6749600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.463+0000 7f6749600640 1 -- 172.21.15.42:0/575264636 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c049070 con 0x56346000a4d0 2024-05-27T10:28:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.463+0000 7f6749600640 1 -- 172.21.15.42:0/575264636 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f674c0ca690 con 0x56346000a4d0 2024-05-27T10:28:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.463+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675801aaf0 con 0x56346000a4d0 2024-05-27T10:28:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.464+0000 7f6749600640 1 -- 172.21.15.42:0/575264636 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f674c0c9d30 con 0x56346000a4d0 2024-05-27T10:28:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.464+0000 7f676dbbe740 10 client.16815 fetch_fsmap learned FSMap version 34 2024-05-27T10:28:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.464+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56346010f620 con 0x56346000a4d0 2024-05-27T10:28:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.464+0000 7f6749600640 1 -- 172.21.15.42:0/575264636 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f675801aaf0 con 0x56346000a4d0 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.465+0000 7f6749600640 1 -- 172.21.15.42:0/575264636 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f674c0d2c80 con 0x56346000a4d0 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.465+0000 7f676dbbe740 10 client.16815 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.466+0000 7f676dbbe740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.466+0000 7f676dbbe740 20 client.16815 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.466+0000 7f676dbbe740 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffe9d80 0x5634603e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.466+0000 7f676dbbe740 4 client.16815 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.466+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563460340ae0 con 0x56345ffe9d80 2024-05-27T10:28:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.467+0000 7f675fe00640 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffe9d80 0x5634603e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.468+0000 7f675fe00640 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffe9d80 0x5634603e9ef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d3cb68dc535398 server_cookie=2eea6d8085cd1c11 in_seq=0 out_seq=0 2024-05-27T10:28:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.468+0000 7f6749600640 10 client.16815 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.469+0000 7f6749600640 1 -- 172.21.15.42:0/575264636 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563460340ae0 con 0x56345ffe9d80 2024-05-27T10:28:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.470+0000 7f6749600640 10 client.16815 handle_command_reply: tid=0 2024-05-27T10:28:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.470+0000 7f676dbbe740 1 client.16815 shutdown 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.470+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 msgr2=0x5634603d8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.470+0000 7f676dbbe740 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 0x5634603d8440 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f674c0c9f20 tx=0x7f674c0c9b70 comp rx=0 tx=0).stop 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.470+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 shutdown_connections 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffe9d80 0x5634603e9ef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346002b1b0 0x5634603e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346000a4d0 0x5634603d8440 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 --2- 172.21.15.42:0/575264636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5634600022f0 0x5634603d7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 >> 172.21.15.42:0/575264636 conn(0x56346001f300 msgr2=0x5634603cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f6749600640 0 client.16815 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 shutdown_connections 2024-05-27T10:28:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 1 -- 172.21.15.42:0/575264636 wait complete. 2024-05-27T10:28:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.471+0000 7f676dbbe740 20 client.16815 trim_cache size 0 max 16384 2024-05-27T10:28:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.492+0000 7f676dbbe740 1 Processor -- start 2024-05-27T10:28:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.492+0000 7f676dbbe740 1 -- start start 2024-05-27T10:28:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f676dbbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b42a0 con 0x56346002b1b0 2024-05-27T10:28:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f676dbbe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b1b10 con 0x56346000a4d0 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f676dbbe740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f67601af980 con 0x5634600022f0 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f6764c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f6764c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603e41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50028/0 (socket says 172.21.15.42:50028) 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.493+0000 7f6764c00640 1 -- 172.21.15.42:0/4244851532 learned_addr learned my addr 172.21.15.42:0/4244851532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f675fe00640 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f675f400640 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f6764c00640 1 -- 172.21.15.42:0/4244851532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 msgr2=0x5634603d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f6764c00640 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f6764c00640 1 -- 172.21.15.42:0/4244851532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 msgr2=0x5634603e3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f6764c00640 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603e3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f6764c00640 1 -- 172.21.15.42:0/4244851532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634603ea260 con 0x56346002b1b0 2024-05-27T10:28:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.494+0000 7f6764c00640 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603e41a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f675800fcf0 tx=0x7f6758009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94efb8dfb912c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.495+0000 7f674be00640 1 -- 172.21.15.42:0/4244851532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675802a070 con 0x56346002b1b0 2024-05-27T10:28:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.495+0000 7f674be00640 1 -- 172.21.15.42:0/4244851532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6758011520 con 0x56346002b1b0 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.495+0000 7f674be00640 1 -- 172.21.15.42:0/4244851532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758015040 con 0x56346002b1b0 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 -- 172.21.15.42:0/4244851532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 msgr2=0x5634603e41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603e41a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f675800fcf0 tx=0x7f6758009ca0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 -- 172.21.15.42:0/4244851532 shutdown_connections 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56346002b1b0 0x5634603e41a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346000a4d0 0x5634603e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 --2- 172.21.15.42:0/4244851532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634600022f0 0x5634603d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 -- 172.21.15.42:0/4244851532 >> 172.21.15.42:0/4244851532 conn(0x56346001f300 msgr2=0x5634603ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 -- 172.21.15.42:0/4244851532 shutdown_connections 2024-05-27T10:28:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.496+0000 7f676dbbe740 1 -- 172.21.15.42:0/4244851532 wait complete. 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.497+0000 7f676dbbe740 1 -- start start 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.497+0000 7f676dbbe740 1 Processor -- start 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.497+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 0x5634603dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346000a4d0 0x5634603dc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f676dbbe740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346002b1b0 0x5634603dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f676dbbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67601affd0 con 0x5634600022f0 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f676dbbe740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b1440 con 0x56346002b1b0 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f676dbbe740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b06a0 con 0x56346000a4d0 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f676dbbe740 10 client.0 _ll_register_callbacks cb 0x7f674c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 0x5634603dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 0x5634603dbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50032/0 (socket says 172.21.15.42:50032) 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 -- 172.21.15.42:0/867966585 learned_addr learned my addr 172.21.15.42:0/867966585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675f400640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346000a4d0 0x5634603dc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f6764c00640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346002b1b0 0x5634603dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 -- 172.21.15.42:0/867966585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346000a4d0 msgr2=0x5634603dc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346000a4d0 0x5634603dc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 -- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346002b1b0 msgr2=0x5634603dc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346002b1b0 0x5634603dc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.498+0000 7f675fe00640 1 -- 172.21.15.42:0/867966585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634604515a0 con 0x5634600022f0 2024-05-27T10:28:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.499+0000 7f675fe00640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 0x5634603dbcd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5634603ea620 tx=0x7f67500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ad6a827fa23e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.499+0000 7f6749600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.499+0000 7f6749600640 1 -- 172.21.15.42:0/867966585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750029530 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.499+0000 7f6749600640 1 -- 172.21.15.42:0/867966585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f67500296d0 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.499+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6758011520 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f6749600640 1 -- 172.21.15.42:0/867966585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6750039050 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 10 client.16821 fetch_fsmap learned FSMap version 34 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563460452870 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f6749600640 1 -- 172.21.15.42:0/867966585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6758011520 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f6749600640 1 -- 172.21.15.42:0/867966585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f675000f040 con 0x5634600022f0 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 10 client.16821 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 20 client.16821 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffbf1c0 0x5634604550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 4 client.16821 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:28:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.500+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56346033a2c0 con 0x56345ffbf1c0 2024-05-27T10:28:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.501+0000 7f675f400640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffbf1c0 0x5634604550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.502+0000 7f675f400640 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffbf1c0 0x5634604550b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f256242a2191f28 server_cookie=c14f79638c1a580a in_seq=0 out_seq=0 2024-05-27T10:28:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.502+0000 7f6749600640 10 client.16821 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:00.508 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:00 smithi042 ceph-mon[31661]: pgmap v285: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 939 KiB/s wr, 155 op/s 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.554+0000 7f6749600640 1 -- 172.21.15.42:0/867966585 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56346033a2c0 con 0x56345ffbf1c0 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.554+0000 7f6749600640 10 client.16821 handle_command_reply: tid=0 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.554+0000 7f676dbbe740 1 client.16821 shutdown 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.554+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 msgr2=0x5634603dbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.554+0000 7f676dbbe740 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 0x5634603dbcd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5634603ea620 tx=0x7f67500046c0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 shutdown_connections 2024-05-27T10:28:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56346002b1b0 0x5634603dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56345ffbf1c0 0x5634604550b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56346000a4d0 0x5634603dc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 --2- 172.21.15.42:0/867966585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5634600022f0 0x5634603dbcd0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 >> 172.21.15.42:0/867966585 conn(0x56346001f300 msgr2=0x5634603c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f6749600640 0 client.16821 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 shutdown_connections 2024-05-27T10:28:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 1 -- 172.21.15.42:0/867966585 wait complete. 2024-05-27T10:28:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.555+0000 7f676dbbe740 20 client.16821 trim_cache size 0 max 16384 2024-05-27T10:28:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:28:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:28:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:28:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.556+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f67440b2aa0 msgr2=0x7f67440b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.556+0000 7f6765600640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f67440b2aa0 0x7f67440b4fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5634603cee10 tx=0x7f6750005f50 comp rx=0 tx=0).stop 2024-05-27T10:28:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.556+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 msgr2=0x7f67601a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.556+0000 7f6765600640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f67601a2a30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f674c02a980 tx=0x7f674c003cd0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 shutdown_connections 2024-05-27T10:28:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f67440b2aa0 0x7f67440b4fb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6760110360 0x7f676019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676010da00 0x7f67601a2a30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 --2- 172.21.15.42:0/2143383968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67601027c0 0x7f67601a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 >> 172.21.15.42:0/2143383968 conn(0x7f67600fe210 msgr2=0x7f6760100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.557+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 shutdown_connections 2024-05-27T10:28:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:00.558+0000 7f6765600640 1 -- 172.21.15.42:0/2143383968 wait complete. 2024-05-27T10:28:00.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:00 smithi143 ceph-mon[35545]: pgmap v285: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 939 KiB/s wr, 155 op/s 2024-05-27T10:28:01.948 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: *** Caught signal (Segmentation fault) ** 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: in thread 7f3c68c00700 thread_name:md_log_replay 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev) 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3c7d25bd20] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x561fd7dc9507] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x561fd7dc999e] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x561fd7dc4015] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x561fd7a83a27] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 6: (MDLog::_replay_thread()+0x1f38) [0x561fd7e2a06c] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561fd7e335ab] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 8: (Thread::entry_wrapper()+0x43) [0x7f3c7e6ad72d] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 9: (Thread::_entry_func(void*)+0xd) [0x7f3c7e6ad749] 2024-05-27T10:28:01.949 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f3c7d2511ca] 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 11: clone() 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.569+0000 7f3c68c00700 -1 *** Caught signal (Segmentation fault) ** 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: in thread 7f3c68c00700 thread_name:md_log_replay 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev) 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3c7d25bd20] 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x561fd7dc9507] 2024-05-27T10:28:01.950 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x561fd7dc999e] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x561fd7dc4015] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x561fd7a83a27] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 6: (MDLog::_replay_thread()+0x1f38) [0x561fd7e2a06c] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561fd7e335ab] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 8: (Thread::entry_wrapper()+0x43) [0x7f3c7e6ad72d] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 9: (Thread::_entry_func(void*)+0xd) [0x7f3c7e6ad749] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f3c7d2511ca] 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 11: clone() 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 0> 2024-05-27T10:28:01.569+0000 7f3c68c00700 -1 *** Caught signal (Segmentation fault) ** 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: in thread 7f3c68c00700 thread_name:md_log_replay 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.951 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev) 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3c7d25bd20] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x561fd7dc9507] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x561fd7dc999e] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x561fd7dc4015] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x561fd7a83a27] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 6: (MDLog::_replay_thread()+0x1f38) [0x561fd7e2a06c] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561fd7e335ab] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 8: (Thread::entry_wrapper()+0x43) [0x7f3c7e6ad72d] 2024-05-27T10:28:01.952 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 9: (Thread::_entry_func(void*)+0xd) [0x7f3c7e6ad749] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f3c7d2511ca] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 11: clone() 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 0> 2024-05-27T10:28:01.569+0000 7f3c68c00700 -1 *** Caught signal (Segmentation fault) ** 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: in thread 7f3c68c00700 thread_name:md_log_replay 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev) 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3c7d25bd20] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x561fd7dc9507] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x561fd7dc999e] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x561fd7dc4015] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x561fd7a83a27] 2024-05-27T10:28:01.953 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 6: (MDLog::_replay_thread()+0x1f38) [0x561fd7e2a06c] 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 7: (MDLog::ReplayThread::entry()+0x11) [0x561fd7e335ab] 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 8: (Thread::entry_wrapper()+0x43) [0x7f3c7e6ad72d] 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 9: (Thread::_entry_func(void*)+0xd) [0x7f3c7e6ad749] 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f3c7d2511ca] 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 11: clone() 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[64239]: 2024-05-27T10:28:01.954 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:01 smithi104 podman[70231]: 2024-05-27 10:28:01.741110029 +0000 UTC m=+0.043618656 container died 53fa28eed6215ffe279d89ea8e94a03f8077d06536a8c96bf550b223053ba4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T10:28:02.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:02 smithi104 ceph-mon[34812]: pgmap v286: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 572 KiB/s wr, 89 op/s 2024-05-27T10:28:02.465 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:02 smithi042 ceph-mon[31661]: pgmap v286: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 572 KiB/s wr, 89 op/s 2024-05-27T10:28:02.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:02 smithi143 ceph-mon[35545]: pgmap v286: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 572 KiB/s wr, 89 op/s 2024-05-27T10:28:02.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 -- 172.21.15.42:0/4264538619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968811d920 msgr2=0x7f968811fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 --2- 172.21.15.42:0/4264538619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968811d920 0x7f968811fd40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9680009f90 tx=0x7f968000b330 comp rx=0 tx=0).stop 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 -- 172.21.15.42:0/4264538619 shutdown_connections 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 --2- 172.21.15.42:0/4264538619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968811d920 0x7f968811fd40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 --2- 172.21.15.42:0/4264538619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968810a920 0x7f968810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 --2- 172.21.15.42:0/4264538619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f968810b270 0x7f968811d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 -- 172.21.15.42:0/4264538619 >> 172.21.15.42:0/4264538619 conn(0x7f968806f4c0 msgr2=0x7f968806f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 -- 172.21.15.42:0/4264538619 shutdown_connections 2024-05-27T10:28:02.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.948+0000 7f968ca00700 1 -- 172.21.15.42:0/4264538619 wait complete. 2024-05-27T10:28:02.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 Processor -- start 2024-05-27T10:28:02.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 -- start start 2024-05-27T10:28:02.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 0x7f96881af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f968810b270 0x7f96881afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968811d920 0x7f96881a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96881231d0 con 0x7f968810a920 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9688123050 con 0x7f968810b270 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.950+0000 7f968ca00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9688123350 con 0x7f968811d920 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 0x7f96881af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 0x7f96881af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40896/0 (socket says 172.21.15.42:40896) 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 -- 172.21.15.42:0/2052653097 learned_addr learned my addr 172.21.15.42:0/2052653097 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685400700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f968810b270 0x7f96881afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9686800700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968811d920 0x7f96881a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 -- 172.21.15.42:0/2052653097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968811d920 msgr2=0x7f96881a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968811d920 0x7f96881a9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 -- 172.21.15.42:0/2052653097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f968810b270 msgr2=0x7f96881afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f968810b270 0x7f96881afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.951+0000 7f9685e00700 1 -- 172.21.15.42:0/2052653097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96881aa1b0 con 0x7f968810a920 2024-05-27T10:28:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.952+0000 7f9686800700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968811d920 0x7f96881a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.952+0000 7f9685e00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 0x7f96881af890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f967800b530 tx=0x7f967800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.953+0000 7f967ea00700 1 -- 172.21.15.42:0/2052653097 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678015020 con 0x7f968810a920 2024-05-27T10:28:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.953+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96881aa4a0 con 0x7f968810a920 2024-05-27T10:28:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.953+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96881b6740 con 0x7f968810a920 2024-05-27T10:28:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.954+0000 7f967ea00700 1 -- 172.21.15.42:0/2052653097 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9678011cc0 con 0x7f968810a920 2024-05-27T10:28:02.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.954+0000 7f967ea00700 1 -- 172.21.15.42:0/2052653097 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678010dc0 con 0x7f968810a920 2024-05-27T10:28:02.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.961+0000 7f967ea00700 1 -- 172.21.15.42:0/2052653097 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9678018020 con 0x7f968810a920 2024-05-27T10:28:02.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.964+0000 7f969b481b80 1 Processor -- start 2024-05-27T10:28:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.964+0000 7f969b481b80 1 -- start start 2024-05-27T10:28:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f967ea00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f966c0b29f0 0x7f966c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f967ea00700 1 -- 172.21.15.42:0/2052653097 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f96780e1bd0 con 0x7f968810a920 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f969b481b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f969b481b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f969b481b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f969b481b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565552fd4fd0 con 0x5655530a74d0 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f969b481b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565552fc4b70 con 0x5655530ad380 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.965+0000 7f969b481b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x565552fc8bb0 con 0x565552f27fd0 2024-05-27T10:28:02.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.966+0000 7f9686800700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f966c0b29f0 0x7f966c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.966+0000 7f9685400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.966+0000 7f9685400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38412/0 (socket says 172.21.15.42:38412) 2024-05-27T10:28:02.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.966+0000 7f9685400700 1 -- 172.21.15.42:0/932091024 learned_addr learned my addr 172.21.15.42:0/932091024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:02.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.966+0000 7f9685e00700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.966+0000 7f9686800700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9685e00700 1 -- 172.21.15.42:0/932091024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 msgr2=0x5655530a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9685e00700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9685e00700 1 -- 172.21.15.42:0/932091024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 msgr2=0x5655530a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9685e00700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9685e00700 1 -- 172.21.15.42:0/932091024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565552f8d5b0 con 0x565552f27fd0 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9686800700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:28:02.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.967+0000 7f9685400700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:28:02.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.968+0000 7f9686800700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f966c0b29f0 0x7f966c0b4f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9680009550 tx=0x7f9680004c30 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:02.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.968+0000 7f9685e00700 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ace40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96780df450 tx=0x7f96780a9c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d5155c11878ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.969+0000 7f9673400700 1 -- 172.21.15.42:0/932091024 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780b22e0 con 0x565552f27fd0 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.969+0000 7f9673400700 1 -- 172.21.15.42:0/932091024 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96780a96e0 con 0x565552f27fd0 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.969+0000 7f9673400700 1 -- 172.21.15.42:0/932091024 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678010770 con 0x565552f27fd0 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 -- 172.21.15.42:0/932091024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 msgr2=0x5655530ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ace40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96780df450 tx=0x7f96780a9c00 comp rx=0 tx=0).stop 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 -- 172.21.15.42:0/932091024 shutdown_connections 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ace40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 --2- 172.21.15.42:0/932091024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 -- 172.21.15.42:0/932091024 >> 172.21.15.42:0/932091024 conn(0x565552f88150 msgr2=0x5655530a2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.970+0000 7f969b481b80 1 -- 172.21.15.42:0/932091024 shutdown_connections 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.971+0000 7f969b481b80 1 -- 172.21.15.42:0/932091024 wait complete. 2024-05-27T10:28:02.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.972+0000 7f969b481b80 1 -- start start 2024-05-27T10:28:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.973+0000 7f969b481b80 1 Processor -- start 2024-05-27T10:28:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.974+0000 7f969b481b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.974+0000 7f969b481b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.974+0000 7f969b481b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.974+0000 7f969b481b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565552ecb130 con 0x5655530ad380 2024-05-27T10:28:02.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.974+0000 7f969b481b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565552f658f0 con 0x5655530a74d0 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.974+0000 7f969b481b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5655530b3e50 con 0x565552f27fd0 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.975+0000 7f969b481b80 10 client.0 _ll_register_callbacks cb 0x565552f6ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.975+0000 7f9685400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.975+0000 7f9685e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.975+0000 7f9686800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.975+0000 7f9685e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:48368/0 (socket says 172.21.15.42:48368) 2024-05-27T10:28:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.975+0000 7f9685e00700 1 -- 172.21.15.42:0/2037028327 learned_addr learned my addr 172.21.15.42:0/2037028327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.976+0000 7f9686800700 1 -- 172.21.15.42:0/2037028327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 msgr2=0x5655530b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.976+0000 7f9686800700 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.976+0000 7f9686800700 1 -- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 msgr2=0x5655530b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.976+0000 7f9686800700 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.976+0000 7f9686800700 1 -- 172.21.15.42:0/2037028327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565552f14a60 con 0x5655530ad380 2024-05-27T10:28:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.976+0000 7f9685e00700 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:28:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.977+0000 7f9686800700 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530b1530 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f968000b6c0 tx=0x7f968000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d8e9844e32b95a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.977+0000 7f9667e00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.977+0000 7f9667e00700 1 -- 172.21.15.42:0/2037028327 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968002f030 con 0x5655530ad380 2024-05-27T10:28:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.977+0000 7f9667e00700 1 -- 172.21.15.42:0/2037028327 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f9680026070 con 0x5655530ad380 2024-05-27T10:28:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.978+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96780a96e0 con 0x5655530ad380 2024-05-27T10:28:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.978+0000 7f9667e00700 1 -- 172.21.15.42:0/2037028327 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9680007710 con 0x5655530ad380 2024-05-27T10:28:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.978+0000 7f969b481b80 10 client.16833 fetch_fsmap learned FSMap version 34 2024-05-27T10:28:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.978+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565552f951f0 con 0x5655530ad380 2024-05-27T10:28:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f9667e00700 1 -- 172.21.15.42:0/2037028327 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96780a96e0 con 0x5655530ad380 2024-05-27T10:28:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f9667e00700 1 -- 172.21.15.42:0/2037028327 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f9680011070 con 0x5655530ad380 2024-05-27T10:28:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f969b481b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T10:28:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f969b481b80 10 client.16833 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f969b481b80 20 client.16833 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f969b481b80 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5655530c0cd0 0x5655530c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f969b481b80 4 client.16833 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.979+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565552fd6f20 con 0x5655530c0cd0 2024-05-27T10:28:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.980+0000 7f9685e00700 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5655530c0cd0 0x5655530c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.981+0000 7f9667e00700 10 client.16833 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.981+0000 7f9685e00700 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5655530c0cd0 0x5655530c3130 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f948e097da0285c7 server_cookie=8db49f4963bb04a5 in_seq=0 out_seq=0 2024-05-27T10:28:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f9667e00700 1 -- 172.21.15.42:0/2037028327 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565552fd6f20 con 0x5655530c0cd0 2024-05-27T10:28:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f9667e00700 10 client.16833 handle_command_reply: tid=0 2024-05-27T10:28:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 client.16833 shutdown 2024-05-27T10:28:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 msgr2=0x5655530b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530b1530 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f968000b6c0 tx=0x7f968000b330 comp rx=0 tx=0).stop 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 shutdown_connections 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5655530c0cd0 0x5655530c3130 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530b1530 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 --2- 172.21.15.42:0/2037028327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f9667e00700 0 client.16833 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.982+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 >> 172.21.15.42:0/2037028327 conn(0x565552f88150 msgr2=0x5655530a5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.983+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 shutdown_connections 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.983+0000 7f969b481b80 1 -- 172.21.15.42:0/2037028327 wait complete. 2024-05-27T10:28:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:02.983+0000 7f969b481b80 20 client.16833 trim_cache size 0 max 16384 2024-05-27T10:28:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.007+0000 7f969b481b80 1 Processor -- start 2024-05-27T10:28:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.007+0000 7f969b481b80 1 -- start start 2024-05-27T10:28:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.007+0000 7f969b481b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.007+0000 7f969b481b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.007+0000 7f969b481b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f969b481b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f966800ff00 con 0x5655530ad380 2024-05-27T10:28:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f969b481b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f966800d730 con 0x5655530a74d0 2024-05-27T10:28:03.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f969b481b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f966800b5a0 con 0x565552f27fd0 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38438/0 (socket says 172.21.15.42:38438) 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 -- 172.21.15.42:0/2167164639 learned_addr learned my addr 172.21.15.42:0/2167164639 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9686800700 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685e00700 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 -- 172.21.15.42:0/2167164639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 msgr2=0x5655530ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 -- 172.21.15.42:0/2167164639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 msgr2=0x5655530a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.008+0000 7f9685400700 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.009+0000 7f9685400700 1 -- 172.21.15.42:0/2167164639 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655530a2540 con 0x5655530a74d0 2024-05-27T10:28:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.009+0000 7f9685400700 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530bcf20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9674010560 tx=0x7f9674010870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c59297791fe60f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.009+0000 7f9672a00700 1 -- 172.21.15.42:0/2167164639 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674004020 con 0x5655530a74d0 2024-05-27T10:28:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.009+0000 7f9672a00700 1 -- 172.21.15.42:0/2167164639 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674003680 con 0x5655530a74d0 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.009+0000 7f9672a00700 1 -- 172.21.15.42:0/2167164639 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967401c650 con 0x5655530a74d0 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 -- 172.21.15.42:0/2167164639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 msgr2=0x5655530bcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530bcf20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9674010560 tx=0x7f9674010870 comp rx=0 tx=0).stop 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 -- 172.21.15.42:0/2167164639 shutdown_connections 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530ad380 0x5655530a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 --2- 172.21.15.42:0/2167164639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530a74d0 0x5655530bcf20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.010+0000 7f969b481b80 1 -- 172.21.15.42:0/2167164639 >> 172.21.15.42:0/2167164639 conn(0x565552f88150 msgr2=0x5655530a5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.011+0000 7f969b481b80 1 -- 172.21.15.42:0/2167164639 shutdown_connections 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.011+0000 7f969b481b80 1 -- 172.21.15.42:0/2167164639 wait complete. 2024-05-27T10:28:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.011+0000 7f969b481b80 1 -- start start 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 Processor -- start 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530b9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f966800bbf0 con 0x5655530a74d0 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f966800d060 con 0x5655530ad380 2024-05-27T10:28:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f966800c2c0 con 0x565552f27fd0 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.012+0000 7f969b481b80 10 client.0 _ll_register_callbacks cb 0x565552f6ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9686800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9686800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:38452/0 (socket says 172.21.15.42:38452) 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9686800700 1 -- 172.21.15.42:0/3699692055 learned_addr learned my addr 172.21.15.42:0/3699692055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685e00700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685400700 1 -- 172.21.15.42:0/3699692055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 msgr2=0x5655530b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685400700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685400700 1 -- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 msgr2=0x5655530b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685400700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685400700 1 -- 172.21.15.42:0/3699692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655530b6670 con 0x5655530a74d0 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9685e00700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:28:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.013+0000 7f9686800700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.014+0000 7f9685400700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530b9650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9674000c00 tx=0x7f9674003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8fc3353c79245 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.014+0000 7f9667e00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.014+0000 7f9667e00700 1 -- 172.21.15.42:0/3699692055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674004230 con 0x5655530a74d0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.014+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9674003680 con 0x5655530a74d0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.014+0000 7f9667e00700 1 -- 172.21.15.42:0/3699692055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8484+0+0 (secure 0 0 0) 0x7f96740043d0 con 0x5655530a74d0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f9667e00700 1 -- 172.21.15.42:0/3699692055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9674005020 con 0x5655530a74d0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 10 client.16839 fetch_fsmap learned FSMap version 34 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56555312bf90 con 0x5655530a74d0 2024-05-27T10:28:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f9667e00700 1 -- 172.21.15.42:0/3699692055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674003680 con 0x5655530a74d0 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f9667e00700 1 -- 172.21.15.42:0/3699692055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f967402c030 con 0x5655530a74d0 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 10 client.16839 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 10 client.16839 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 20 client.16839 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56555312c740 0x56555312eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 4 client.16839 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.3, "await_for": 45.0}] 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.015+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.3, "await_for": 45.0}) -- 0x565552fd0700 con 0x56555312c740 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.016+0000 7f9685e00700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56555312c740 0x56555312eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.016+0000 7f9685e00700 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56555312c740 0x56555312eb60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a726c704290f4bcd server_cookie=a4409351060fbe16 in_seq=0 out_seq=0 2024-05-27T10:28:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:03.016+0000 7f9667e00700 10 client.16839 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:03.196 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:02 smithi104 podman[70231]: 2024-05-27 10:28:02.793347034 +0000 UTC m=+1.095855682 container remove 53fa28eed6215ffe279d89ea8e94a03f8077d06536a8c96bf550b223053ba4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T10:28:03.196 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:02 smithi104 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service: Main process exited, code=exited, status=139/n/a 2024-05-27T10:28:03.696 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:03 smithi104 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-05-27T10:28:03.697 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:03 smithi104 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service: Consumed 5.635s CPU time. 2024-05-27T10:28:04.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:04 smithi143 ceph-mon[35545]: pgmap v287: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 572 KiB/s wr, 89 op/s 2024-05-27T10:28:04.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:04 smithi042 ceph-mon[31661]: pgmap v287: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 572 KiB/s wr, 89 op/s 2024-05-27T10:28:04.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:04 smithi104 ceph-mon[34812]: pgmap v287: 97 pgs: 97 active+clean; 383 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 572 KiB/s wr, 89 op/s 2024-05-27T10:28:06.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:06 smithi143 ceph-mon[35545]: pgmap v288: 97 pgs: 97 active+clean; 387 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-27T10:28:06.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:06 smithi143 ceph-mon[35545]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T10:28:06.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:06 smithi042 ceph-mon[31661]: pgmap v288: 97 pgs: 97 active+clean; 387 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-27T10:28:06.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:06 smithi042 ceph-mon[31661]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T10:28:06.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:06 smithi104 ceph-mon[34812]: pgmap v288: 97 pgs: 97 active+clean; 387 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-27T10:28:06.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:06 smithi104 ceph-mon[34812]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T10:28:07.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.103+0000 7f9667e00700 1 -- 172.21.15.42:0/3699692055 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565552fd0700 con 0x56555312c740 2024-05-27T10:28:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.103+0000 7f9667e00700 10 client.16839 handle_command_reply: tid=0 2024-05-27T10:28:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.103+0000 7f969b481b80 1 client.16839 shutdown 2024-05-27T10:28:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 msgr2=0x5655530b9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530b9650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9674000c00 tx=0x7f9674003c80 comp rx=0 tx=0).stop 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 shutdown_connections 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56555312c740 0x56555312eb60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655530ad380 0x5655530b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565552f27fd0 0x5655530b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 --2- 172.21.15.42:0/3699692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5655530a74d0 0x5655530b9650 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 >> 172.21.15.42:0/3699692055 conn(0x565552f88150 msgr2=0x5655530aaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.104+0000 7f9667e00700 0 client.16839 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.105+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 shutdown_connections 2024-05-27T10:28:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.105+0000 7f969b481b80 1 -- 172.21.15.42:0/3699692055 wait complete. 2024-05-27T10:28:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.105+0000 7f969b481b80 20 client.16839 trim_cache size 0 max 16384 2024-05-27T10:28:07.105 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:28:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-27T10:28:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:28:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 26, 2024-05-27T10:28:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "6a108fbe": { 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 26, 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:28:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 142.3, 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:07.107 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f966c0b29f0 msgr2=0x7f966c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f966c0b29f0 0x7f966c0b4f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9680009550 tx=0x7f9680004c30 comp rx=0 tx=0).stop 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 msgr2=0x7f96881af890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 0x7f96881af890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f967800b530 tx=0x7f967800b840 comp rx=0 tx=0).stop 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 shutdown_connections 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f966c0b29f0 0x7f966c0b4f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f968811d920 0x7f96881a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f968810a920 0x7f96881af890 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 --2- 172.21.15.42:0/2052653097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f968810b270 0x7f96881afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.106+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 >> 172.21.15.42:0/2052653097 conn(0x7f968806f4c0 msgr2=0x7f968811b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.107+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 shutdown_connections 2024-05-27T10:28:07.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:07.107+0000 7f968ca00700 1 -- 172.21.15.42:0/2052653097 wait complete. 2024-05-27T10:28:08.149 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a108fbe', quiesce took 12.7 seconds. Will release after: 9.600000000000001 2024-05-27T10:28:08.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:08 smithi042 ceph-mon[31661]: pgmap v290: 97 pgs: 97 active+clean; 387 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 995 KiB/s wr, 153 op/s 2024-05-27T10:28:08.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:08 smithi143 ceph-mon[35545]: pgmap v290: 97 pgs: 97 active+clean; 387 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 995 KiB/s wr, 153 op/s 2024-05-27T10:28:08.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:08 smithi104 ceph-mon[34812]: pgmap v290: 97 pgs: 97 active+clean; 387 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 995 KiB/s wr, 153 op/s 2024-05-27T10:28:10.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:10 smithi143 ceph-mon[35545]: pgmap v291: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 740 KiB/s wr, 167 op/s 2024-05-27T10:28:10.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:10 smithi042 ceph-mon[31661]: pgmap v291: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 740 KiB/s wr, 167 op/s 2024-05-27T10:28:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:10 smithi104 ceph-mon[34812]: pgmap v291: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 740 KiB/s wr, 167 op/s 2024-05-27T10:28:12.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:11 smithi104 ceph-mon[34812]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T10:28:12.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:11 smithi143 ceph-mon[35545]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T10:28:12.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:11 smithi042 ceph-mon[31661]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T10:28:13.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:12 smithi104 ceph-mon[34812]: pgmap v293: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 925 KiB/s wr, 209 op/s 2024-05-27T10:28:13.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:12 smithi143 ceph-mon[35545]: pgmap v293: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 925 KiB/s wr, 209 op/s 2024-05-27T10:28:13.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:12 smithi042 ceph-mon[31661]: pgmap v293: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 925 KiB/s wr, 209 op/s 2024-05-27T10:28:13.772 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:13 smithi104 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 1. 2024-05-27T10:28:13.772 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:13 smithi104 systemd[1]: Stopped Ceph mds.k for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:28:13.772 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:13 smithi104 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service: Consumed 5.635s CPU time. 2024-05-27T10:28:13.773 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:13 smithi104 systemd[1]: Starting Ceph mds.k for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:28:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.790+0000 7f094a400640 1 -- 172.21.15.42:0/4097889918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09441148f0 msgr2=0x7f0944116d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.790+0000 7f094a400640 1 --2- 172.21.15.42:0/4097889918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09441148f0 0x7f0944116d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f093c009be0 tx=0x7f093c02a6c0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 -- 172.21.15.42:0/4097889918 shutdown_connections 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 --2- 172.21.15.42:0/4097889918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09441148f0 0x7f0944116d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 --2- 172.21.15.42:0/4097889918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0944111f90 0x7f09441143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 --2- 172.21.15.42:0/4097889918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0944078720 0x7f0944078b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 -- 172.21.15.42:0/4097889918 >> 172.21.15.42:0/4097889918 conn(0x7f0944101300 msgr2=0x7f0944103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 -- 172.21.15.42:0/4097889918 shutdown_connections 2024-05-27T10:28:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.791+0000 7f094a400640 1 -- 172.21.15.42:0/4097889918 wait complete. 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 Processor -- start 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 -- start start 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 0x7f09441a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0944111f90 0x7f09441a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09441148f0 0x7f09441a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f094411a130 con 0x7f09441148f0 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0944119fb0 con 0x7f0944078720 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.792+0000 7f094a400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f094411a2b0 con 0x7f0944111f90 2024-05-27T10:28:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 0x7f09441a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 0x7f09441a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:60966/0 (socket says 172.21.15.42:60966) 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 -- 172.21.15.42:0/1166840234 learned_addr learned my addr 172.21.15.42:0/1166840234 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949a00640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09441148f0 0x7f09441a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0943e00640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0944111f90 0x7f09441a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 -- 172.21.15.42:0/1166840234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0944111f90 msgr2=0x7f09441a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0944111f90 0x7f09441a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 -- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09441148f0 msgr2=0x7f09441a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09441148f0 0x7f09441a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 -- 172.21.15.42:0/1166840234 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09441a1050 con 0x7f0944078720 2024-05-27T10:28:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949a00640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09441148f0 0x7f09441a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.793+0000 7f0949000640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 0x7f09441a67a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f093800e9c0 tx=0x7f093800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.794+0000 7f0941600640 1 -- 172.21.15.42:0/1166840234 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093800cde0 con 0x7f0944078720 2024-05-27T10:28:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.794+0000 7f0941600640 1 -- 172.21.15.42:0/1166840234 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09380035f0 con 0x7f0944078720 2024-05-27T10:28:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.794+0000 7f0941600640 1 -- 172.21.15.42:0/1166840234 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938010690 con 0x7f0944078720 2024-05-27T10:28:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.794+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09441a1340 con 0x7f0944078720 2024-05-27T10:28:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.794+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09441ad620 con 0x7f0944078720 2024-05-27T10:28:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.797+0000 7f09529ce740 1 Processor -- start 2024-05-27T10:28:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.797+0000 7f09529ce740 1 -- start start 2024-05-27T10:28:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f09529ce740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f09529ce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f0941600640 1 -- 172.21.15.42:0/1166840234 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0938004000 con 0x7f0944078720 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f09529ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f09529ce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55755072cb90 con 0x5575503f02f0 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f09529ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55755071c730 con 0x5575503f84d0 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f09529ce740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557550720770 con 0x5575504191b0 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f0949000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f0949000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507bbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57118/0 (socket says 172.21.15.42:57118) 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f0949000640 1 -- 172.21.15.42:0/398128786 learned_addr learned my addr 172.21.15.42:0/398128786 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.798+0000 7f0943e00640 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949a00640 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949000640 1 -- 172.21.15.42:0/398128786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 msgr2=0x5575507bba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949000640 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507bba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949000640 1 -- 172.21.15.42:0/398128786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 msgr2=0x5575507bc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949000640 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507bc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949000640 1 -- 172.21.15.42:0/398128786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575506b58b0 con 0x5575503f02f0 2024-05-27T10:28:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.799+0000 7f0949000640 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507bbfa0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f09380e0010 tx=0x7f09380e8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3441f6c8dc9561e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f0932000640 1 -- 172.21.15.42:0/398128786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380f2020 con 0x5575503f02f0 2024-05-27T10:28:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f0932000640 1 -- 172.21.15.42:0/398128786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09380eae10 con 0x5575503f02f0 2024-05-27T10:28:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f0932000640 1 -- 172.21.15.42:0/398128786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380eee50 con 0x5575503f02f0 2024-05-27T10:28:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f0941600640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f092c0b29d0 0x7f092c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f0941600640 1 -- 172.21.15.42:0/1166840234 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f09380e29c0 con 0x7f0944078720 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f0949a00640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f092c0b29d0 0x7f092c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.800+0000 7f09529ce740 1 -- 172.21.15.42:0/398128786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 msgr2=0x5575507bbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507bbfa0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f09380e0010 tx=0x7f09380e8ac0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 -- 172.21.15.42:0/398128786 shutdown_connections 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507bc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507bbfa0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 --2- 172.21.15.42:0/398128786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 -- 172.21.15.42:0/398128786 >> 172.21.15.42:0/398128786 conn(0x55755040d300 msgr2=0x5575507b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 -- 172.21.15.42:0/398128786 shutdown_connections 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 -- 172.21.15.42:0/398128786 wait complete. 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f0949a00640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f092c0b29d0 0x7f092c0b4ee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5575507b8bd0 tx=0x7f093c004a80 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.801+0000 7f09529ce740 1 -- start start 2024-05-27T10:28:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 Processor -- start 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575503f02f0 0x5575507c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575504191b0 0x5575507d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557550402790 con 0x5575504191b0 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5575506c4bc0 con 0x5575503f84d0 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5575507c8b90 con 0x5575503f02f0 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.802+0000 7f09529ce740 10 client.0 _ll_register_callbacks cb 0x5575506a2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575504191b0 0x5575507d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0943e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0943e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:60988/0 (socket says 172.21.15.42:60988) 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0943e00640 1 -- 172.21.15.42:0/1809514443 learned_addr learned my addr 172.21.15.42:0/1809514443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949000640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575503f02f0 0x5575507c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 -- 172.21.15.42:0/1809514443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575503f02f0 msgr2=0x5575507c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575503f02f0 0x5575507c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 -- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 msgr2=0x5575507c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 -- 172.21.15.42:0/1809514443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557550431210 con 0x5575504191b0 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949000640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575503f02f0 0x5575507c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.803+0000 7f0949a00640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575504191b0 0x5575507d1f20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f093c02abe0 tx=0x7f093c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2239503338c5fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f0926a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f0926a00640 1 -- 172.21.15.42:0/1809514443 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c044070 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09380eae10 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f0926a00640 1 -- 172.21.15.42:0/1809514443 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f093c004250 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f0926a00640 1 -- 172.21.15.42:0/1809514443 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f093c058050 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f09529ce740 10 client.16851 fetch_fsmap learned FSMap version 34 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.804+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575504fd620 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f0926a00640 1 -- 172.21.15.42:0/1809514443 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09380eae10 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f0926a00640 1 -- 172.21.15.42:0/1809514443 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f093c03c040 con 0x5575504191b0 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f09529ce740 10 client.16851 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f09529ce740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f09529ce740 20 client.16851 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f09529ce740 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503d7d80 0x5575507d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f09529ce740 4 client.16851 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755072eae0 con 0x5575503d7d80 2024-05-27T10:28:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.805+0000 7f0949000640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503d7d80 0x5575507d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.806+0000 7f0949000640 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503d7d80 0x5575507d7f50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d016757c249fcfc7 server_cookie=4b991fab3578fc92 in_seq=0 out_seq=0 2024-05-27T10:28:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.806+0000 7f0926a00640 10 client.16851 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.807+0000 7f0926a00640 1 -- 172.21.15.42:0/1809514443 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55755072eae0 con 0x5575503d7d80 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.807+0000 7f0926a00640 10 client.16851 handle_command_reply: tid=0 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.807+0000 7f09529ce740 1 client.16851 shutdown 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575504191b0 msgr2=0x5575507d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575504191b0 0x5575507d1f20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f093c02abe0 tx=0x7f093c02a6c0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 shutdown_connections 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575504191b0 0x5575507d1f20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503d7d80 0x5575507d7f50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 --2- 172.21.15.42:0/1809514443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575503f02f0 0x5575507c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f0926a00640 0 client.16851 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 >> 172.21.15.42:0/1809514443 conn(0x55755040d300 msgr2=0x5575507bb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 shutdown_connections 2024-05-27T10:28:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 1 -- 172.21.15.42:0/1809514443 wait complete. 2024-05-27T10:28:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.808+0000 7f09529ce740 20 client.16851 trim_cache size 0 max 16384 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 Processor -- start 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 -- start start 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b86f0 con 0x5575503f02f0 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b5f60 con 0x5575503f84d0 2024-05-27T10:28:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.825+0000 7f09529ce740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b3dd0 con 0x5575504191b0 2024-05-27T10:28:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57150/0 (socket says 172.21.15.42:57150) 2024-05-27T10:28:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 -- 172.21.15.42:0/2021972855 learned_addr learned my addr 172.21.15.42:0/2021972855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0943e00640 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949a00640 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 -- 172.21.15.42:0/2021972855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 msgr2=0x5575507d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 -- 172.21.15.42:0/2021972855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 msgr2=0x5575507c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.826+0000 7f0949000640 1 -- 172.21.15.42:0/2021972855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575507b8c00 con 0x5575503f02f0 2024-05-27T10:28:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.827+0000 7f0949000640 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507b72e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0938003db0 tx=0x7f09380e82a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc31b52dd618be1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.827+0000 7f0931600640 1 -- 172.21.15.42:0/2021972855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380f2070 con 0x5575503f02f0 2024-05-27T10:28:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.827+0000 7f0931600640 1 -- 172.21.15.42:0/2021972855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0938091040 con 0x5575503f02f0 2024-05-27T10:28:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.827+0000 7f0931600640 1 -- 172.21.15.42:0/2021972855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380a53f0 con 0x5575503f02f0 2024-05-27T10:28:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 -- 172.21.15.42:0/2021972855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 msgr2=0x5575507b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507b72e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0938003db0 tx=0x7f09380e82a0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 -- 172.21.15.42:0/2021972855 shutdown_connections 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 --2- 172.21.15.42:0/2021972855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507b72e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 -- 172.21.15.42:0/2021972855 >> 172.21.15.42:0/2021972855 conn(0x55755040d300 msgr2=0x5575507c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 -- 172.21.15.42:0/2021972855 shutdown_connections 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.828+0000 7f09529ce740 1 -- 172.21.15.42:0/2021972855 wait complete. 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.829+0000 7f09529ce740 1 -- start start 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.829+0000 7f09529ce740 1 Processor -- start 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b4420 con 0x5575503f02f0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b5890 con 0x5575503f84d0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b4af0 con 0x5575504191b0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f09529ce740 10 client.0 _ll_register_callbacks cb 0x7f093c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57160/0 (socket says 172.21.15.42:57160) 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 -- 172.21.15.42:0/2133335362 learned_addr learned my addr 172.21.15.42:0/2133335362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0943e00640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949a00640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 -- 172.21.15.42:0/2133335362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 msgr2=0x5575507c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 -- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 msgr2=0x5575507ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.830+0000 7f0949000640 1 -- 172.21.15.42:0/2133335362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575507c9d30 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f0949000640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507ce130 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0938003fd0 tx=0x7f09380043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f6c2f09318c89f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f0926a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f0926a00640 1 -- 172.21.15.42:0/2133335362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380f2070 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f0926a00640 1 -- 172.21.15.42:0/2133335362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f09380037a0 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0938091040 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f0926a00640 1 -- 172.21.15.42:0/2133335362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f09380a53f0 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f09529ce740 10 client.16857 fetch_fsmap learned FSMap version 34 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575508400b0 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.831+0000 7f0926a00640 1 -- 172.21.15.42:0/2133335362 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0938091040 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f0926a00640 1 -- 172.21.15.42:0/2133335362 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f09380c4050 con 0x5575503f02f0 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f09529ce740 10 client.16857 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T10:28:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f09529ce740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f09529ce740 20 client.16857 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f09529ce740 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503ad1c0 0x5575508426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f09529ce740 4 client.16857 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.832+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575507282c0 con 0x5575503ad1c0 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.833+0000 7f0943e00640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503ad1c0 0x5575508426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.833+0000 7f0943e00640 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503ad1c0 0x5575508426e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de974677c0df2ef server_cookie=f3de6860cdb79b53 in_seq=0 out_seq=0 2024-05-27T10:28:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.833+0000 7f0926a00640 10 client.16857 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.856+0000 7f0926a00640 1 -- 172.21.15.42:0/2133335362 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5575507282c0 con 0x5575503ad1c0 2024-05-27T10:28:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f0926a00640 10 client.16857 handle_command_reply: tid=0 2024-05-27T10:28:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 client.16857 shutdown 2024-05-27T10:28:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 msgr2=0x5575507ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507ce130 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0938003fd0 tx=0x7f09380043f0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 shutdown_connections 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5575503ad1c0 0x5575508426e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575504191b0 0x5575507c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575503f84d0 0x5575507ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f0926a00640 0 client.16857 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 --2- 172.21.15.42:0/2133335362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575503f02f0 0x5575507ce130 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 >> 172.21.15.42:0/2133335362 conn(0x55755040d300 msgr2=0x5575507bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 shutdown_connections 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.857+0000 7f09529ce740 1 -- 172.21.15.42:0/2133335362 wait complete. 2024-05-27T10:28:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.858+0000 7f09529ce740 20 client.16857 trim_cache size 0 max 16384 2024-05-27T10:28:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:28:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:28:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:28:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f092c0b29d0 msgr2=0x7f092c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f092c0b29d0 0x7f092c0b4ee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5575507b8bd0 tx=0x7f093c004a80 comp rx=0 tx=0).stop 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 msgr2=0x7f09441a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 0x7f09441a67a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f093800e9c0 tx=0x7f093800ee80 comp rx=0 tx=0).stop 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 shutdown_connections 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f092c0b29d0 0x7f092c0b4ee0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09441148f0 0x7f09441a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0944111f90 0x7f09441a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 --2- 172.21.15.42:0/1166840234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0944078720 0x7f09441a67a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 >> 172.21.15.42:0/1166840234 conn(0x7f0944101300 msgr2=0x7f0944113310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 shutdown_connections 2024-05-27T10:28:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:13.859+0000 7f094a400640 1 -- 172.21.15.42:0/1166840234 wait complete. 2024-05-27T10:28:14.058 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:13 smithi104 podman[70326]: 2024-05-27 10:28:13.750760124 +0000 UTC m=+0.031975229 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:28:14.059 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:13 smithi104 podman[70326]: 2024-05-27 10:28:13.911889292 +0000 UTC m=+0.193104430 container create 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521) 2024-05-27T10:28:14.447 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 podman[70326]: 2024-05-27 10:28:14.16569689 +0000 UTC m=+0.446912054 container init 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T10:28:14.447 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 podman[70326]: 2024-05-27 10:28:14.174709311 +0000 UTC m=+0.455924430 container start 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:28:14.448 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 bash[70326]: 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 2024-05-27T10:28:14.448 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 systemd[1]: Started Ceph mds.k for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:28:14.448 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 ceph-mds[70352]: -- 172.21.15.104:0/558764246 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa18f6c4e0 con 0x55fa18fb7a80 2024-05-27T10:28:14.448 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 ceph-mds[70352]: -- 172.21.15.104:0/558764246 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa18f6fa40 con 0x55fa18fb7a80 2024-05-27T10:28:14.448 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:28:14 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[70346]: starting mds.k at 2024-05-27T10:28:14.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:14 smithi104 ceph-mon[34812]: pgmap v294: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 189 KiB/s wr, 92 op/s 2024-05-27T10:28:14.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:14 smithi104 ceph-mon[34812]: daemon mds.k restarted 2024-05-27T10:28:15.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:14 smithi143 ceph-mon[35545]: pgmap v294: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 189 KiB/s wr, 92 op/s 2024-05-27T10:28:15.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:14 smithi143 ceph-mon[35545]: daemon mds.k restarted 2024-05-27T10:28:15.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:14 smithi042 ceph-mon[31661]: pgmap v294: 97 pgs: 97 active+clean; 388 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 189 KiB/s wr, 92 op/s 2024-05-27T10:28:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:14 smithi042 ceph-mon[31661]: daemon mds.k restarted 2024-05-27T10:28:16.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:15 smithi104 ceph-mon[34812]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T10:28:16.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:15 smithi104 ceph-mon[34812]: mds.? [v2:172.21.15.104:6832/755631209,v1:172.21.15.104:6833/755631209] up:boot 2024-05-27T10:28:16.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:15 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T10:28:16.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:15 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:28:16.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:15 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:16.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:15 smithi143 ceph-mon[35545]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T10:28:16.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:15 smithi143 ceph-mon[35545]: mds.? [v2:172.21.15.104:6832/755631209,v1:172.21.15.104:6833/755631209] up:boot 2024-05-27T10:28:16.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:15 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T10:28:16.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:15 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:28:16.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:15 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:16.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:15 smithi042 ceph-mon[31661]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T10:28:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:15 smithi042 ceph-mon[31661]: mds.? [v2:172.21.15.104:6832/755631209,v1:172.21.15.104:6833/755631209] up:boot 2024-05-27T10:28:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:15 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T10:28:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:15 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T10:28:16.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:15 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:17.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:16 smithi104 ceph-mon[34812]: pgmap v296: 97 pgs: 97 active+clean; 389 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 212 KiB/s wr, 99 op/s 2024-05-27T10:28:17.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:16 smithi143 ceph-mon[35545]: pgmap v296: 97 pgs: 97 active+clean; 389 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 212 KiB/s wr, 99 op/s 2024-05-27T10:28:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:16 smithi042 ceph-mon[31661]: pgmap v296: 97 pgs: 97 active+clean; 389 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 212 KiB/s wr, 99 op/s 2024-05-27T10:28:17.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:28:18.224 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:19.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:18 smithi104 ceph-mon[34812]: pgmap v297: 97 pgs: 97 active+clean; 389 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 6 op/s 2024-05-27T10:28:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:18 smithi143 ceph-mon[35545]: pgmap v297: 97 pgs: 97 active+clean; 389 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 6 op/s 2024-05-27T10:28:19.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:18 smithi042 ceph-mon[31661]: pgmap v297: 97 pgs: 97 active+clean; 389 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 6 op/s 2024-05-27T10:28:20.973 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:20 smithi042 ceph-mon[31661]: pgmap v298: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-27T10:28:20.973 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:20 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:28:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.072+0000 7ff10cc00700 1 -- 172.21.15.42:0/2694217412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 msgr2=0x7ff108109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.072+0000 7ff10cc00700 1 --2- 172.21.15.42:0/2694217412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff108109130 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40099c0 tx=0x7ff0f4009cd0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 -- 172.21.15.42:0/2694217412 shutdown_connections 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 --2- 172.21.15.42:0/2694217412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff108109670 0x7ff108110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 --2- 172.21.15.42:0/2694217412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1081043f0 0x7ff108104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 --2- 172.21.15.42:0/2694217412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff108109130 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 -- 172.21.15.42:0/2694217412 >> 172.21.15.42:0/2694217412 conn(0x7ff108079d00 msgr2=0x7ff10807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 -- 172.21.15.42:0/2694217412 shutdown_connections 2024-05-27T10:28:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.073+0000 7ff10cc00700 1 -- 172.21.15.42:0/2694217412 wait complete. 2024-05-27T10:28:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.074+0000 7ff10cc00700 1 Processor -- start 2024-05-27T10:28:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.074+0000 7ff10cc00700 1 -- start start 2024-05-27T10:28:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.074+0000 7ff10cc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 0x7ff1081a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.074+0000 7ff10cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff1081a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.074+0000 7ff10cc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff108109670 0x7ff10819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff10cc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff108114180 con 0x7ff108108d20 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff10cc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff108114000 con 0x7ff108109670 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff10cc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff108114300 con 0x7ff1081043f0 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 0x7ff1081a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff106a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff1081a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 0x7ff1081a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.42:42824/0 (socket says 172.21.15.42:42824) 2024-05-27T10:28:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 -- 172.21.15.42:0/839478449 learned_addr learned my addr 172.21.15.42:0/839478449 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107e00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff108109670 0x7ff10819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 -- 172.21.15.42:0/839478449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff108109670 msgr2=0x7ff10819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff108109670 0x7ff10819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 -- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 msgr2=0x7ff1081a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.075+0000 7ff107400700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff1081a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff107400700 1 -- 172.21.15.42:0/839478449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10819b070 con 0x7ff1081043f0 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff107e00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff108109670 0x7ff10819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff106a00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff1081a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff107400700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 0x7ff1081a07a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d980 tx=0x7ff0f800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8016070 con 0x7ff1081043f0 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10819b360 con 0x7ff1081043f0 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0f8004830 con 0x7ff1081043f0 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.076+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8005440 con 0x7ff1081043f0 2024-05-27T10:28:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.077+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081a7630 con 0x7ff1081043f0 2024-05-27T10:28:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.078+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1081058a0 con 0x7ff1081043f0 2024-05-27T10:28:21.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.082+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff0f80055e0 con 0x7ff1081043f0 2024-05-27T10:28:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.083+0000 7ff0ffe00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff0ec0b29f0 0x7ff0ec0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.083+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff0f80e28e0 con 0x7ff1081043f0 2024-05-27T10:28:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.084+0000 7ff106a00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff0ec0b29f0 0x7ff0ec0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.084+0000 7ff106a00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff0ec0b29f0 0x7ff0ec0b4f00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4006010 tx=0x7ff0f401a040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:21.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.087+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0f801f020 con 0x7ff1081043f0 2024-05-27T10:28:21.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:20 smithi104 ceph-mon[34812]: pgmap v298: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-27T10:28:21.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:20 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:28:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:20 smithi143 ceph-mon[35545]: pgmap v298: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-27T10:28:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:20 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:28:21.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.379+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff10802e5e0 con 0x7ff1081043f0 2024-05-27T10:28:21.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.382+0000 7ff0ffe00700 1 -- 172.21.15.42:0/839478449 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11879 (secure 0 0 0) 0x7ff0f80aa2b0 con 0x7ff1081043f0 2024-05-27T10:28:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:28:21.381 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:28:14.904306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_36359":{"gid":36359,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/755631209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":755631209},{"type":"v1","addr":"172.21.15.104:6833","nonce":755631209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.385+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff0ec0b29f0 msgr2=0x7ff0ec0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.385+0000 7ff10cc00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff0ec0b29f0 0x7ff0ec0b4f00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4006010 tx=0x7ff0f401a040 comp rx=0 tx=0).stop 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.385+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 msgr2=0x7ff1081a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.385+0000 7ff10cc00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 0x7ff1081a07a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d980 tx=0x7ff0f800dc90 comp rx=0 tx=0).stop 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.385+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 shutdown_connections 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff0ec0b29f0 0x7ff0ec0b4f00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff108109670 0x7ff10819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1081043f0 0x7ff1081a07a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 --2- 172.21.15.42:0/839478449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff108108d20 0x7ff1081a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 >> 172.21.15.42:0/839478449 conn(0x7ff108079d00 msgr2=0x7ff108101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 shutdown_connections 2024-05-27T10:28:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:21.386+0000 7ff10cc00700 1 -- 172.21.15.42:0/839478449 wait complete. 2024-05-27T10:28:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-05-27T10:28:22.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:21 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/839478449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:22.199 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:21 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/839478449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:22.239 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id 6a108fbe --release --await' 2024-05-27T10:28:22.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id 6a108fbe --release --await 2024-05-27T10:28:22.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:21 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/839478449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:22.702 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:23.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:22 smithi042 ceph-mon[31661]: pgmap v299: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-27T10:28:23.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:22 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:28:23.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:22 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:28:23.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:22 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:28:23.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:22 smithi104 ceph-mon[34812]: pgmap v299: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-27T10:28:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:22 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:28:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:22 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:28:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:22 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:28:23.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:22 smithi143 ceph-mon[35545]: pgmap v299: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-27T10:28:23.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:22 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:28:23.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:22 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:28:23.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:22 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:28:25.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:24 smithi104 ceph-mon[34812]: pgmap v300: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-27T10:28:25.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.244+0000 7fac1ca00700 1 -- 172.21.15.42:0/4069371593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac18106610 msgr2=0x7fac18106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.244+0000 7fac1ca00700 1 --2- 172.21.15.42:0/4069371593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac18106610 0x7fac18106a20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fac080099c0 tx=0x7fac08009cd0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 -- 172.21.15.42:0/4069371593 shutdown_connections 2024-05-27T10:28:25.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 --2- 172.21.15.42:0/4069371593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac1810b890 0x7fac181131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 --2- 172.21.15.42:0/4069371593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac18106610 0x7fac18106a20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 --2- 172.21.15.42:0/4069371593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac1810af40 0x7fac1810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 -- 172.21.15.42:0/4069371593 >> 172.21.15.42:0/4069371593 conn(0x7fac18102020 msgr2=0x7fac18104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:25.245 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:24 smithi042 ceph-mon[31661]: pgmap v300: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-27T10:28:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 -- 172.21.15.42:0/4069371593 shutdown_connections 2024-05-27T10:28:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.245+0000 7fac1ca00700 1 -- 172.21.15.42:0/4069371593 wait complete. 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 Processor -- start 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 -- start start 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac18106610 0x7fac1819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 0x7fac1819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac1810b890 0x7fac181a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac181163c0 con 0x7fac1810af40 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac18116240 con 0x7fac18106610 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.247+0000 7fac1ca00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fac18116540 con 0x7fac1810b890 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 0x7fac1819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 0x7fac1819d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33142/0 (socket says 172.21.15.42:33142) 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 -- 172.21.15.42:0/3420547934 learned_addr learned my addr 172.21.15.42:0/3420547934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac16800700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac18106610 0x7fac1819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac17200700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac1810b890 0x7fac181a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 -- 172.21.15.42:0/3420547934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac1810b890 msgr2=0x7fac181a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac1810b890 0x7fac181a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 -- 172.21.15.42:0/3420547934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac18106610 msgr2=0x7fac1819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac18106610 0x7fac1819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.248+0000 7fac15e00700 1 -- 172.21.15.42:0/3420547934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac181a1c20 con 0x7fac1810af40 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.249+0000 7fac15e00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 0x7fac1819d2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e970 tx=0x7fac0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.249+0000 7fac0f400700 1 -- 172.21.15.42:0/3420547934 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04004a30 con 0x7fac1810af40 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.249+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac181a1f10 con 0x7fac1810af40 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.249+0000 7fac0f400700 1 -- 172.21.15.42:0/3420547934 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac0400cc70 con 0x7fac1810af40 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.249+0000 7fac0f400700 1 -- 172.21.15.42:0/3420547934 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04005520 con 0x7fac1810af40 2024-05-27T10:28:25.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.250+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac181a9830 con 0x7fac1810af40 2024-05-27T10:28:25.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.254+0000 7fac0f400700 1 -- 172.21.15.42:0/3420547934 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fac0400c7e0 con 0x7fac1810af40 2024-05-27T10:28:25.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.255+0000 7fac2bf45b80 1 Processor -- start 2024-05-27T10:28:25.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.255+0000 7fac2bf45b80 1 -- start start 2024-05-27T10:28:25.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f468b3f40 0x564f46a334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 0x564f46a3d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a3f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564f46960f40 con 0x564f46a339f0 2024-05-27T10:28:25.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564f46950ae0 con 0x564f468b3f40 2024-05-27T10:28:25.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564f46954b20 con 0x564f46a33e00 2024-05-27T10:28:25.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac16800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 0x564f46a3d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac16800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 0x564f46a3d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33146/0 (socket says 172.21.15.42:33146) 2024-05-27T10:28:25.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac16800700 1 -- 172.21.15.42:0/3519092918 learned_addr learned my addr 172.21.15.42:0/3519092918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac15e00700 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a3f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.256+0000 7fac17200700 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f468b3f40 0x564f46a334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac16800700 1 -- 172.21.15.42:0/3519092918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 msgr2=0x564f46a3f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac16800700 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a3f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac16800700 1 -- 172.21.15.42:0/3519092918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f468b3f40 msgr2=0x564f46a334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac16800700 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f468b3f40 0x564f46a334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac16800700 1 -- 172.21.15.42:0/3519092918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f46919520 con 0x564f46a339f0 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac15e00700 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a3f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac0f400700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fabfc0b29f0 0x7fabfc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac16800700 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 0x564f46a3d050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fac08015000 tx=0x7fac08005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de3a84da2eff774 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac0f400700 1 -- 172.21.15.42:0/3420547934 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fac040e2800 con 0x7fac1810af40 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac17200700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fabfc0b29f0 0x7fabfc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.257+0000 7fac03e00700 1 -- 172.21.15.42:0/3519092918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08027070 con 0x564f46a339f0 2024-05-27T10:28:25.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.258+0000 7fac03e00700 1 -- 172.21.15.42:0/3519092918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac080037c0 con 0x564f46a339f0 2024-05-27T10:28:25.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.258+0000 7fac03e00700 1 -- 172.21.15.42:0/3519092918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0800b9f0 con 0x564f46a339f0 2024-05-27T10:28:25.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3519092918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 msgr2=0x564f46a3d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 0x564f46a3d050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fac08015000 tx=0x7fac08005040 comp rx=0 tx=0).stop 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac17200700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fabfc0b29f0 0x7fabfc0b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x564f46a2ec70 tx=0x7fac1000b040 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3519092918 shutdown_connections 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a3f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f468b3f40 0x564f46a334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3519092918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a339f0 0x564f46a3d050 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3519092918 >> 172.21.15.42:0/3519092918 conn(0x564f469140c0 msgr2=0x564f46a2e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3519092918 shutdown_connections 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.259+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3519092918 wait complete. 2024-05-27T10:28:25.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.260+0000 7fac2bf45b80 1 -- start start 2024-05-27T10:28:25.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.260+0000 7fac2bf45b80 1 Processor -- start 2024-05-27T10:28:25.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a3cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a339f0 0x564f46a3d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a33e00 0x564f46a3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564f468570a0 con 0x564f468b3f40 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564f468f1860 con 0x564f46a33e00 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564f46a3feb0 con 0x564f46a339f0 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac2bf45b80 10 client.0 _ll_register_callbacks cb 0x564f468fad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac16800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a3cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac16800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a3cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33148/0 (socket says 172.21.15.42:33148) 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.261+0000 7fac16800700 1 -- 172.21.15.42:0/3973730349 learned_addr learned my addr 172.21.15.42:0/3973730349 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac15e00700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a339f0 0x564f46a3d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac17200700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a33e00 0x564f46a3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac15e00700 1 -- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a33e00 msgr2=0x564f46a3d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac15e00700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a33e00 0x564f46a3d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac15e00700 1 -- 172.21.15.42:0/3973730349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 msgr2=0x564f46a3cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac15e00700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a3cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.262+0000 7fac15e00700 1 -- 172.21.15.42:0/3973730349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f468a09d0 con 0x564f46a339f0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.263+0000 7fac15e00700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a339f0 0x564f46a3d160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac040dff40 tx=0x7fac040a94a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeaeaface26815f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.263+0000 7fac00c00700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.263+0000 7fac00c00700 1 -- 172.21.15.42:0/3973730349 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac040a5260 con 0x564f46a339f0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.264+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac080037c0 con 0x564f46a339f0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.264+0000 7fac00c00700 1 -- 172.21.15.42:0/3973730349 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fac040a60c0 con 0x564f46a339f0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.264+0000 7fac00c00700 1 -- 172.21.15.42:0/3973730349 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fac040c2030 con 0x564f46a339f0 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.264+0000 7fac2bf45b80 10 client.26383 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:25.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.264+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564f46921160 con 0x564f46a339f0 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.264+0000 7fac00c00700 1 -- 172.21.15.42:0/3973730349 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac080037c0 con 0x564f46a339f0 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac00c00700 1 -- 172.21.15.42:0/3973730349 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fac040a9d30 con 0x564f46a339f0 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac2bf45b80 10 client.26383 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac2bf45b80 10 client.26383 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac2bf45b80 20 client.26383 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46a4cf90 0x564f46a4f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac2bf45b80 4 client.26383 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.265+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f46962e90 con 0x564f46a4cf90 2024-05-27T10:28:25.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.266+0000 7fac16800700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46a4cf90 0x564f46a4f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.267+0000 7fac16800700 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46a4cf90 0x564f46a4f410 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d818c50ea403117d server_cookie=2f359c791491d9b3 in_seq=0 out_seq=0 2024-05-27T10:28:25.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.267+0000 7fac00c00700 10 client.26383 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac00c00700 1 -- 172.21.15.42:0/3973730349 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564f46962e90 con 0x564f46a4cf90 2024-05-27T10:28:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac00c00700 10 client.26383 handle_command_reply: tid=0 2024-05-27T10:28:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 client.26383 shutdown 2024-05-27T10:28:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a339f0 msgr2=0x564f46a3d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a339f0 0x564f46a3d160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac040dff40 tx=0x7fac040a94a0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 shutdown_connections 2024-05-27T10:28:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46a4cf90 0x564f46a4f410 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a33e00 0x564f46a3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a3cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/3973730349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a339f0 0x564f46a3d160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac00c00700 0 client.26383 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 >> 172.21.15.42:0/3973730349 conn(0x564f469140c0 msgr2=0x564f46a31e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 shutdown_connections 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.269+0000 7fac2bf45b80 1 -- 172.21.15.42:0/3973730349 wait complete. 2024-05-27T10:28:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.270+0000 7fac2bf45b80 20 client.26383 trim_cache size 0 max 16384 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 Processor -- start 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 -- start start 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f468b3f40 0x564f46a2dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a2e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 0x564f46a37da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac181b3550 con 0x564f46a33e00 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac181b0d50 con 0x564f46a339f0 2024-05-27T10:28:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.292+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fac181aeba0 con 0x564f468b3f40 2024-05-27T10:28:25.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.293+0000 7fac17200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 0x564f46a37da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.293+0000 7fac17200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 0x564f46a37da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33152/0 (socket says 172.21.15.42:33152) 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.293+0000 7fac17200700 1 -- 172.21.15.42:0/1721588936 learned_addr learned my addr 172.21.15.42:0/1721588936 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.293+0000 7fac15e00700 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a2e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.293+0000 7fac16800700 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f468b3f40 0x564f46a2dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac17200700 1 -- 172.21.15.42:0/1721588936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f468b3f40 msgr2=0x564f46a2dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac17200700 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f468b3f40 0x564f46a2dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac17200700 1 -- 172.21.15.42:0/1721588936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 msgr2=0x564f46a2e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac17200700 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a2e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac17200700 1 -- 172.21.15.42:0/1721588936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f46a3d430 con 0x564f46a33e00 2024-05-27T10:28:25.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac17200700 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 0x564f46a37da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fac10016910 tx=0x7fac10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33c5127ccfa04ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:25.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.294+0000 7fac03400700 1 -- 172.21.15.42:0/1721588936 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10009220 con 0x564f46a33e00 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.295+0000 7fac03400700 1 -- 172.21.15.42:0/1721588936 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac10023c10 con 0x564f46a33e00 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.295+0000 7fac03400700 1 -- 172.21.15.42:0/1721588936 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10008c40 con 0x564f46a33e00 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.295+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1721588936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 msgr2=0x564f46a37da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.295+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 0x564f46a37da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fac10016910 tx=0x7fac10017000 comp rx=0 tx=0).stop 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1721588936 shutdown_connections 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f46a33e00 0x564f46a37da0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f468b3f40 0x564f46a2dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1721588936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a2e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1721588936 >> 172.21.15.42:0/1721588936 conn(0x564f469140c0 msgr2=0x564f46a2d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:25.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1721588936 shutdown_connections 2024-05-27T10:28:25.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1721588936 wait complete. 2024-05-27T10:28:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.296+0000 7fac2bf45b80 1 -- start start 2024-05-27T10:28:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 Processor -- start 2024-05-27T10:28:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a41e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a42360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac181af210 con 0x564f468b3f40 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac181b0680 con 0x564f46a339f0 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fac181af8e0 con 0x564f46a33e00 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.297+0000 7fac2bf45b80 10 client.0 _ll_register_callbacks cb 0x564f46a49860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a41e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a41e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33164/0 (socket says 172.21.15.42:33164) 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 -- 172.21.15.42:0/1923410547 learned_addr learned my addr 172.21.15.42:0/1923410547 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac15e00700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a42360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 -- 172.21.15.42:0/1923410547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 msgr2=0x564f46a428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac17200700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 -- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 msgr2=0x564f46a42360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:25.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a42360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac16800700 1 -- 172.21.15.42:0/1923410547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f46ab7ab0 con 0x564f468b3f40 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac15e00700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a42360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.298+0000 7fac17200700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.299+0000 7fac16800700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a41e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fac08005b40 tx=0x7fac08009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b771737194e13caa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.299+0000 7fac00c00700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.299+0000 7fac00c00700 1 -- 172.21.15.42:0/1923410547 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08027040 con 0x564f468b3f40 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.299+0000 7fac00c00700 1 -- 172.21.15.42:0/1923410547 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fac080052b0 con 0x564f468b3f40 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.299+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac10023c10 con 0x564f468b3f40 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac00c00700 1 -- 172.21.15.42:0/1923410547 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fac080054b0 con 0x564f468b3f40 2024-05-27T10:28:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 10 client.16881 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564f46ab8c30 con 0x564f468b3f40 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac00c00700 1 -- 172.21.15.42:0/1923410547 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac10023c10 con 0x564f468b3f40 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac00c00700 1 -- 172.21.15.42:0/1923410547 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fac08021070 con 0x564f468b3f40 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 10 client.16881 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 20 client.16881 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46ab8e00 0x564f46abb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 4 client.16881 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a108fbe", "release": true, "await": true}] 2024-05-27T10:28:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.300+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a108fbe", "release": true, "await": true}) -- 0x564f4695c670 con 0x564f46ab8e00 2024-05-27T10:28:25.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.301+0000 7fac15e00700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46ab8e00 0x564f46abb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:25.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.302+0000 7fac15e00700 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46ab8e00 0x564f46abb220 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9289430598fb91f1 server_cookie=8758f2b1a3eadc2 in_seq=0 out_seq=0 2024-05-27T10:28:25.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.302+0000 7fac00c00700 10 client.16881 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:25.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:24 smithi143 ceph-mon[35545]: pgmap v300: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-27T10:28:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.334+0000 7fac00c00700 1 -- 172.21.15.42:0/1923410547 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564f4695c670 con 0x564f46ab8e00 2024-05-27T10:28:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.334+0000 7fac00c00700 10 client.16881 handle_command_reply: tid=0 2024-05-27T10:28:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.334+0000 7fac2bf45b80 1 client.16881 shutdown 2024-05-27T10:28:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.334+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 msgr2=0x564f46a41e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.334+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a41e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fac08005b40 tx=0x7fac08009690 comp rx=0 tx=0).stop 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 shutdown_connections 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564f46ab8e00 0x564f46abb220 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f46a33e00 0x564f46a428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f468b3f40 0x564f46a41e20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac00c00700 0 client.16881 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 --2- 172.21.15.42:0/1923410547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f46a339f0 0x564f46a42360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 >> 172.21.15.42:0/1923410547 conn(0x564f469140c0 msgr2=0x564f46a2e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 shutdown_connections 2024-05-27T10:28:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 1 -- 172.21.15.42:0/1923410547 wait complete. 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.335+0000 7fac2bf45b80 20 client.16881 trim_cache size 0 max 16384 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 36, 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 28, 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "6a108fbe": { 2024-05-27T10:28:25.335 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 28, 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 142.3, 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:25.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:25.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:28:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fabfc0b29f0 msgr2=0x7fabfc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fabfc0b29f0 0x7fabfc0b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x564f46a2ec70 tx=0x7fac1000b040 comp rx=0 tx=0).stop 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 msgr2=0x7fac1819d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 0x7fac1819d2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e970 tx=0x7fac0400ec80 comp rx=0 tx=0).stop 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 shutdown_connections 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fabfc0b29f0 0x7fabfc0b4f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac1810b890 0x7fac181a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac18106610 0x7fac1819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 --2- 172.21.15.42:0/3420547934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac1810af40 0x7fac1819d2f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 >> 172.21.15.42:0/3420547934 conn(0x7fac18102020 msgr2=0x7fac1810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 shutdown_connections 2024-05-27T10:28:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:25.337+0000 7fac1ca00700 1 -- 172.21.15.42:0/3420547934 wait complete. 2024-05-27T10:28:26.289 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a108fbe', total seconds elapsed: 30.9 2024-05-27T10:28:26.289 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.3 2024-05-27T10:28:26.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:28:26.759 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 -- 172.21.15.42:0/1565231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010f3c0 msgr2=0x7fd740116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 --2- 172.21.15.42:0/1565231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010f3c0 0x7fd740116b60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd738009ca0 tx=0x7fd73802a8c0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 -- 172.21.15.42:0/1565231986 shutdown_connections 2024-05-27T10:28:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 --2- 172.21.15.42:0/1565231986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010f3c0 0x7fd740116b60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 --2- 172.21.15.42:0/1565231986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 0x7fd74010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 --2- 172.21.15.42:0/1565231986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010a390 0x7fd74010a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.089+0000 7fd745e00640 1 -- 172.21.15.42:0/1565231986 >> 172.21.15.42:0/1565231986 conn(0x7fd740076070 msgr2=0x7fd740076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.090+0000 7fd745e00640 1 -- 172.21.15.42:0/1565231986 shutdown_connections 2024-05-27T10:28:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.090+0000 7fd745e00640 1 -- 172.21.15.42:0/1565231986 wait complete. 2024-05-27T10:28:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.091+0000 7fd745e00640 1 Processor -- start 2024-05-27T10:28:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.091+0000 7fd745e00640 1 -- start start 2024-05-27T10:28:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.091+0000 7fd745e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 0x7fd7401a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd745e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 0x7fd7401a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd745e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010f3c0 0x7fd7401a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd745e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd74011a1e0 con 0x7fd74010a390 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd745e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd74011a060 con 0x7fd74010f3c0 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd745e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd74011a360 con 0x7fd74010e900 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd744a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 0x7fd7401a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd744a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 0x7fd7401a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33192/0 (socket says 172.21.15.42:33192) 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd744a00640 1 -- 172.21.15.42:0/1283254107 learned_addr learned my addr 172.21.15.42:0/1283254107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 0x7fd7401a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd745400640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010f3c0 0x7fd7401a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd744a00640 1 -- 172.21.15.42:0/1283254107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 msgr2=0x7fd7401a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.092+0000 7fd744a00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 0x7fd7401a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd744a00640 1 -- 172.21.15.42:0/1283254107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010f3c0 msgr2=0x7fd7401a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd744a00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010f3c0 0x7fd7401a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd744a00640 1 -- 172.21.15.42:0/1283254107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7401a0fb0 con 0x7fd74010a390 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 0x7fd7401a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd745400640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010f3c0 0x7fd7401a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd744a00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 0x7fd7401a67b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd73400d770 tx=0x7fd73400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd73d600640 1 -- 172.21.15.42:0/1283254107 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734017070 con 0x7fd74010a390 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7401a12a0 con 0x7fd74010a390 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd73d600640 1 -- 172.21.15.42:0/1283254107 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd734009c30 con 0x7fd74010a390 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd73d600640 1 -- 172.21.15.42:0/1283254107 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400f620 con 0x7fd74010a390 2024-05-27T10:28:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.093+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7401ad630 con 0x7fd74010a390 2024-05-27T10:28:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.097+0000 7fd74e38c740 1 Processor -- start 2024-05-27T10:28:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.097+0000 7fd74e38c740 1 -- start start 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da10ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd74e38c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5636da07db90 con 0x5636d9d494d0 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd74e38c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636da06d730 con 0x5636d9d412f0 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd74e38c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5636da071770 con 0x5636d9d6a1b0 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd73d600640 1 -- 172.21.15.42:0/1283254107 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd73401a070 con 0x7fd74010a390 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd73fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd73fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33198/0 (socket says 172.21.15.42:33198) 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd73fe00640 1 -- 172.21.15.42:0/3454377657 learned_addr learned my addr 172.21.15.42:0/3454377657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.098+0000 7fd745400640 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da10ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd744a00640 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd73fe00640 1 -- 172.21.15.42:0/3454377657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 msgr2=0x5636da10ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd73fe00640 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da10ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd73fe00640 1 -- 172.21.15.42:0/3454377657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 msgr2=0x5636da10cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd73fe00640 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da10cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd73fe00640 1 -- 172.21.15.42:0/3454377657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636da0068b0 con 0x5636d9d494d0 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.099+0000 7fd73fe00640 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10d4e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd72c008b30 tx=0x7fd72c00e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53873b1fd859d387 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.100+0000 7fd731c00640 1 -- 172.21.15.42:0/3454377657 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c014070 con 0x5636d9d494d0 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.100+0000 7fd731c00640 1 -- 172.21.15.42:0/3454377657 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd72c0054b0 con 0x5636d9d494d0 2024-05-27T10:28:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.100+0000 7fd731c00640 1 -- 172.21.15.42:0/3454377657 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004e60 con 0x5636d9d494d0 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 -- 172.21.15.42:0/3454377657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 msgr2=0x5636da10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10d4e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd72c008b30 tx=0x7fd72c00e350 comp rx=0 tx=0).stop 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 -- 172.21.15.42:0/3454377657 shutdown_connections 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10d4e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 --2- 172.21.15.42:0/3454377657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da10ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd74e38c740 1 -- 172.21.15.42:0/3454377657 >> 172.21.15.42:0/3454377657 conn(0x5636d9d5e300 msgr2=0x5636da109210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.101+0000 7fd73d600640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7280b2950 0x7fd7280b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.102+0000 7fd73d600640 1 -- 172.21.15.42:0/1283254107 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd7340e2170 con 0x7fd74010a390 2024-05-27T10:28:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.102+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7280b2950 0x7fd7280b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.102+0000 7fd74e38c740 1 -- 172.21.15.42:0/3454377657 shutdown_connections 2024-05-27T10:28:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.102+0000 7fd74e38c740 1 -- 172.21.15.42:0/3454377657 wait complete. 2024-05-27T10:28:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.103+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7280b2950 0x7fd7280b4e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd72c004620 tx=0x7fd72c00ef90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.103+0000 7fd74e38c740 1 -- start start 2024-05-27T10:28:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.103+0000 7fd74e38c740 1 Processor -- start 2024-05-27T10:28:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d412f0 0x5636da116bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d494d0 0x5636da117130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 0x5636da122ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5636d9d53790 con 0x5636d9d6a1b0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636da015bc0 con 0x5636d9d494d0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5636da119b90 con 0x5636d9d412f0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd74e38c740 10 client.0 _ll_register_callbacks cb 0x5636d9ff3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd745400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 0x5636da122ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd745400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 0x5636da122ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33214/0 (socket says 172.21.15.42:33214) 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd745400640 1 -- 172.21.15.42:0/2505531764 learned_addr learned my addr 172.21.15.42:0/2505531764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd744a00640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d412f0 0x5636da116bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.104+0000 7fd73fe00640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d494d0 0x5636da117130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd745400640 1 -- 172.21.15.42:0/2505531764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d412f0 msgr2=0x5636da116bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd745400640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d412f0 0x5636da116bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd745400640 1 -- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d494d0 msgr2=0x5636da117130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd745400640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d494d0 0x5636da117130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd745400640 1 -- 172.21.15.42:0/2505531764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d9d82210 con 0x5636d9d6a1b0 2024-05-27T10:28:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd73fe00640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d494d0 0x5636da117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd745400640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 0x5636da122ec0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd738009ca0 tx=0x7fd738031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7935c9e50fde4fc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd726000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd726000640 1 -- 172.21.15.42:0/2505531764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738003970 con 0x5636d9d6a1b0 2024-05-27T10:28:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd726000640 1 -- 172.21.15.42:0/2505531764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd738003b10 con 0x5636d9d6a1b0 2024-05-27T10:28:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.105+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72c0054b0 con 0x5636d9d6a1b0 2024-05-27T10:28:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd726000640 1 -- 172.21.15.42:0/2505531764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd738003d10 con 0x5636d9d6a1b0 2024-05-27T10:28:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 10 client.16893 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5636d9e4e620 con 0x5636d9d6a1b0 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd726000640 1 -- 172.21.15.42:0/2505531764 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd72c0054b0 con 0x5636d9d6a1b0 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd726000640 1 -- 172.21.15.42:0/2505531764 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fd7380305c0 con 0x5636d9d6a1b0 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 10 client.16893 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 20 client.16893 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9d28d80 0x5636da128f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 4 client.16893 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.106+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636da07fae0 con 0x5636d9d28d80 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.107+0000 7fd744a00640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9d28d80 0x5636da128f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.108+0000 7fd744a00640 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9d28d80 0x5636da128f70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb97290073cd981 server_cookie=165464bea3f2f31f in_seq=0 out_seq=0 2024-05-27T10:28:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.108+0000 7fd726000640 10 client.16893 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd726000640 1 -- 172.21.15.42:0/2505531764 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636da07fae0 con 0x5636d9d28d80 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd726000640 10 client.16893 handle_command_reply: tid=0 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 client.16893 shutdown 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 msgr2=0x5636da122ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 0x5636da122ec0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd738009ca0 tx=0x7fd738031bb0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 shutdown_connections 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d6a1b0 0x5636da122ec0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d494d0 0x5636da117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9d28d80 0x5636da128f70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 --2- 172.21.15.42:0/2505531764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d412f0 0x5636da116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 >> 172.21.15.42:0/2505531764 conn(0x5636d9d5e300 msgr2=0x5636da10e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd726000640 0 client.16893 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.110+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 shutdown_connections 2024-05-27T10:28:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.111+0000 7fd74e38c740 1 -- 172.21.15.42:0/2505531764 wait complete. 2024-05-27T10:28:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.111+0000 7fd74e38c740 20 client.16893 trim_cache size 0 max 16384 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 Processor -- start 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 -- start start 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da1123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da122c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b8620 con 0x5636d9d494d0 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b5e60 con 0x5636d9d412f0 2024-05-27T10:28:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.129+0000 7fd74e38c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b3cb0 con 0x5636d9d6a1b0 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33218/0 (socket says 172.21.15.42:33218) 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 -- 172.21.15.42:0/1636956735 learned_addr learned my addr 172.21.15.42:0/1636956735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd744a00640 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da1123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd745400640 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 -- 172.21.15.42:0/1636956735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 msgr2=0x5636da122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 -- 172.21.15.42:0/1636956735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 msgr2=0x5636da1123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da1123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.130+0000 7fd73fe00640 1 -- 172.21.15.42:0/1636956735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636da10d8d0 con 0x5636d9d494d0 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.131+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10ca20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd72c000c00 tx=0x7fd72c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971734968ca09d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.131+0000 7fd731200640 1 -- 172.21.15.42:0/1636956735 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c010430 con 0x5636d9d494d0 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.131+0000 7fd731200640 1 -- 172.21.15.42:0/1636956735 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd72c034070 con 0x5636d9d494d0 2024-05-27T10:28:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.131+0000 7fd731200640 1 -- 172.21.15.42:0/1636956735 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c02f7c0 con 0x5636d9d494d0 2024-05-27T10:28:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 -- 172.21.15.42:0/1636956735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 msgr2=0x5636da10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10ca20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd72c000c00 tx=0x7fd72c022000 comp rx=0 tx=0).stop 2024-05-27T10:28:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 -- 172.21.15.42:0/1636956735 shutdown_connections 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d6a1b0 0x5636da122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d494d0 0x5636da10ca20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1636956735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d412f0 0x5636da1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 -- 172.21.15.42:0/1636956735 >> 172.21.15.42:0/1636956735 conn(0x5636d9d5e300 msgr2=0x5636da118d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 -- 172.21.15.42:0/1636956735 shutdown_connections 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.132+0000 7fd74e38c740 1 -- 172.21.15.42:0/1636956735 wait complete. 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.133+0000 7fd74e38c740 1 -- start start 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.133+0000 7fd74e38c740 1 Processor -- start 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 0x5636da119e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d494d0 0x5636da11a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d6a1b0 0x5636da18f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b4320 con 0x5636d9d412f0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b5790 con 0x5636d9d6a1b0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401b49f0 con 0x5636d9d494d0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd74e38c740 10 client.0 _ll_register_callbacks cb 0x7fd738009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 0x5636da119e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 0x5636da119e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33226/0 (socket says 172.21.15.42:33226) 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 -- 172.21.15.42:0/1316303120 learned_addr learned my addr 172.21.15.42:0/1316303120 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d494d0 0x5636da11a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd745400640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d6a1b0 0x5636da18f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 -- 172.21.15.42:0/1316303120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d494d0 msgr2=0x5636da11a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d494d0 0x5636da11a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 -- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d6a1b0 msgr2=0x5636da18f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d6a1b0 0x5636da18f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd744a00640 1 -- 172.21.15.42:0/1316303120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636da11ac40 con 0x5636d9d412f0 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.134+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d494d0 0x5636da11a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd744a00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 0x5636da119e80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd7340a6710 tx=0x7fd7340e2370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba40b2d4f26a361 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd726000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd726000640 1 -- 172.21.15.42:0/1316303120 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734021040 con 0x5636d9d412f0 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd726000640 1 -- 172.21.15.42:0/1316303120 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd7340b1e70 con 0x5636d9d412f0 2024-05-27T10:28:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72c034070 con 0x5636d9d412f0 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd726000640 1 -- 172.21.15.42:0/1316303120 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd734009de0 con 0x5636d9d412f0 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd74e38c740 10 client.16899 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.135+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5636da190b90 con 0x5636d9d412f0 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd726000640 1 -- 172.21.15.42:0/1316303120 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd72c034070 con 0x5636d9d412f0 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd726000640 1 -- 172.21.15.42:0/1316303120 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fd7340a3bc0 con 0x5636d9d412f0 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd74e38c740 10 client.16899 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd74e38c740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd74e38c740 20 client.16899 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9cfe1c0 0x5636da1932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd74e38c740 4 client.16899 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:28:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.136+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636da0792c0 con 0x5636d9cfe1c0 2024-05-27T10:28:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.137+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9cfe1c0 0x5636da1932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.138+0000 7fd73fe00640 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9cfe1c0 0x5636da1932e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f643452b9bdb0fd0 server_cookie=56a21042f92c5bd4 in_seq=0 out_seq=0 2024-05-27T10:28:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.138+0000 7fd726000640 10 client.16899 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd726000640 1 -- 172.21.15.42:0/1316303120 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5636da0792c0 con 0x5636d9cfe1c0 2024-05-27T10:28:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd726000640 10 client.16899 handle_command_reply: tid=0 2024-05-27T10:28:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 client.16899 shutdown 2024-05-27T10:28:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 msgr2=0x5636da119e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 0x5636da119e80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd7340a6710 tx=0x7fd7340e2370 comp rx=0 tx=0).stop 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 shutdown_connections 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636d9d6a1b0 0x5636da18f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5636d9cfe1c0 0x5636da1932e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5636d9d494d0 0x5636da11a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 --2- 172.21.15.42:0/1316303120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636d9d412f0 0x5636da119e80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd726000640 0 client.16899 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 >> 172.21.15.42:0/1316303120 conn(0x5636d9d5e300 msgr2=0x5636da116450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 shutdown_connections 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.165+0000 7fd74e38c740 1 -- 172.21.15.42:0/1316303120 wait complete. 2024-05-27T10:28:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.166+0000 7fd74e38c740 20 client.16899 trim_cache size 0 max 16384 2024-05-27T10:28:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:28:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:28:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.166+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7280b2950 msgr2=0x7fd7280b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.166+0000 7fd745e00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7280b2950 0x7fd7280b4e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd72c004620 tx=0x7fd72c00ef90 comp rx=0 tx=0).stop 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.166+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 msgr2=0x7fd7401a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 0x7fd7401a67b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd73400d770 tx=0x7fd73400dc30 comp rx=0 tx=0).stop 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 shutdown_connections 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7280b2950 0x7fd7280b4e60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd74010f3c0 0x7fd7401a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74010e900 0x7fd7401a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 --2- 172.21.15.42:0/1283254107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd74010a390 0x7fd7401a67b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 >> 172.21.15.42:0/1283254107 conn(0x7fd740076070 msgr2=0x7fd740107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 shutdown_connections 2024-05-27T10:28:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:27.167+0000 7fd745e00640 1 -- 172.21.15.42:0/1283254107 wait complete. 2024-05-27T10:28:27.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:26 smithi042 ceph-mon[31661]: pgmap v301: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-05-27T10:28:27.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:26 smithi143 ceph-mon[35545]: pgmap v301: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-05-27T10:28:27.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:26 smithi104 ceph-mon[34812]: pgmap v301: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-05-27T10:28:29.227 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:28 smithi042 ceph-mon[31661]: pgmap v302: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-05-27T10:28:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 -- 172.21.15.42:0/969404812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 msgr2=0x7f6d4c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 --2- 172.21.15.42:0/969404812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c105900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009990 tx=0x7f6d3c009ca0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 -- 172.21.15.42:0/969404812 shutdown_connections 2024-05-27T10:28:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 --2- 172.21.15.42:0/969404812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c10e8d0 0x7f6d4c110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 --2- 172.21.15.42:0/969404812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c105900 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 --2- 172.21.15.42:0/969404812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d4c105e40 0x7f6d4c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 -- 172.21.15.42:0/969404812 >> 172.21.15.42:0/969404812 conn(0x7f6d4c0fedb0 msgr2=0x7f6d4c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:29.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.227+0000 7f6d51e00700 1 -- 172.21.15.42:0/969404812 shutdown_connections 2024-05-27T10:28:29.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.228+0000 7f6d51e00700 1 -- 172.21.15.42:0/969404812 wait complete. 2024-05-27T10:28:29.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.229+0000 7f6d51e00700 1 Processor -- start 2024-05-27T10:28:29.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 -- start start 2024-05-27T10:28:29.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:29.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c105e40 0x7f6d4c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:29.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d4c10e8d0 0x7f6d4c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:29.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c114350 con 0x7f6d4c1054f0 2024-05-27T10:28:29.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1141d0 con 0x7f6d4c105e40 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.230+0000 7f6d51e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1144d0 con 0x7f6d4c10e8d0 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4a000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c105e40 0x7f6d4c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33248/0 (socket says 172.21.15.42:33248) 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4aa00700 1 -- 172.21.15.42:0/3335193067 learned_addr learned my addr 172.21.15.42:0/3335193067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4b400700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d4c10e8d0 0x7f6d4c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4a000700 1 -- 172.21.15.42:0/3335193067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d4c10e8d0 msgr2=0x7f6d4c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4a000700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d4c10e8d0 0x7f6d4c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4a000700 1 -- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 msgr2=0x7f6d4c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4a000700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.231+0000 7f6d4a000700 1 -- 172.21.15.42:0/3335193067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c19fc50 con 0x7f6d4c105e40 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.232+0000 7f6d4a000700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c105e40 0x7f6d4c19b320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b450 tx=0x7f6d3800b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.233+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38015020 con 0x7f6d4c105e40 2024-05-27T10:28:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.233+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d38011cc0 con 0x7f6d4c105e40 2024-05-27T10:28:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.233+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38010be0 con 0x7f6d4c105e40 2024-05-27T10:28:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.233+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c19ff40 con 0x7f6d4c105e40 2024-05-27T10:28:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.233+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c1a7860 con 0x7f6d4c105e40 2024-05-27T10:28:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.235+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c0512b0 con 0x7f6d4c105e40 2024-05-27T10:28:29.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.240+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6d38047d10 con 0x7f6d4c105e40 2024-05-27T10:28:29.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.241+0000 7f6d50a00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6d300b2b40 0x7f6d300b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:29.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.241+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6d380e20b0 con 0x7f6d4c105e40 2024-05-27T10:28:29.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.242+0000 7f6d4aa00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6d300b2b40 0x7f6d300b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.243+0000 7f6d4aa00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6d300b2b40 0x7f6d300b5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009750 tx=0x7f6d3c005960 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.244+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d3801f020 con 0x7f6d4c105e40 2024-05-27T10:28:29.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:28 smithi143 ceph-mon[35545]: pgmap v302: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-05-27T10:28:29.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:28 smithi104 ceph-mon[34812]: pgmap v302: 97 pgs: 97 active+clean; 389 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-05-27T10:28:29.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.547+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d4c19c0e0 con 0x7f6d4c105e40 2024-05-27T10:28:29.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.550+0000 7f6d50a00700 1 -- 172.21.15.42:0/3335193067 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11879 (secure 0 0 0) 0x7f6d380a9c10 con 0x7f6d4c105e40 2024-05-27T10:28:29.549 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:28:29.549 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:28:14.904306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_36359":{"gid":36359,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/755631209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":755631209},{"type":"v1","addr":"172.21.15.104:6833","nonce":755631209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:28:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6d300b2b40 msgr2=0x7f6d300b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6d300b2b40 0x7f6d300b5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009750 tx=0x7f6d3c005960 comp rx=0 tx=0).stop 2024-05-27T10:28:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c105e40 msgr2=0x7f6d4c19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c105e40 0x7f6d4c19b320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b450 tx=0x7f6d3800b760 comp rx=0 tx=0).stop 2024-05-27T10:28:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 shutdown_connections 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f6d300b2b40 0x7f6d300b5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d4c10e8d0 0x7f6d4c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d4c1054f0 0x7f6d4c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 --2- 172.21.15.42:0/3335193067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d4c105e40 0x7f6d4c19b320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 >> 172.21.15.42:0/3335193067 conn(0x7f6d4c0fedb0 msgr2=0x7f6d4c0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.554+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 shutdown_connections 2024-05-27T10:28:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:29.555+0000 7f6d51e00700 1 -- 172.21.15.42:0/3335193067 wait complete. 2024-05-27T10:28:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-05-27T10:28:30.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:29 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3335193067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:30.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:29 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3335193067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:30.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:29 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3335193067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:30.499 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db / --timeout 90 --expiration 142.3 --await-for 45.0' 2024-05-27T10:28:30.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db / --timeout 90 --expiration 142.3 --await-for 45.0 2024-05-27T10:28:30.971 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:31.307 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:30 smithi143 ceph-mon[35545]: pgmap v303: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 259 KiB/s wr, 25 op/s 2024-05-27T10:28:31.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:30 smithi042 ceph-mon[31661]: pgmap v303: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 259 KiB/s wr, 25 op/s 2024-05-27T10:28:31.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:30 smithi104 ceph-mon[34812]: pgmap v303: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 259 KiB/s wr, 25 op/s 2024-05-27T10:28:33.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:32 smithi042 ceph-mon[31661]: pgmap v304: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-05-27T10:28:33.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:32 smithi143 ceph-mon[35545]: pgmap v304: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-05-27T10:28:33.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:32 smithi104 ceph-mon[34812]: pgmap v304: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 -- 172.21.15.42:0/2705849974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 msgr2=0x7f20041111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 --2- 172.21.15.42:0/2705849974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f20041111c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00b0d0 tx=0x7f1ffc00b3e0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 -- 172.21.15.42:0/2705849974 shutdown_connections 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 --2- 172.21.15.42:0/2705849974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f20041111c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 --2- 172.21.15.42:0/2705849974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20041045e0 0x7f20041049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 --2- 172.21.15.42:0/2705849974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004108f10 0x7f2004109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 -- 172.21.15.42:0/2705849974 >> 172.21.15.42:0/2705849974 conn(0x7f2004100030 msgr2=0x7f2004102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.541+0000 7f200b000700 1 -- 172.21.15.42:0/2705849974 shutdown_connections 2024-05-27T10:28:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.542+0000 7f200b000700 1 -- 172.21.15.42:0/2705849974 wait complete. 2024-05-27T10:28:33.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.543+0000 7f200b000700 1 Processor -- start 2024-05-27T10:28:33.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.543+0000 7f200b000700 1 -- start start 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.543+0000 7f200b000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20041045e0 0x7f2004075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f200b000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004108f10 0x7f2004075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f200b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f2004076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f200b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2004114340 con 0x7f2004109860 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f200b000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20041141c0 con 0x7f20041045e0 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f200b000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20041144c0 con 0x7f2004108f10 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f2009000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f2004076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f2003e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20041045e0 0x7f2004075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f2009000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f2004076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36746/0 (socket says 172.21.15.42:36746) 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f2009000700 1 -- 172.21.15.42:0/4041693831 learned_addr learned my addr 172.21.15.42:0/4041693831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.544+0000 7f2003400700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004108f10 0x7f2004075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2009000700 1 -- 172.21.15.42:0/4041693831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004108f10 msgr2=0x7f2004075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:33.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2009000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004108f10 0x7f2004075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2009000700 1 -- 172.21.15.42:0/4041693831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20041045e0 msgr2=0x7f2004075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2009000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20041045e0 0x7f2004075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2009000700 1 -- 172.21.15.42:0/4041693831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041a9070 con 0x7f2004109860 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2009000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f2004076530 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009850 tx=0x7f1ffc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2000c00700 1 -- 172.21.15.42:0/4041693831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc01f070 con 0x7f2004109860 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2000c00700 1 -- 172.21.15.42:0/4041693831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ffc004c50 con 0x7f2004109860 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.545+0000 7f2000c00700 1 -- 172.21.15.42:0/4041693831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc01b690 con 0x7f2004109860 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.546+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20041a9300 con 0x7f2004109860 2024-05-27T10:28:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.546+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a9700 con 0x7f2004109860 2024-05-27T10:28:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.550+0000 7f2000c00700 1 -- 172.21.15.42:0/4041693831 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ffc022070 con 0x7f2004109860 2024-05-27T10:28:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.551+0000 7f2019acbb80 1 Processor -- start 2024-05-27T10:28:33.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.551+0000 7f2019acbb80 1 -- start start 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2000c00700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1fe80b2a70 0x7f1fe80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036ae2fd0 0x565036c67e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036c68380 0x565036c61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 0x565036c628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2019acbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565036b8ffd0 con 0x565036c68380 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2019acbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565036b7fb70 con 0x565036c624d0 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2019acbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x565036b83bb0 con 0x565036ae2fd0 2024-05-27T10:28:33.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.552+0000 7f2000c00700 1 -- 172.21.15.42:0/4041693831 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ffc00f020 con 0x7f2004109860 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2003400700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1fe80b2a70 0x7f1fe80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 0x565036c628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 0x565036c628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:36134/0 (socket says 172.21.15.42:36134) 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 -- 172.21.15.42:0/2628193796 learned_addr learned my addr 172.21.15.42:0/2628193796 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2003e00700 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036ae2fd0 0x565036c67e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2003400700 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036c68380 0x565036c61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 -- 172.21.15.42:0/2628193796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036ae2fd0 msgr2=0x565036c67e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036ae2fd0 0x565036c67e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 -- 172.21.15.42:0/2628193796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036c68380 msgr2=0x565036c61f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036c68380 0x565036c61f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2009000700 1 -- 172.21.15.42:0/2628193796 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565036b485b0 con 0x565036c624d0 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.553+0000 7f2003400700 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036c68380 0x565036c61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.554+0000 7f2009000700 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 0x565036c628e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0a6a10 tx=0x7f1ffc0aa1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397483e93b2f5c3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.554+0000 7f2003400700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1fe80b2a70 0x7f1fe80b4f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00076a0 tx=0x7f1ff0007630 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.554+0000 7f1ff9600700 1 -- 172.21.15.42:0/2628193796 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00d130 con 0x565036c624d0 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.554+0000 7f1ff9600700 1 -- 172.21.15.42:0/2628193796 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ffc0aa4a0 con 0x565036c624d0 2024-05-27T10:28:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.554+0000 7f1ff9600700 1 -- 172.21.15.42:0/2628193796 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0aa990 con 0x565036c624d0 2024-05-27T10:28:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 -- 172.21.15.42:0/2628193796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 msgr2=0x565036c628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 0x565036c628e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0a6a10 tx=0x7f1ffc0aa1b0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 -- 172.21.15.42:0/2628193796 shutdown_connections 2024-05-27T10:28:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c624d0 0x565036c628e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036ae2fd0 0x565036c67e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 --2- 172.21.15.42:0/2628193796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036c68380 0x565036c61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 -- 172.21.15.42:0/2628193796 >> 172.21.15.42:0/2628193796 conn(0x565036b43150 msgr2=0x565036c5d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 -- 172.21.15.42:0/2628193796 shutdown_connections 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.555+0000 7f2019acbb80 1 -- 172.21.15.42:0/2628193796 wait complete. 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.556+0000 7f2019acbb80 1 -- start start 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.556+0000 7f2019acbb80 1 Processor -- start 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565036a86130 con 0x565036ae2fd0 2024-05-27T10:28:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565036b208f0 con 0x565036c68380 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x565036b7fb70 con 0x565036c624d0 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2019acbb80 10 client.0 _ll_register_callbacks cb 0x565036b29e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2003e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2003e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c6bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36772/0 (socket says 172.21.15.42:36772) 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2003e00700 1 -- 172.21.15.42:0/28320516 learned_addr learned my addr 172.21.15.42:0/28320516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2003400700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.557+0000 7f2009000700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f2003e00700 1 -- 172.21.15.42:0/28320516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 msgr2=0x565036c6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f2003e00700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f2003e00700 1 -- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 msgr2=0x565036c6c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f2003e00700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c6c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f2003e00700 1 -- 172.21.15.42:0/28320516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565036acfa60 con 0x565036ae2fd0 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f2003e00700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c6bb80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40127f0 tx=0x7f1ff4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db473b8ed614d5c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.558+0000 7f1fee000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f1fee000700 1 -- 172.21.15.42:0/28320516 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400f4e0 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ffc0aa4a0 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f1fee000700 1 -- 172.21.15.42:0/28320516 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ff400f680 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f1fee000700 1 -- 172.21.15.42:0/28320516 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1ff400f880 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f2019acbb80 10 client.16914 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565036b501f0 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.559+0000 7f1fee000700 1 -- 172.21.15.42:0/28320516 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ffc0aa4a0 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f1fee000700 1 -- 172.21.15.42:0/28320516 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f1ff40109a0 con 0x565036ae2fd0 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2019acbb80 10 client.16914 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2019acbb80 10 client.16914 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2019acbb80 20 client.16914 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2019acbb80 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036c7b970 0x565036c7de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2019acbb80 4 client.16914 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:33.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565036b91f20 con 0x565036c7b970 2024-05-27T10:28:33.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.560+0000 7f2003400700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036c7b970 0x565036c7de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.561+0000 7f2003400700 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036c7b970 0x565036c7de20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25a7732b927ffd2 server_cookie=b74a54e784f1ca9e in_seq=0 out_seq=0 2024-05-27T10:28:33.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.561+0000 7f1fee000700 10 client.16914 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:33.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.564+0000 7f1fee000700 1 -- 172.21.15.42:0/28320516 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565036b91f20 con 0x565036c7b970 2024-05-27T10:28:33.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.564+0000 7f1fee000700 10 client.16914 handle_command_reply: tid=0 2024-05-27T10:28:33.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 client.16914 shutdown 2024-05-27T10:28:33.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 msgr2=0x565036c6bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:33.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c6bb80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40127f0 tx=0x7f1ff4012b00 comp rx=0 tx=0).stop 2024-05-27T10:28:33.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 shutdown_connections 2024-05-27T10:28:33.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036c7b970 0x565036c7de20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c6c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c6bb80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 --2- 172.21.15.42:0/28320516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f1fee000700 0 client.16914 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 >> 172.21.15.42:0/28320516 conn(0x565036b43150 msgr2=0x565036c60da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 shutdown_connections 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 1 -- 172.21.15.42:0/28320516 wait complete. 2024-05-27T10:28:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.565+0000 7f2019acbb80 20 client.16914 trim_cache size 0 max 16384 2024-05-27T10:28:33.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.589+0000 7f2019acbb80 1 Processor -- start 2024-05-27T10:28:33.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.589+0000 7f2019acbb80 1 -- start start 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c77af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c65ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2019acbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b37f0 con 0x565036ae2fd0 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2019acbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b0ff0 con 0x565036c68380 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2019acbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20041aee60 con 0x565036c624d0 2024-05-27T10:28:33.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2003e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c77af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2003e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c77af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36782/0 (socket says 172.21.15.42:36782) 2024-05-27T10:28:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.590+0000 7f2003e00700 1 -- 172.21.15.42:0/1801487489 learned_addr learned my addr 172.21.15.42:0/1801487489 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2009000700 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003400700 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c65ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003e00700 1 -- 172.21.15.42:0/1801487489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 msgr2=0x565036c65ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003e00700 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c65ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003e00700 1 -- 172.21.15.42:0/1801487489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 msgr2=0x565036c835e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003e00700 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c835e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003e00700 1 -- 172.21.15.42:0/1801487489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565036c6c970 con 0x565036ae2fd0 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.591+0000 7f2003e00700 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c77af0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401c6b0 tx=0x7f1ff401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55206f88b474b685 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.592+0000 7f1ff8c00700 1 -- 172.21.15.42:0/1801487489 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4029070 con 0x565036ae2fd0 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.592+0000 7f1ff8c00700 1 -- 172.21.15.42:0/1801487489 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ff400f5a0 con 0x565036ae2fd0 2024-05-27T10:28:33.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.592+0000 7f1ff8c00700 1 -- 172.21.15.42:0/1801487489 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40050b0 con 0x565036ae2fd0 2024-05-27T10:28:33.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 -- 172.21.15.42:0/1801487489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 msgr2=0x565036c77af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:33.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c77af0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401c6b0 tx=0x7f1ff401c6e0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 -- 172.21.15.42:0/1801487489 shutdown_connections 2024-05-27T10:28:33.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c835e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c77af0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1801487489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c65ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 -- 172.21.15.42:0/1801487489 >> 172.21.15.42:0/1801487489 conn(0x565036b43150 msgr2=0x565036c65790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:33.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 -- 172.21.15.42:0/1801487489 shutdown_connections 2024-05-27T10:28:33.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.593+0000 7f2019acbb80 1 -- 172.21.15.42:0/1801487489 wait complete. 2024-05-27T10:28:33.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.594+0000 7f2019acbb80 1 -- start start 2024-05-27T10:28:33.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.594+0000 7f2019acbb80 1 Processor -- start 2024-05-27T10:28:33.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.594+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.594+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c70cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2019acbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2019acbb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20041af4b0 con 0x565036ae2fd0 2024-05-27T10:28:33.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2019acbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b0920 con 0x565036c68380 2024-05-27T10:28:33.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2019acbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20041afb80 con 0x565036c624d0 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2019acbb80 10 client.0 _ll_register_callbacks cb 0x565036b29e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2003e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2003e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36788/0 (socket says 172.21.15.42:36788) 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2003e00700 1 -- 172.21.15.42:0/1463684300 learned_addr learned my addr 172.21.15.42:0/1463684300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2003400700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c70cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.595+0000 7f2009000700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2003e00700 1 -- 172.21.15.42:0/1463684300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 msgr2=0x565036c70cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2003e00700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c70cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2003e00700 1 -- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 msgr2=0x565036c711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2003e00700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2003e00700 1 -- 172.21.15.42:0/1463684300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565036c71730 con 0x565036ae2fd0 2024-05-27T10:28:33.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2009000700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2003e00700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c740a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4000c00 tx=0x7f1ff401dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1cf4ec45b494e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f1fee000700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f1fee000700 1 -- 172.21.15.42:0/1463684300 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4029070 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f1fee000700 1 -- 172.21.15.42:0/1463684300 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ff400fc30 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.596+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff400f5a0 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.597+0000 7f1fee000700 1 -- 172.21.15.42:0/1463684300 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1ff403d050 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.597+0000 7f2019acbb80 10 client.16920 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.597+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565036ce7040 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.597+0000 7f1fee000700 1 -- 172.21.15.42:0/1463684300 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ff400f5a0 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.597+0000 7f1fee000700 1 -- 172.21.15.42:0/1463684300 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f1ff4010430 con 0x565036ae2fd0 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.597+0000 7f2019acbb80 10 client.16920 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.598+0000 7f2019acbb80 10 client.16920 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.598+0000 7f2019acbb80 20 client.16920 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.598+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036ce7750 0x565036ce9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.598+0000 7f2019acbb80 4 client.16920 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.3, "await_for": 45.0}] 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.598+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.3, "await_for": 45.0}) -- 0x565036b8b700 con 0x565036ce7750 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.598+0000 7f2003400700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036ce7750 0x565036ce9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.599+0000 7f2003400700 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036ce7750 0x565036ce9b70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64eafd2f6b7dec58 server_cookie=dde7791217e0eaf5 in_seq=0 out_seq=0 2024-05-27T10:28:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:33.599+0000 7f1fee000700 10 client.16920 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:35.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:34 smithi143 ceph-mon[35545]: pgmap v305: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-05-27T10:28:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:34 smithi042 ceph-mon[31661]: pgmap v305: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-05-27T10:28:35.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:34 smithi104 ceph-mon[34812]: pgmap v305: 97 pgs: 97 active+clean; 392 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-05-27T10:28:36.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:36 smithi143 ceph-mon[35545]: pgmap v306: 97 pgs: 97 active+clean; 380 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 663 KiB/s wr, 84 op/s 2024-05-27T10:28:36.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:36 smithi042 ceph-mon[31661]: pgmap v306: 97 pgs: 97 active+clean; 380 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 663 KiB/s wr, 84 op/s 2024-05-27T10:28:36.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:36 smithi104 ceph-mon[34812]: pgmap v306: 97 pgs: 97 active+clean; 380 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 663 KiB/s wr, 84 op/s 2024-05-27T10:28:37.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.711+0000 7f1fee000700 1 -- 172.21.15.42:0/1463684300 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565036b8b700 con 0x565036ce7750 2024-05-27T10:28:37.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.711+0000 7f1fee000700 10 client.16920 handle_command_reply: tid=0 2024-05-27T10:28:37.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.711+0000 7f2019acbb80 1 client.16920 shutdown 2024-05-27T10:28:37.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 msgr2=0x565036c740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:37.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c740a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4000c00 tx=0x7f1ff401dde0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 shutdown_connections 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x565036ce7750 0x565036ce9b70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565036c68380 0x565036c711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x565036ae2fd0 0x565036c740a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 --2- 172.21.15.42:0/1463684300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x565036c624d0 0x565036c70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f1fee000700 0 client.16920 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.712+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 >> 172.21.15.42:0/1463684300 conn(0x565036b43150 msgr2=0x565036c6b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.713+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 shutdown_connections 2024-05-27T10:28:37.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.714+0000 7f2019acbb80 1 -- 172.21.15.42:0/1463684300 wait complete. 2024-05-27T10:28:37.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.714+0000 7f2019acbb80 20 client.16920 trim_cache size 0 max 16384 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 36, 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 30, 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "14aec5ce": { 2024-05-27T10:28:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 30, 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 142.3, 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "QUIESCED", 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1fe80b2a70 msgr2=0x7f1fe80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:37.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1fe80b2a70 0x7f1fe80b4f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00076a0 tx=0x7f1ff0007630 comp rx=0 tx=0).stop 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 msgr2=0x7f2004076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f2004076530 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009850 tx=0x7f1ffc007a20 comp rx=0 tx=0).stop 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 shutdown_connections 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1fe80b2a70 0x7f1fe80b4f80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2004109860 0x7f2004076530 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20041045e0 0x7f2004075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 --2- 172.21.15.42:0/4041693831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004108f10 0x7f2004075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.716+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 >> 172.21.15.42:0/4041693831 conn(0x7f2004100030 msgr2=0x7f200410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.717+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 shutdown_connections 2024-05-27T10:28:37.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:37.717+0000 7f200b000700 1 -- 172.21.15.42:0/4041693831 wait complete. 2024-05-27T10:28:38.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:38 smithi104 ceph-mon[34812]: pgmap v307: 97 pgs: 97 active+clean; 380 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 660 KiB/s wr, 81 op/s 2024-05-27T10:28:38.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:38 smithi143 ceph-mon[35545]: pgmap v307: 97 pgs: 97 active+clean; 380 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 660 KiB/s wr, 81 op/s 2024-05-27T10:28:38.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:38 smithi042 ceph-mon[31661]: pgmap v307: 97 pgs: 97 active+clean; 380 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 660 KiB/s wr, 81 op/s 2024-05-27T10:28:38.700 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '14aec5ce', quiesce took 12.4 seconds. Will release after: 9.9 2024-05-27T10:28:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.395+0000 7fb524400640 1 -- 172.21.15.42:0/1494005184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c109c70 msgr2=0x7fb51c103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.395+0000 7fb524400640 1 --2- 172.21.15.42:0/1494005184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c109c70 0x7fb51c103810 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb510009ca0 tx=0x7fb51002a8c0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 -- 172.21.15.42:0/1494005184 shutdown_connections 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 --2- 172.21.15.42:0/1494005184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb51c1043c0 0x7fb51c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 --2- 172.21.15.42:0/1494005184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c10a110 0x7fb51c103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 --2- 172.21.15.42:0/1494005184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c109c70 0x7fb51c103810 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 -- 172.21.15.42:0/1494005184 >> 172.21.15.42:0/1494005184 conn(0x7fb51c0ff350 msgr2=0x7fb51c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 -- 172.21.15.42:0/1494005184 shutdown_connections 2024-05-27T10:28:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.396+0000 7fb524400640 1 -- 172.21.15.42:0/1494005184 wait complete. 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 Processor -- start 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 -- start start 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 0x7fb51c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c109c70 0x7fb51c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb51c10a110 0x7fb51c1a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c113d60 con 0x7fb51c1043c0 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c113be0 con 0x7fb51c109c70 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.397+0000 7fb524400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c113ee0 con 0x7fb51c10a110 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 0x7fb51c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 0x7fb51c19d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36808/0 (socket says 172.21.15.42:36808) 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 -- 172.21.15.42:0/178864067 learned_addr learned my addr 172.21.15.42:0/178864067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521000640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c109c70 0x7fb51c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb522400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb51c10a110 0x7fb51c1a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 -- 172.21.15.42:0/178864067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb51c10a110 msgr2=0x7fb51c1a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb51c10a110 0x7fb51c1a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 -- 172.21.15.42:0/178864067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c109c70 msgr2=0x7fb51c19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c109c70 0x7fb51c19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521a00640 1 -- 172.21.15.42:0/178864067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c1a8be0 con 0x7fb51c1043c0 2024-05-27T10:28:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.398+0000 7fb521000640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c109c70 0x7fb51c19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.399+0000 7fb521a00640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 0x7fb51c19d800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb510000f80 tx=0x7fb510032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.399+0000 7fb51a000640 1 -- 172.21.15.42:0/178864067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510032e00 con 0x7fb51c1043c0 2024-05-27T10:28:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.399+0000 7fb51a000640 1 -- 172.21.15.42:0/178864067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5100044d0 con 0x7fb51c1043c0 2024-05-27T10:28:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.399+0000 7fb51a000640 1 -- 172.21.15.42:0/178864067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510030b80 con 0x7fb51c1043c0 2024-05-27T10:28:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.399+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c1a8e70 con 0x7fb51c1043c0 2024-05-27T10:28:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.399+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c1a9280 con 0x7fb51c1043c0 2024-05-27T10:28:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 Processor -- start 2024-05-27T10:28:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 -- start start 2024-05-27T10:28:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d471b0 0x55b5b40e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40ea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b5b405ab90 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b5b404a730 con 0x55b5b3d264d0 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.403+0000 7fb52b47d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b5b404e770 con 0x55b5b3d471b0 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb51a000640 1 -- 172.21.15.42:0/178864067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb510004680 con 0x7fb51c1043c0 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36816/0 (socket says 172.21.15.42:36816) 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb522400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d471b0 0x55b5b40e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb521a00640 1 -- 172.21.15.42:0/2872444522 learned_addr learned my addr 172.21.15.42:0/2872444522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb521000640 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40ea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb521a00640 1 -- 172.21.15.42:0/2872444522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d471b0 msgr2=0x55b5b40e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.404+0000 7fb521a00640 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d471b0 0x55b5b40e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb521a00640 1 -- 172.21.15.42:0/2872444522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 msgr2=0x55b5b40ea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb521a00640 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40ea2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb521a00640 1 -- 172.21.15.42:0/2872444522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b3fe38b0 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb521a00640 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40e9da0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb510101270 tx=0x7fb510107b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85667d2401e28bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb50ea00640 1 -- 172.21.15.42:0/2872444522 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51010a440 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb50ea00640 1 -- 172.21.15.42:0/2872444522 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb51010aa70 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.405+0000 7fb50ea00640 1 -- 172.21.15.42:0/2872444522 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510115800 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.406+0000 7fb51a000640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5040b2950 0x7fb5040b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.406+0000 7fb51a000640 1 -- 172.21.15.42:0/178864067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb510103ae0 con 0x7fb51c1043c0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.406+0000 7fb522400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5040b2950 0x7fb5040b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.406+0000 7fb52b47d740 1 -- 172.21.15.42:0/2872444522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 msgr2=0x55b5b40e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.406+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40e9da0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb510101270 tx=0x7fb510107b00 comp rx=0 tx=0).stop 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 -- 172.21.15.42:0/2872444522 shutdown_connections 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40ea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40e9da0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2872444522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d471b0 0x55b5b40e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 -- 172.21.15.42:0/2872444522 >> 172.21.15.42:0/2872444522 conn(0x55b5b3d3b300 msgr2=0x55b5b40e6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 -- 172.21.15.42:0/2872444522 shutdown_connections 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb522400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5040b2950 0x7fb5040b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55b5b40e69d0 tx=0x7fb51400d480 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 -- 172.21.15.42:0/2872444522 wait complete. 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.407+0000 7fb52b47d740 1 -- start start 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 Processor -- start 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 0x55b5b40ff880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b5b3d30790 con 0x55b5b3d471b0 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b5b3ff2bc0 con 0x55b5b3d264d0 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b5b405ab90 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb52b47d740 10 client.0 _ll_register_callbacks cb 0x55b5b3fd0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.408+0000 7fb522400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 0x55b5b40ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 0x55b5b40ff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36832/0 (socket says 172.21.15.42:36832) 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 -- 172.21.15.42:0/2194637172 learned_addr learned my addr 172.21.15.42:0/2194637172 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb521000640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb521a00640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 -- 172.21.15.42:0/2194637172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d1e2f0 msgr2=0x55b5b40f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 -- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 msgr2=0x55b5b40f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 -- 172.21.15.42:0/2194637172 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b3d5f210 con 0x55b5b3d471b0 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb522400640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 0x55b5b40ff880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb514010750 tx=0x7fb5140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01ce7a992e964ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.409+0000 7fb503400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb503400640 1 -- 172.21.15.42:0/2194637172 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400fad0 con 0x55b5b3d471b0 2024-05-27T10:28:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51010aa70 con 0x55b5b3d471b0 2024-05-27T10:28:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb503400640 1 -- 172.21.15.42:0/2194637172 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb51400fc70 con 0x55b5b3d471b0 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb503400640 1 -- 172.21.15.42:0/2194637172 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb51400fe70 con 0x55b5b3d471b0 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb52b47d740 10 client.16932 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b5b3e2b620 con 0x55b5b3d471b0 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.410+0000 7fb503400640 1 -- 172.21.15.42:0/2194637172 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb51010aa70 con 0x55b5b3d471b0 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb503400640 1 -- 172.21.15.42:0/2194637172 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fb514027400 con 0x55b5b3d471b0 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb52b47d740 10 client.16932 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb52b47d740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb52b47d740 20 client.16932 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3d05d80 0x55b5b4105960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb52b47d740 4 client.16932 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5b405cae0 con 0x55b5b3d05d80 2024-05-27T10:28:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.411+0000 7fb521a00640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3d05d80 0x55b5b4105960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.412+0000 7fb521a00640 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3d05d80 0x55b5b4105960 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d868d34e3295d72 server_cookie=e609f64d97ee9900 in_seq=0 out_seq=0 2024-05-27T10:28:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.412+0000 7fb503400640 10 client.16932 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.414+0000 7fb503400640 1 -- 172.21.15.42:0/2194637172 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b5b405cae0 con 0x55b5b3d05d80 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.414+0000 7fb503400640 10 client.16932 handle_command_reply: tid=0 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 client.16932 shutdown 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 msgr2=0x55b5b40ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 0x55b5b40ff880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb514010750 tx=0x7fb5140046c0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 shutdown_connections 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d471b0 0x55b5b40ff880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d264d0 0x55b5b40f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3d05d80 0x55b5b4105960 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 --2- 172.21.15.42:0/2194637172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 >> 172.21.15.42:0/2194637172 conn(0x55b5b3d3b300 msgr2=0x55b5b40e9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb503400640 0 client.16932 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 shutdown_connections 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 1 -- 172.21.15.42:0/2194637172 wait complete. 2024-05-27T10:28:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.415+0000 7fb52b47d740 20 client.16932 trim_cache size 0 max 16384 2024-05-27T10:28:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.432+0000 7fb52b47d740 1 Processor -- start 2024-05-27T10:28:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.432+0000 7fb52b47d740 1 -- start start 2024-05-27T10:28:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b40fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb52b47d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1b4520 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb52b47d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1b1d60 con 0x55b5b3d471b0 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb52b47d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1afbb0 con 0x55b5b3d264d0 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36834/0 (socket says 172.21.15.42:36834) 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb521a00640 1 -- 172.21.15.42:0/1872137692 learned_addr learned my addr 172.21.15.42:0/1872137692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.433+0000 7fb521000640 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb522400640 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b40fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521a00640 1 -- 172.21.15.42:0/1872137692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 msgr2=0x55b5b40ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521a00640 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521a00640 1 -- 172.21.15.42:0/1872137692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 msgr2=0x55b5b40fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521a00640 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b40fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521a00640 1 -- 172.21.15.42:0/1872137692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b40e6c00 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521000640 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb521a00640 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40ff630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb5100cabc0 tx=0x7fb510107840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5314f963bfb0a56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb50e000640 1 -- 172.21.15.42:0/1872137692 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100ed040 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb50e000640 1 -- 172.21.15.42:0/1872137692 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb510036040 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.434+0000 7fb50e000640 1 -- 172.21.15.42:0/1872137692 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100098b0 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 -- 172.21.15.42:0/1872137692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 msgr2=0x55b5b40ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40ff630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb5100cabc0 tx=0x7fb510107840 comp rx=0 tx=0).stop 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 -- 172.21.15.42:0/1872137692 shutdown_connections 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b40fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 --2- 172.21.15.42:0/1872137692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40ff630 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 -- 172.21.15.42:0/1872137692 >> 172.21.15.42:0/1872137692 conn(0x55b5b3d3b300 msgr2=0x55b5b40ee230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 -- 172.21.15.42:0/1872137692 shutdown_connections 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.435+0000 7fb52b47d740 1 -- 172.21.15.42:0/1872137692 wait complete. 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.436+0000 7fb52b47d740 1 -- start start 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.436+0000 7fb52b47d740 1 Processor -- start 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40fa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b416c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1b0220 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1b1690 con 0x55b5b3d471b0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1b08f0 con 0x55b5b3d264d0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb52b47d740 10 client.0 _ll_register_callbacks cb 0x7fb51400d980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40fa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36846/0 (socket says 172.21.15.42:36846) 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 -- 172.21.15.42:0/861296984 learned_addr learned my addr 172.21.15.42:0/861296984 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521000640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb522400640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b416c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 -- 172.21.15.42:0/861296984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 msgr2=0x55b5b40fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40fada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 -- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 msgr2=0x55b5b416c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b416c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521a00640 1 -- 172.21.15.42:0/861296984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b40fb5b0 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.437+0000 7fb521000640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb521a00640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40fa860 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb510005e30 tx=0x7fb510107790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e02de8f44d321f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb503400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb503400640 1 -- 172.21.15.42:0/861296984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100ed040 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb503400640 1 -- 172.21.15.42:0/861296984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb5100c74b0 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510036040 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb503400640 1 -- 172.21.15.42:0/861296984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb5100cac90 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb52b47d740 10 client.16938 fetch_fsmap learned FSMap version 36 2024-05-27T10:28:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.438+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b5b416d510 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb503400640 1 -- 172.21.15.42:0/861296984 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb510036040 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb503400640 1 -- 172.21.15.42:0/861296984 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fb5100cb120 con 0x55b5b3d1e2f0 2024-05-27T10:28:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb52b47d740 10 client.16938 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb52b47d740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb52b47d740 20 client.16938 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb52b47d740 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3cdb1c0 0x55b5b416fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb52b47d740 4 client.16938 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.439+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5b40562c0 con 0x55b5b3cdb1c0 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.440+0000 7fb521000640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3cdb1c0 0x55b5b416fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.440+0000 7fb521000640 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3cdb1c0 0x55b5b416fc10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8dd88b14d31e91 server_cookie=131821fb5e5dda70 in_seq=0 out_seq=0 2024-05-27T10:28:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.441+0000 7fb503400640 10 client.16938 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:40.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:40 smithi104 ceph-mon[34812]: pgmap v308: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 816 KiB/s wr, 105 op/s 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.465+0000 7fb503400640 1 -- 172.21.15.42:0/861296984 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b5b40562c0 con 0x55b5b3cdb1c0 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.465+0000 7fb503400640 10 client.16938 handle_command_reply: tid=0 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.465+0000 7fb52b47d740 1 client.16938 shutdown 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 msgr2=0x55b5b40fa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40fa860 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb510005e30 tx=0x7fb510107790 comp rx=0 tx=0).stop 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 shutdown_connections 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b5b3d471b0 0x55b5b416c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55b5b3cdb1c0 0x55b5b416fc10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5b3d264d0 0x55b5b40fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 --2- 172.21.15.42:0/861296984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5b3d1e2f0 0x55b5b40fa860 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb503400640 0 client.16938 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 >> 172.21.15.42:0/861296984 conn(0x55b5b3d3b300 msgr2=0x55b5b40eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 shutdown_connections 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 1 -- 172.21.15.42:0/861296984 wait complete. 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.466+0000 7fb52b47d740 20 client.16938 trim_cache size 0 max 16384 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:28:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:28:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.467+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5040b2950 msgr2=0x7fb5040b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.467+0000 7fb524400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5040b2950 0x7fb5040b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55b5b40e69d0 tx=0x7fb51400d480 comp rx=0 tx=0).stop 2024-05-27T10:28:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.467+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 msgr2=0x7fb51c19d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.467+0000 7fb524400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 0x7fb51c19d800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb510000f80 tx=0x7fb510032a10 comp rx=0 tx=0).stop 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 shutdown_connections 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb5040b2950 0x7fb5040b4e60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb51c10a110 0x7fb51c1a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb51c109c70 0x7fb51c19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 --2- 172.21.15.42:0/178864067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb51c1043c0 0x7fb51c19d800 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 >> 172.21.15.42:0/178864067 conn(0x7fb51c0ff350 msgr2=0x7fb51c076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 shutdown_connections 2024-05-27T10:28:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:40.468+0000 7fb524400640 1 -- 172.21.15.42:0/178864067 wait complete. 2024-05-27T10:28:40.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:40 smithi143 ceph-mon[35545]: pgmap v308: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 816 KiB/s wr, 105 op/s 2024-05-27T10:28:40.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:40 smithi042 ceph-mon[31661]: pgmap v308: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 816 KiB/s wr, 105 op/s 2024-05-27T10:28:42.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:42 smithi143 ceph-mon[35545]: pgmap v309: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 560 KiB/s wr, 85 op/s 2024-05-27T10:28:42.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:42 smithi042 ceph-mon[31661]: pgmap v309: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 560 KiB/s wr, 85 op/s 2024-05-27T10:28:42.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:42 smithi104 ceph-mon[34812]: pgmap v309: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 560 KiB/s wr, 85 op/s 2024-05-27T10:28:44.499 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:44 smithi104 ceph-mon[34812]: pgmap v310: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 560 KiB/s wr, 85 op/s 2024-05-27T10:28:44.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:44 smithi143 ceph-mon[35545]: pgmap v310: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 560 KiB/s wr, 85 op/s 2024-05-27T10:28:44.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:44 smithi042 ceph-mon[31661]: pgmap v310: 97 pgs: 97 active+clean; 382 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 560 KiB/s wr, 85 op/s 2024-05-27T10:28:46.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:46 smithi143 ceph-mon[35545]: pgmap v311: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 582 KiB/s wr, 105 op/s 2024-05-27T10:28:46.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:46 smithi042 ceph-mon[31661]: pgmap v311: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 582 KiB/s wr, 105 op/s 2024-05-27T10:28:46.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:46 smithi104 ceph-mon[34812]: pgmap v311: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 582 KiB/s wr, 105 op/s 2024-05-27T10:28:47.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:47 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:47.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:47 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:47.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:47 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:48.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:48 smithi143 ceph-mon[35545]: pgmap v312: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-05-27T10:28:48.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:48 smithi042 ceph-mon[31661]: pgmap v312: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-05-27T10:28:48.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:28:48.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:48 smithi104 ceph-mon[34812]: pgmap v312: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-05-27T10:28:49.063 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:50.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:50 smithi143 ceph-mon[35545]: pgmap v313: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 178 KiB/s wr, 46 op/s 2024-05-27T10:28:50.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:50 smithi042 ceph-mon[31661]: pgmap v313: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 178 KiB/s wr, 46 op/s 2024-05-27T10:28:50.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:50 smithi104 ceph-mon[34812]: pgmap v313: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 178 KiB/s wr, 46 op/s 2024-05-27T10:28:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 -- 172.21.15.42:0/3000149170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 msgr2=0x7f19a4109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 --2- 172.21.15.42:0/3000149170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a4109300 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f198c0099c0 tx=0x7f198c009cd0 comp rx=0 tx=0).stop 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 -- 172.21.15.42:0/3000149170 shutdown_connections 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 --2- 172.21.15.42:0/3000149170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19a4109840 0x7f19a41111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 --2- 172.21.15.42:0/3000149170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a41045c0 0x7f19a41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 --2- 172.21.15.42:0/3000149170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a4109300 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 -- 172.21.15.42:0/3000149170 >> 172.21.15.42:0/3000149170 conn(0x7f19a40ffe90 msgr2=0x7f19a41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 -- 172.21.15.42:0/3000149170 shutdown_connections 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.740+0000 7f19aa800700 1 -- 172.21.15.42:0/3000149170 wait complete. 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.741+0000 7f19aa800700 1 Processor -- start 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 -- start start 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a41045c0 0x7f19a419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19a4109840 0x7f19a41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4114350 con 0x7f19a4108ef0 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41141d0 con 0x7f19a4109840 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f19aa800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41144d0 con 0x7f19a41045c0 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.742+0000 7f199be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49976/0 (socket says 172.21.15.42:49976) 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 -- 172.21.15.42:0/4001136854 learned_addr learned my addr 172.21.15.42:0/4001136854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f19a3400700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a41045c0 0x7f19a419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f19a3e00700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19a4109840 0x7f19a41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 -- 172.21.15.42:0/4001136854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a41045c0 msgr2=0x7f19a419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a41045c0 0x7f19a419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 -- 172.21.15.42:0/4001136854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19a4109840 msgr2=0x7f19a41a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19a4109840 0x7f19a41a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 -- 172.21.15.42:0/4001136854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a41a1eb0 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.743+0000 7f199be00700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a419d580 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f198c0098e0 tx=0x7f198c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.744+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c027070 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.744+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a41a2140 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.744+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f198c00fdd0 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.744+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c022e30 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.744+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a41a9ac0 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.745+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a4105a70 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.748+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f198c02a070 con 0x7f19a4108ef0 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.749+0000 7f19a0c00700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f19880b2a70 0x7f19880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.749+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f198c0e36d0 con 0x7f19a4108ef0 2024-05-27T10:28:51.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.749+0000 7f19a3400700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f19880b2a70 0x7f19880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:51.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.750+0000 7f19a3400700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f19880b2a70 0x7f19880b4f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1994005950 tx=0x7f19940058e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:51.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:51.752+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f198c013070 con 0x7f19a4108ef0 2024-05-27T10:28:52.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.037+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19a419e340 con 0x7f19a4108ef0 2024-05-27T10:28:52.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.038+0000 7f19a0c00700 1 -- 172.21.15.42:0/4001136854 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11877 (secure 0 0 0) 0x7f198c0ab130 con 0x7f19a4108ef0 2024-05-27T10:28:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:28:52.037 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:28:46.065695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_36359":{"gid":36359,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/755631209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":755631209},{"type":"v1","addr":"172.21.15.104:6833","nonce":755631209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34466,34499,34442]},"id":1}]} 2024-05-27T10:28:52.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f19880b2a70 msgr2=0x7f19880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:52.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f19880b2a70 0x7f19880b4f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1994005950 tx=0x7f19940058e0 comp rx=0 tx=0).stop 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 msgr2=0x7f19a419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a419d580 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f198c0098e0 tx=0x7f198c00f510 comp rx=0 tx=0).stop 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 shutdown_connections 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f19880b2a70 0x7f19880b4f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19a4109840 0x7f19a41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a41045c0 0x7f19a419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 --2- 172.21.15.42:0/4001136854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a4108ef0 0x7f19a419d580 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:52.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.043+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 >> 172.21.15.42:0/4001136854 conn(0x7f19a40ffe90 msgr2=0x7f19a410a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.044+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 shutdown_connections 2024-05-27T10:28:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:52.044+0000 7f19aa800700 1 -- 172.21.15.42:0/4001136854 wait complete. 2024-05-27T10:28:52.044 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 37 2024-05-27T10:28:52.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:52 smithi042 ceph-mon[31661]: pgmap v314: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:28:52.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:52 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/4001136854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:52.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:52 smithi143 ceph-mon[35545]: pgmap v314: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:28:52.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:52 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/4001136854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:52.698 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:52 smithi104 ceph-mon[34812]: pgmap v314: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:28:52.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:52 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/4001136854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:28:52.969 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --set-id 14aec5ce --release --await' 2024-05-27T10:28:52.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --set-id 14aec5ce --release --await 2024-05-27T10:28:53.447 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 -- 172.21.15.42:0/4217963835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c104d80 msgr2=0x7f295c105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 --2- 172.21.15.42:0/4217963835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c104d80 0x7f295c105190 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f294c009900 tx=0x7f294c02a400 comp rx=0 tx=0).stop 2024-05-27T10:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 -- 172.21.15.42:0/4217963835 shutdown_connections 2024-05-27T10:28:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 --2- 172.21.15.42:0/4217963835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10e1d0 0x7f295c1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 --2- 172.21.15.42:0/4217963835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f295c10b7e0 0x7f295c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 --2- 172.21.15.42:0/4217963835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c104d80 0x7f295c105190 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.691+0000 7f2962a00640 1 -- 172.21.15.42:0/4217963835 >> 172.21.15.42:0/4217963835 conn(0x7f295c0fe380 msgr2=0x7f295c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.692+0000 7f2962a00640 1 -- 172.21.15.42:0/4217963835 shutdown_connections 2024-05-27T10:28:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.692+0000 7f2962a00640 1 -- 172.21.15.42:0/4217963835 wait complete. 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 Processor -- start 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 -- start start 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f295c104d80 0x7f295c1a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10b7e0 0x7f295c1a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 0x7f295c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f295c113ce0 con 0x7f295c10e1d0 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f295c113b60 con 0x7f295c10b7e0 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.693+0000 7f2962a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f295c113e60 con 0x7f295c104d80 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 0x7f295c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 0x7f295c19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43758/0 (socket says 172.21.15.42:43758) 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 -- 172.21.15.42:0/1041334373 learned_addr learned my addr 172.21.15.42:0/1041334373 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f295be00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f295c104d80 0x7f295c1a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f295b400640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10b7e0 0x7f295c1a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 -- 172.21.15.42:0/1041334373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f295c104d80 msgr2=0x7f295c1a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f295c104d80 0x7f295c1a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 -- 172.21.15.42:0/1041334373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10b7e0 msgr2=0x7f295c1a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10b7e0 0x7f295c1a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 -- 172.21.15.42:0/1041334373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c19cda0 con 0x7f295c10e1d0 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f295b400640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10b7e0 0x7f295c1a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.694+0000 7f2960a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 0x7f295c19c760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f295400cce0 tx=0x7f295400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.695+0000 7f2958c00640 1 -- 172.21.15.42:0/1041334373 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295400f450 con 0x7f295c10e1d0 2024-05-27T10:28:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.695+0000 7f2958c00640 1 -- 172.21.15.42:0/1041334373 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f295400fa80 con 0x7f295c10e1d0 2024-05-27T10:28:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.695+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295c19d090 con 0x7f295c10e1d0 2024-05-27T10:28:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.695+0000 7f2958c00640 1 -- 172.21.15.42:0/1041334373 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954018ad0 con 0x7f295c10e1d0 2024-05-27T10:28:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.695+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295c19d590 con 0x7f295c10e1d0 2024-05-27T10:28:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.699+0000 7f2969a0d740 1 Processor -- start 2024-05-27T10:28:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.700+0000 7f2958c00640 1 -- 172.21.15.42:0/1041334373 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f295400f5f0 con 0x7f295c10e1d0 2024-05-27T10:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.700+0000 7f2969a0d740 1 -- start start 2024-05-27T10:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f2969a0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611e7043b90 con 0x5611e6d072f0 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f2969a0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611e7033730 con 0x5611e6d0f4d0 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f2969a0d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5611e7037770 con 0x5611e6d301b0 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f295be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f295be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70d2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43768/0 (socket says 172.21.15.42:43768) 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f295be00640 1 -- 172.21.15.42:0/3767714855 learned_addr learned my addr 172.21.15.42:0/3767714855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f295b400640 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.701+0000 7f295b400640 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295be00640 1 -- 172.21.15.42:0/3767714855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 msgr2=0x5611e70d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295be00640 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295be00640 1 -- 172.21.15.42:0/3767714855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 msgr2=0x5611e70d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295be00640 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295be00640 1 -- 172.21.15.42:0/3767714855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e6fcc8b0 con 0x5611e6d072f0 2024-05-27T10:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295b400640 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f295be00640 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70d2da0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f294c02a920 tx=0x7f294c031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2c1e6d825333ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f2958c00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f29400b2950 0x7f29400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.702+0000 7f2958c00640 1 -- 172.21.15.42:0/1041334373 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f29540e2e00 con 0x7f295c10e1d0 2024-05-27T10:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.703+0000 7f2951200640 1 -- 172.21.15.42:0/3767714855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c035030 con 0x5611e6d072f0 2024-05-27T10:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.703+0000 7f295b400640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f29400b2950 0x7f29400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.703+0000 7f2951200640 1 -- 172.21.15.42:0/3767714855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f294c0044c0 con 0x5611e6d072f0 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.703+0000 7f2951200640 1 -- 172.21.15.42:0/3767714855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c003b40 con 0x5611e6d072f0 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 -- 172.21.15.42:0/3767714855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 msgr2=0x5611e70d2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f295b400640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f29400b2950 0x7f29400b4e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5611e70cf9d0 tx=0x7f2948005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70d2da0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f294c02a920 tx=0x7f294c031dc0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 -- 172.21.15.42:0/3767714855 shutdown_connections 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70d2da0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3767714855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 -- 172.21.15.42:0/3767714855 >> 172.21.15.42:0/3767714855 conn(0x5611e6d24300 msgr2=0x5611e70cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 -- 172.21.15.42:0/3767714855 shutdown_connections 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 -- 172.21.15.42:0/3767714855 wait complete. 2024-05-27T10:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.704+0000 7f2969a0d740 1 -- start start 2024-05-27T10:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 Processor -- start 2024-05-27T10:28:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 0x5611e70dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d301b0 0x5611e70e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611e6d19790 con 0x5611e6d0f4d0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611e6fdbbc0 con 0x5611e6d301b0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.705+0000 7f2969a0d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5611e70df930 con 0x5611e6d072f0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f2969a0d740 10 client.0 _ll_register_callbacks cb 0x5611e6fb9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 0x5611e70dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 0x5611e70dcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43778/0 (socket says 172.21.15.42:43778) 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 -- 172.21.15.42:0/1373593577 learned_addr learned my addr 172.21.15.42:0/1373593577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295be00640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f2960a00640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d301b0 0x5611e70e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 -- 172.21.15.42:0/1373593577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 msgr2=0x5611e70dca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70dca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 -- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d301b0 msgr2=0x5611e70e8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d301b0 0x5611e70e8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f295b400640 1 -- 172.21.15.42:0/1373593577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e6d48210 con 0x5611e6d0f4d0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.706+0000 7f2960a00640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d301b0 0x5611e70e8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f295b400640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 0x5611e70dcfb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2948010f50 tx=0x7f2948005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a361f7705c0c300 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2945600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2945600640 1 -- 172.21.15.42:0/1373593577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948019070 con 0x5611e6d0f4d0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611e7045ae0 con 0x5611e6d0f4d0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2945600640 1 -- 172.21.15.42:0/1373593577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f294802a680 con 0x5611e6d0f4d0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2945600640 1 -- 172.21.15.42:0/1373593577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f294802a880 con 0x5611e6d0f4d0 2024-05-27T10:28:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2969a0d740 10 client.16956 fetch_fsmap learned FSMap version 37 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.707+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5611e6e14620 con 0x5611e6d0f4d0 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2945600640 1 -- 172.21.15.42:0/1373593577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5611e7045ae0 con 0x5611e6d0f4d0 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2945600640 1 -- 172.21.15.42:0/1373593577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2948034ad0 con 0x5611e6d0f4d0 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2969a0d740 10 client.16956 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2969a0d740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2969a0d740 20 client.16956 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2969a0d740 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6ceed80 0x5611e70eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2969a0d740 4 client.16956 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.708+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e703f2c0 con 0x5611e6ceed80 2024-05-27T10:28:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.709+0000 7f295be00640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6ceed80 0x5611e70eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.710+0000 7f295be00640 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6ceed80 0x5611e70eecc0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0be6c9d08ad85c server_cookie=bfdb9e5cd7882fdf in_seq=0 out_seq=0 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.710+0000 7f2945600640 10 client.16956 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2945600640 1 -- 172.21.15.42:0/1373593577 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5611e703f2c0 con 0x5611e6ceed80 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2945600640 10 client.16956 handle_command_reply: tid=0 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 client.16956 shutdown 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 msgr2=0x5611e70dcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 0x5611e70dcfb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2948010f50 tx=0x7f2948005da0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 shutdown_connections 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d301b0 0x5611e70e8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d0f4d0 0x5611e70dcfb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6ceed80 0x5611e70eecc0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 --2- 172.21.15.42:0/1373593577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 >> 172.21.15.42:0/1373593577 conn(0x5611e6d24300 msgr2=0x5611e70d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2945600640 0 client.16956 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 shutdown_connections 2024-05-27T10:28:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 1 -- 172.21.15.42:0/1373593577 wait complete. 2024-05-27T10:28:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.712+0000 7f2969a0d740 20 client.16956 trim_cache size 0 max 16384 2024-05-27T10:28:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.729+0000 7f2969a0d740 1 Processor -- start 2024-05-27T10:28:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.729+0000 7f2969a0d740 1 -- start start 2024-05-27T10:28:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 0x5611e70f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2969a0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1b4300 con 0x5611e6d301b0 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2969a0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1b1b70 con 0x5611e6d0f4d0 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2969a0d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1af9c0 con 0x5611e6d072f0 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2960a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 0x5611e70f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2960a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 0x5611e70f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43794/0 (socket says 172.21.15.42:43794) 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f2960a00640 1 -- 172.21.15.42:0/3346870147 learned_addr learned my addr 172.21.15.42:0/3346870147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f295be00640 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.730+0000 7f295b400640 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2960a00640 1 -- 172.21.15.42:0/3346870147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 msgr2=0x5611e70e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2960a00640 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70e89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2960a00640 1 -- 172.21.15.42:0/3346870147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 msgr2=0x5611e70e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2960a00640 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2960a00640 1 -- 172.21.15.42:0/3346870147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e70e9450 con 0x5611e6d301b0 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f295b400640 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2960a00640 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 0x5611e70f44d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f29540aa060 tx=0x7f29540a9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c37ce44f6afb003 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2947e00640 1 -- 172.21.15.42:0/3346870147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29540aa0d0 con 0x5611e6d301b0 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2947e00640 1 -- 172.21.15.42:0/3346870147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29540c4070 con 0x5611e6d301b0 2024-05-27T10:28:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.731+0000 7f2947e00640 1 -- 172.21.15.42:0/3346870147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29540a64d0 con 0x5611e6d301b0 2024-05-27T10:28:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 -- 172.21.15.42:0/3346870147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 msgr2=0x5611e70f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 0x5611e70f44d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f29540aa060 tx=0x7f29540a9970 comp rx=0 tx=0).stop 2024-05-27T10:28:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 -- 172.21.15.42:0/3346870147 shutdown_connections 2024-05-27T10:28:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d301b0 0x5611e70f44d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3346870147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d072f0 0x5611e70e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 -- 172.21.15.42:0/3346870147 >> 172.21.15.42:0/3346870147 conn(0x5611e6d24300 msgr2=0x5611e70dcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 -- 172.21.15.42:0/3346870147 shutdown_connections 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.732+0000 7f2969a0d740 1 -- 172.21.15.42:0/3346870147 wait complete. 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 -- start start 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 Processor -- start 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1b0030 con 0x5611e6d072f0 2024-05-27T10:28:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1b14a0 con 0x5611e6d0f4d0 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.733+0000 7f2969a0d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1b0700 con 0x5611e6d301b0 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f2969a0d740 10 client.0 _ll_register_callbacks cb 0x7f294800d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70e4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43806/0 (socket says 172.21.15.42:43806) 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 -- 172.21.15.42:0/3750476976 learned_addr learned my addr 172.21.15.42:0/3750476976 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295b400640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f2960a00640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 -- 172.21.15.42:0/3750476976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 msgr2=0x5611e70e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70e2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 -- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 msgr2=0x5611e70e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.734+0000 7f295be00640 1 -- 172.21.15.42:0/3750476976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e70e2d40 con 0x5611e6d072f0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f295be00640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70e4f50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f294c005940 tx=0x7f294c005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6527656c66a442 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2945600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2945600640 1 -- 172.21.15.42:0/3750476976 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c035040 con 0x5611e6d072f0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2945600640 1 -- 172.21.15.42:0/3750476976 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f294c050480 con 0x5611e6d072f0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29540c4070 con 0x5611e6d072f0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2945600640 1 -- 172.21.15.42:0/3750476976 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f294c060050 con 0x5611e6d072f0 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2969a0d740 10 client.16962 fetch_fsmap learned FSMap version 37 2024-05-27T10:28:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.735+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5611e7156e90 con 0x5611e6d072f0 2024-05-27T10:28:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2945600640 1 -- 172.21.15.42:0/3750476976 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29540c4070 con 0x5611e6d072f0 2024-05-27T10:28:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2945600640 1 -- 172.21.15.42:0/3750476976 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f294c04d070 con 0x5611e6d072f0 2024-05-27T10:28:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2969a0d740 10 client.16962 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T10:28:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2969a0d740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2969a0d740 20 client.16962 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6cc41c0 0x5611e7159200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2969a0d740 4 client.16962 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.736+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611e6fb94b0 con 0x5611e6cc41c0 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.737+0000 7f295b400640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6cc41c0 0x5611e7159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.737+0000 7f295b400640 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6cc41c0 0x5611e7159200 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2138c60127bc64aa server_cookie=d7e1720cc48b0cb0 in_seq=0 out_seq=0 2024-05-27T10:28:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.737+0000 7f2945600640 10 client.16962 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.762+0000 7f2945600640 1 -- 172.21.15.42:0/3750476976 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5611e6fb94b0 con 0x5611e6cc41c0 2024-05-27T10:28:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.762+0000 7f2945600640 10 client.16962 handle_command_reply: tid=0 2024-05-27T10:28:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.762+0000 7f2969a0d740 1 client.16962 shutdown 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 msgr2=0x5611e70e4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70e4f50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f294c005940 tx=0x7f294c005e10 comp rx=0 tx=0).stop 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 shutdown_connections 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5611e6cc41c0 0x5611e7159200 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5611e6d301b0 0x5611e70e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611e6d0f4d0 0x5611e70e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 --2- 172.21.15.42:0/3750476976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611e6d072f0 0x5611e70e4f50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2945600640 0 client.16962 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 >> 172.21.15.42:0/3750476976 conn(0x5611e6d24300 msgr2=0x5611e70de120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 shutdown_connections 2024-05-27T10:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 1 -- 172.21.15.42:0/3750476976 wait complete. 2024-05-27T10:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.763+0000 7f2969a0d740 20 client.16962 trim_cache size 0 max 16384 2024-05-27T10:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:28:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.764+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f29400b2950 msgr2=0x7f29400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.764+0000 7f2962a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f29400b2950 0x7f29400b4e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5611e70cf9d0 tx=0x7f2948005f90 comp rx=0 tx=0).stop 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.764+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 msgr2=0x7f295c19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.764+0000 7f2962a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 0x7f295c19c760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f295400cce0 tx=0x7f295400a620 comp rx=0 tx=0).stop 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.764+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 shutdown_connections 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.764+0000 7f2962a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f29400b2950 0x7f29400b4e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.765+0000 7f2962a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f295c10e1d0 0x7f295c19c760 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.765+0000 7f2962a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f295c10b7e0 0x7f295c1a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.765+0000 7f2962a00640 1 --2- 172.21.15.42:0/1041334373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f295c104d80 0x7f295c1a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.765+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 >> 172.21.15.42:0/1041334373 conn(0x7f295c0fe380 msgr2=0x7f295c109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.765+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 shutdown_connections 2024-05-27T10:28:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:28:53.765+0000 7f2962a00640 1 -- 172.21.15.42:0/1041334373 wait complete. 2024-05-27T10:28:54.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:54 smithi143 ceph-mon[35545]: pgmap v315: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:28:54.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:54 smithi042 ceph-mon[31661]: pgmap v315: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:28:54.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:54 smithi104 ceph-mon[34812]: pgmap v315: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.011+0000 7f190de00700 1 -- 172.21.15.42:0/2171650138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107df0 msgr2=0x7f1908108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.011+0000 7f190de00700 1 --2- 172.21.15.42:0/2171650138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107df0 0x7f1908108200 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f18f00099c0 tx=0x7f18f0009cd0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 -- 172.21.15.42:0/2171650138 shutdown_connections 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 --2- 172.21.15.42:0/2171650138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908108740 0x7f19081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 --2- 172.21.15.42:0/2171650138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19081034c0 0x7f19081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 --2- 172.21.15.42:0/2171650138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107df0 0x7f1908108200 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 -- 172.21.15.42:0/2171650138 >> 172.21.15.42:0/2171650138 conn(0x7f19080fef10 msgr2=0x7f1908101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:56.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 -- 172.21.15.42:0/2171650138 shutdown_connections 2024-05-27T10:28:56.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.012+0000 7f190de00700 1 -- 172.21.15.42:0/2171650138 wait complete. 2024-05-27T10:28:56.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.013+0000 7f190de00700 1 Processor -- start 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.013+0000 7f190de00700 1 -- start start 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f190de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 0x7f190819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f190de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1908107df0 0x7f190819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f190de00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908108740 0x7f19081a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f190de00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1908113250 con 0x7f19081034c0 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f190de00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19081130d0 con 0x7f1908108740 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f190de00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19081133d0 con 0x7f1908107df0 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f1906a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 0x7f190819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f1906a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 0x7f190819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43818/0 (socket says 172.21.15.42:43818) 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f1906a00700 1 -- 172.21.15.42:0/1518631134 learned_addr learned my addr 172.21.15.42:0/1518631134 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.014+0000 7f18fe000700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1908107df0 0x7f190819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1907400700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908108740 0x7f19081a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1906a00700 1 -- 172.21.15.42:0/1518631134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1908107df0 msgr2=0x7f190819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1906a00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1908107df0 0x7f190819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1906a00700 1 -- 172.21.15.42:0/1518631134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908108740 msgr2=0x7f19081a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1906a00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908108740 0x7f19081a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1906a00700 1 -- 172.21.15.42:0/1518631134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19081a1ea0 con 0x7f19081034c0 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f18fe000700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1908107df0 0x7f190819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:28:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f1906a00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 0x7f190819cfd0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d950 tx=0x7f18f800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.015+0000 7f190ca00700 1 -- 172.21.15.42:0/1518631134 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8016070 con 0x7f19081034c0 2024-05-27T10:28:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.016+0000 7f190ca00700 1 -- 172.21.15.42:0/1518631134 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f8010460 con 0x7f19081034c0 2024-05-27T10:28:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.016+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19081a2190 con 0x7f19081034c0 2024-05-27T10:28:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.016+0000 7f190ca00700 1 -- 172.21.15.42:0/1518631134 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f800f620 con 0x7f19081034c0 2024-05-27T10:28:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.016+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19081a9ab0 con 0x7f19081034c0 2024-05-27T10:28:56.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.019+0000 7f190ca00700 1 -- 172.21.15.42:0/1518631134 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f18f8019070 con 0x7f19081034c0 2024-05-27T10:28:56.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.021+0000 7f191c9a6b80 1 Processor -- start 2024-05-27T10:28:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.021+0000 7f191c9a6b80 1 -- start start 2024-05-27T10:28:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.021+0000 7f190ca00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f18ec0b2a70 0x7f18ec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f190ca00700 1 -- 172.21.15.42:0/1518631134 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f18f80e2520 con 0x7f19081034c0 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 0x55a4da5f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a4da51bf40 con 0x55a4da46ef40 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a4da50bae0 con 0x55a4da55aaf0 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a4da50fb20 con 0x55a4da5ee5e0 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f1907400700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f18ec0b2a70 0x7f18ec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f18fe000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 0x55a4da5f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f18fe000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 0x55a4da5f4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43824/0 (socket says 172.21.15.42:43824) 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f18fe000700 1 -- 172.21.15.42:0/927262611 learned_addr learned my addr 172.21.15.42:0/927262611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.022+0000 7f1906a00700 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fe000700 1 -- 172.21.15.42:0/927262611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 msgr2=0x55a4da5ee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fe000700 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fe000700 1 -- 172.21.15.42:0/927262611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 msgr2=0x55a4da5f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fe000700 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fe000700 1 -- 172.21.15.42:0/927262611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4da4d4520 con 0x55a4da46ef40 2024-05-27T10:28:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f1907400700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f18ec0b2a70 0x7f18ec0b4f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1900006100 tx=0x7f1900008780 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fe000700 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 0x55a4da5f4a90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1908076aa0 tx=0x7f18f00051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd628e02ad2c2a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fcc00700 1 -- 172.21.15.42:0/927262611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0026070 con 0x55a4da46ef40 2024-05-27T10:28:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fcc00700 1 -- 172.21.15.42:0/927262611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f0009590 con 0x55a4da46ef40 2024-05-27T10:28:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.023+0000 7f18fcc00700 1 -- 172.21.15.42:0/927262611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0003970 con 0x55a4da46ef40 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 -- 172.21.15.42:0/927262611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 msgr2=0x55a4da5f4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 0x55a4da5f4a90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1908076aa0 tx=0x7f18f00051d0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 -- 172.21.15.42:0/927262611 shutdown_connections 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5ee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/927262611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da46ef40 0x55a4da5f4a90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.024+0000 7f191c9a6b80 1 -- 172.21.15.42:0/927262611 >> 172.21.15.42:0/927262611 conn(0x55a4da508490 msgr2=0x55a4da5e9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.025+0000 7f191c9a6b80 1 -- 172.21.15.42:0/927262611 shutdown_connections 2024-05-27T10:28:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.025+0000 7f191c9a6b80 1 -- 172.21.15.42:0/927262611 wait complete. 2024-05-27T10:28:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.025+0000 7f191c9a6b80 1 -- start start 2024-05-27T10:28:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 Processor -- start 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da46ef40 0x55a4da5f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 0x55a4da5f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a4da4120a0 con 0x55a4da55aaf0 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a4da4ac860 con 0x55a4da46ef40 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a4da5ef370 con 0x55a4da5ee5e0 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.026+0000 7f191c9a6b80 10 client.0 _ll_register_callbacks cb 0x55a4da4b5d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 0x55a4da5f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 0x55a4da5f88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43828/0 (socket says 172.21.15.42:43828) 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 -- 172.21.15.42:0/2455894230 learned_addr learned my addr 172.21.15.42:0/2455894230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f1906a00700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da46ef40 0x55a4da5f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f1907400700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 -- 172.21.15.42:0/2455894230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 msgr2=0x55a4da5f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 -- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da46ef40 msgr2=0x55a4da5f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da46ef40 0x55a4da5f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.027+0000 7f18fe000700 1 -- 172.21.15.42:0/2455894230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4da45b9d0 con 0x55a4da55aaf0 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f18fe000700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 0x55a4da5f88d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009990 tx=0x7f18f000bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d9ce6c5b994308 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:56.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f18f5600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f18f5600700 1 -- 172.21.15.42:0/2455894230 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0026040 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f18f5600700 1 -- 172.21.15.42:0/2455894230 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f18f000bc80 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f0009590 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f18f5600700 1 -- 172.21.15.42:0/2455894230 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f18f0003680 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f191c9a6b80 10 client.16974 fetch_fsmap learned FSMap version 37 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.028+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a4da4dc160 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f18f5600700 1 -- 172.21.15.42:0/2455894230 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f0009590 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f18f5600700 1 -- 172.21.15.42:0/2455894230 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f18f000f970 con 0x55a4da55aaf0 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f191c9a6b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f191c9a6b80 10 client.16974 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f191c9a6b80 20 client.16974 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:56.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da607f80 0x55a4da60a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f191c9a6b80 4 client.16974 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:28:56.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.029+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4da51de90 con 0x55a4da607f80 2024-05-27T10:28:56.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.030+0000 7f1906a00700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da607f80 0x55a4da60a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.031+0000 7f1906a00700 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da607f80 0x55a4da60a3a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602717f24c8f6cd3 server_cookie=e4baa69fa13b855c in_seq=0 out_seq=0 2024-05-27T10:28:56.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.031+0000 7f18f5600700 10 client.16974 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:56.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f18f5600700 1 -- 172.21.15.42:0/2455894230 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a4da51de90 con 0x55a4da607f80 2024-05-27T10:28:56.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f18f5600700 10 client.16974 handle_command_reply: tid=0 2024-05-27T10:28:56.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 client.16974 shutdown 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 msgr2=0x55a4da5f88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 0x55a4da5f88d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009990 tx=0x7f18f000bae0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 shutdown_connections 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da607f80 0x55a4da60a3a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da5ee5e0 0x55a4da5f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da46ef40 0x55a4da5f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/2455894230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da55aaf0 0x55a4da5f88d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 >> 172.21.15.42:0/2455894230 conn(0x55a4da508490 msgr2=0x55a4da5ed5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:56.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f18f5600700 0 client.16974 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:56.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 shutdown_connections 2024-05-27T10:28:56.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.033+0000 7f191c9a6b80 1 -- 172.21.15.42:0/2455894230 wait complete. 2024-05-27T10:28:56.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.034+0000 7f191c9a6b80 20 client.16974 trim_cache size 0 max 16384 2024-05-27T10:28:56.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 Processor -- start 2024-05-27T10:28:56.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 -- start start 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da5f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da604410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19081b3690 con 0x55a4da5ee5e0 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19081b0ec0 con 0x55a4da55aaf0 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19081aed30 con 0x55a4da46ef40 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.056+0000 7f1907400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da604410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da604410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43834/0 (socket says 172.21.15.42:43834) 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 -- 172.21.15.42:0/1817604150 learned_addr learned my addr 172.21.15.42:0/1817604150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1906a00700 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da5f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f18fe000700 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 -- 172.21.15.42:0/1817604150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 msgr2=0x55a4da5f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da5f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 -- 172.21.15.42:0/1817604150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 msgr2=0x55a4da5fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 -- 172.21.15.42:0/1817604150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4da5e5940 con 0x55a4da5ee5e0 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f18fe000700 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:28:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.057+0000 7f1907400700 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da604410 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1900007580 tx=0x7f1900013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693e19d0332a9a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:56.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.058+0000 7f18f7e00700 1 -- 172.21.15.42:0/1817604150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190001b240 con 0x55a4da5ee5e0 2024-05-27T10:28:56.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.058+0000 7f18f7e00700 1 -- 172.21.15.42:0/1817604150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f190001a8b0 con 0x55a4da5ee5e0 2024-05-27T10:28:56.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.058+0000 7f18f7e00700 1 -- 172.21.15.42:0/1817604150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900023800 con 0x55a4da5ee5e0 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 -- 172.21.15.42:0/1817604150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 msgr2=0x55a4da604410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da604410 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1900007580 tx=0x7f1900013000 comp rx=0 tx=0).stop 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 -- 172.21.15.42:0/1817604150 shutdown_connections 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da604410 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da5f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/1817604150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da5fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 -- 172.21.15.42:0/1817604150 >> 172.21.15.42:0/1817604150 conn(0x55a4da508490 msgr2=0x55a4da5f2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 -- 172.21.15.42:0/1817604150 shutdown_connections 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 -- 172.21.15.42:0/1817604150 wait complete. 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.059+0000 7f191c9a6b80 1 -- start start 2024-05-27T10:28:56.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 Processor -- start 2024-05-27T10:28:56.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da60b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da60b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da6717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19081af380 con 0x55a4da5ee5e0 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19081b07f0 con 0x55a4da55aaf0 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19081afa50 con 0x55a4da46ef40 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.060+0000 7f191c9a6b80 10 client.0 _ll_register_callbacks cb 0x55a4da4b5d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da6717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da6717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43836/0 (socket says 172.21.15.42:43836) 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 -- 172.21.15.42:0/3876862961 learned_addr learned my addr 172.21.15.42:0/3876862961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1906a00700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da60b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f18fe000700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da60b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 -- 172.21.15.42:0/3876862961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 msgr2=0x55a4da60b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da60b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 -- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 msgr2=0x55a4da60b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da60b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.061+0000 7f1907400700 1 -- 172.21.15.42:0/3876862961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4da5f9180 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f1907400700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da6717f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1900008bc0 tx=0x7f190001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27ba5df1da277da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f18f5600700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f18f5600700 1 -- 172.21.15.42:0/3876862961 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900029040 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f18f5600700 1 -- 172.21.15.42:0/3876862961 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1900004a90 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f190001a8b0 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f18f5600700 1 -- 172.21.15.42:0/3876862961 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1900004c90 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f191c9a6b80 10 client.16980 fetch_fsmap learned FSMap version 37 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.062+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a4da672ad0 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f18f5600700 1 -- 172.21.15.42:0/3876862961 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f190001a8b0 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f18f5600700 1 -- 172.21.15.42:0/3876862961 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f190002c070 con 0x55a4da5ee5e0 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f191c9a6b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f191c9a6b80 10 client.16980 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:28:56.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f191c9a6b80 20 client.16980 populate_metadata read hostname 'smithi042' 2024-05-27T10:28:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da673130 0x55a4da675550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:28:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f191c9a6b80 4 client.16980 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "14aec5ce", "release": true, "await": true}] 2024-05-27T10:28:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.063+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "14aec5ce", "release": true, "await": true}) -- 0x55a4da5e1810 con 0x55a4da673130 2024-05-27T10:28:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.064+0000 7f1906a00700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da673130 0x55a4da675550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:28:56.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.065+0000 7f1906a00700 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da673130 0x55a4da675550 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=189e2ea18f967a16 server_cookie=76f5528f6c2b0e3f in_seq=0 out_seq=0 2024-05-27T10:28:56.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.065+0000 7f18f5600700 10 client.16980 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:56.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.088+0000 7f18f5600700 1 -- 172.21.15.42:0/3876862961 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a4da5e1810 con 0x55a4da673130 2024-05-27T10:28:56.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.088+0000 7f18f5600700 10 client.16980 handle_command_reply: tid=0 2024-05-27T10:28:56.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.088+0000 7f191c9a6b80 1 client.16980 shutdown 2024-05-27T10:28:56.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.088+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 msgr2=0x55a4da6717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.088+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da6717f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1900008bc0 tx=0x7f190001ac90 comp rx=0 tx=0).stop 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 shutdown_connections 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55a4da673130 0x55a4da675550 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a4da5ee5e0 0x55a4da6717f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a4da46ef40 0x55a4da60b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 --2- 172.21.15.42:0/3876862961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4da55aaf0 0x55a4da60b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 >> 172.21.15.42:0/3876862961 conn(0x55a4da508490 msgr2=0x55a4da5f2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f18f5600700 0 client.16980 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 shutdown_connections 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 1 -- 172.21.15.42:0/3876862961 wait complete. 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.089+0000 7f191c9a6b80 20 client.16980 trim_cache size 0 max 16384 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 32, 2024-05-27T10:28:56.090 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": { 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "14aec5ce": { 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "version": 32, 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age_ref": 0.0, 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "timeout": 90.0, 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "expiration": 142.3, 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "members": { 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "file:/": { 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "excluded": false, 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "state": { 2024-05-27T10:28:56.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RELEASED", 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:28:56.092 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f18ec0b2a70 msgr2=0x7f18ec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f18ec0b2a70 0x7f18ec0b4f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1900006100 tx=0x7f1900008780 comp rx=0 tx=0).stop 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 msgr2=0x7f190819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 0x7f190819cfd0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d950 tx=0x7f18f800dd10 comp rx=0 tx=0).stop 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 shutdown_connections 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f18ec0b2a70 0x7f18ec0b4f80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908108740 0x7f19081a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19081034c0 0x7f190819cfd0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 --2- 172.21.15.42:0/1518631134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1908107df0 0x7f190819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.090+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 >> 172.21.15.42:0/1518631134 conn(0x7f19080fef10 msgr2=0x7f190806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.091+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 shutdown_connections 2024-05-27T10:28:56.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:28:56.091+0000 7f190de00700 1 -- 172.21.15.42:0/1518631134 wait complete. 2024-05-27T10:28:56.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:56 smithi143 ceph-mon[35545]: pgmap v316: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 27 op/s 2024-05-27T10:28:56.561 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:56 smithi042 ceph-mon[31661]: pgmap v316: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 27 op/s 2024-05-27T10:28:56.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:56 smithi104 ceph-mon[34812]: pgmap v316: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 27 op/s 2024-05-27T10:28:57.139 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '14aec5ce', total seconds elapsed: 30.8 2024-05-27T10:28:58.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:58 smithi143 ceph-mon[35545]: pgmap v317: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-27T10:28:58.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:58 smithi042 ceph-mon[31661]: pgmap v317: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-27T10:28:58.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:58 smithi104 ceph-mon[34812]: pgmap v317: 97 pgs: 97 active+clean; 382 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-27T10:28:59.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:28:59 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:59.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:28:59 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:28:59.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:28:59 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:29:00.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:00 smithi143 ceph-mon[35545]: pgmap v318: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 145 KiB/s wr, 31 op/s 2024-05-27T10:29:00.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:00 smithi042 ceph-mon[31661]: pgmap v318: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 145 KiB/s wr, 31 op/s 2024-05-27T10:29:00.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:00 smithi104 ceph-mon[34812]: pgmap v318: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 145 KiB/s wr, 31 op/s 2024-05-27T10:29:02.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:02 smithi042 ceph-mon[31661]: pgmap v319: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 KiB/s wr, 28 op/s 2024-05-27T10:29:02.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:02 smithi104 ceph-mon[34812]: pgmap v319: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 KiB/s wr, 28 op/s 2024-05-27T10:29:02.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:02 smithi143 ceph-mon[35545]: pgmap v319: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 KiB/s wr, 28 op/s 2024-05-27T10:29:04.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:04 smithi042 ceph-mon[31661]: pgmap v320: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 KiB/s wr, 28 op/s 2024-05-27T10:29:04.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:04 smithi104 ceph-mon[34812]: pgmap v320: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 KiB/s wr, 28 op/s 2024-05-27T10:29:04.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:04 smithi143 ceph-mon[35545]: pgmap v320: 97 pgs: 97 active+clean; 384 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 144 KiB/s wr, 28 op/s 2024-05-27T10:29:06.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:06 smithi042 ceph-mon[31661]: pgmap v321: 97 pgs: 97 active+clean; 388 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 548 KiB/s wr, 88 op/s 2024-05-27T10:29:06.607 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:06 smithi143 ceph-mon[35545]: pgmap v321: 97 pgs: 97 active+clean; 388 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 548 KiB/s wr, 88 op/s 2024-05-27T10:29:06.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:06 smithi104 ceph-mon[34812]: pgmap v321: 97 pgs: 97 active+clean; 388 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 548 KiB/s wr, 88 op/s 2024-05-27T10:29:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.997+0000 7f1d2b200640 1 -- 172.21.15.42:0/3087832971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 msgr2=0x7f1d24102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.997+0000 7f1d2b200640 1 --2- 172.21.15.42:0/3087832971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d24102570 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009900 tx=0x7f1d1802a410 comp rx=0 tx=0).stop 2024-05-27T10:29:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 -- 172.21.15.42:0/3087832971 shutdown_connections 2024-05-27T10:29:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 --2- 172.21.15.42:0/3087832971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d2410d340 0x7f1d2410f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 --2- 172.21.15.42:0/3087832971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d24102ab0 0x7f1d24102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 --2- 172.21.15.42:0/3087832971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d24102570 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 -- 172.21.15.42:0/3087832971 >> 172.21.15.42:0/3087832971 conn(0x7f1d240fe210 msgr2=0x7f1d24100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 -- 172.21.15.42:0/3087832971 shutdown_connections 2024-05-27T10:29:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.998+0000 7f1d2b200640 1 -- 172.21.15.42:0/3087832971 wait complete. 2024-05-27T10:29:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.999+0000 7f1d2b200640 1 Processor -- start 2024-05-27T10:29:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:06.999+0000 7f1d2b200640 1 -- start start 2024-05-27T10:29:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d2b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d24102ab0 0x7f1d241a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d2b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d241a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d2b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d2410d340 0x7f1d2419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d2b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24111950 con 0x7f1d241089e0 2024-05-27T10:29:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d2b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241117d0 con 0x7f1d2410d340 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d2b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24111ad0 con 0x7f1d24102ab0 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d29400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d241a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d29400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d241a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37566/0 (socket says 172.21.15.42:37566) 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d29400640 1 -- 172.21.15.42:0/1962794942 learned_addr learned my addr 172.21.15.42:0/1962794942 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.000+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d24102ab0 0x7f1d241a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d2410d340 0x7f1d2419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d29400640 1 -- 172.21.15.42:0/1962794942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d24102ab0 msgr2=0x7f1d241a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d29400640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d24102ab0 0x7f1d241a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d29400640 1 -- 172.21.15.42:0/1962794942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d2410d340 msgr2=0x7f1d2419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d29400640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d2410d340 0x7f1d2419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d29400640 1 -- 172.21.15.42:0/1962794942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2419cdb0 con 0x7f1d241089e0 2024-05-27T10:29:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d29400640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d241a2a40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005fd0 tx=0x7f1d1400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.001+0000 7f1d1ea00640 1 -- 172.21.15.42:0/1962794942 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14017070 con 0x7f1d241089e0 2024-05-27T10:29:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.002+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2419d040 con 0x7f1d241089e0 2024-05-27T10:29:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.002+0000 7f1d1ea00640 1 -- 172.21.15.42:0/1962794942 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d14009c30 con 0x7f1d241089e0 2024-05-27T10:29:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.002+0000 7f1d1ea00640 1 -- 172.21.15.42:0/1962794942 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14003c90 con 0x7f1d241089e0 2024-05-27T10:29:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.002+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a92a0 con 0x7f1d241089e0 2024-05-27T10:29:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.005+0000 7f1d337aa740 1 Processor -- start 2024-05-27T10:29:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.005+0000 7f1d337aa740 1 -- start start 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d1ea00640 1 -- 172.21.15.42:0/1962794942 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1d14003e30 con 0x7f1d241089e0 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x562607926a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x562607926fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075634d0 0x5626079274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d337aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562607897b90 con 0x56260755b2f0 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d337aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562607887730 con 0x5626075634d0 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d337aa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56260788b770 con 0x5626075841b0 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d29e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x562607926fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d29e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x562607926fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37576/0 (socket says 172.21.15.42:37576) 2024-05-27T10:29:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d29e00640 1 -- 172.21.15.42:0/1696573676 learned_addr learned my addr 172.21.15.42:0/1696573676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.006+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x562607926a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29400640 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075634d0 0x5626079274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29e00640 1 -- 172.21.15.42:0/1696573676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 msgr2=0x562607926a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x562607926a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29e00640 1 -- 172.21.15.42:0/1696573676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075634d0 msgr2=0x5626079274e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075634d0 0x5626079274e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29e00640 1 -- 172.21.15.42:0/1696573676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626078208b0 con 0x56260755b2f0 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29400640 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075634d0 0x5626079274e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x562607926fa0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1d18000c00 tx=0x7f1d18005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d41a9960ec368ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.007+0000 7f1d13000640 1 -- 172.21.15.42:0/1696573676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1803f070 con 0x56260755b2f0 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.008+0000 7f1d13000640 1 -- 172.21.15.42:0/1696573676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d18032d80 con 0x56260755b2f0 2024-05-27T10:29:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.008+0000 7f1d13000640 1 -- 172.21.15.42:0/1696573676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18035070 con 0x56260755b2f0 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.008+0000 7f1d1ea00640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1d0c0b2940 0x7f1d0c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.008+0000 7f1d1ea00640 1 -- 172.21.15.42:0/1962794942 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1d140e2360 con 0x7f1d241089e0 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.008+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1d0c0b2940 0x7f1d0c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 -- 172.21.15.42:0/1696573676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 msgr2=0x562607926fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x562607926fa0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1d18000c00 tx=0x7f1d18005ea0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 -- 172.21.15.42:0/1696573676 shutdown_connections 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075634d0 0x5626079274e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x562607926fa0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1696573676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x562607926a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 -- 172.21.15.42:0/1696573676 >> 172.21.15.42:0/1696573676 conn(0x562607578300 msgr2=0x562607923210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 -- 172.21.15.42:0/1696573676 shutdown_connections 2024-05-27T10:29:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1d0c0b2940 0x7f1d0c0b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x562607923bd0 tx=0x7f1d20009500 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.009+0000 7f1d337aa740 1 -- 172.21.15.42:0/1696573676 wait complete. 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 -- start start 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 Processor -- start 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607930c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 0x5626079311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x56260793ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56260756d790 con 0x5626075634d0 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.010+0000 7f1d337aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56260782fbc0 con 0x56260755b2f0 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d337aa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562607933b30 con 0x5626075841b0 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d337aa740 10 client.0 _ll_register_callbacks cb 0x56260780d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 0x5626079311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 0x5626079311c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37582/0 (socket says 172.21.15.42:37582) 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 -- 172.21.15.42:0/1749718020 learned_addr learned my addr 172.21.15.42:0/1749718020 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x56260793ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607930c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 -- 172.21.15.42:0/1749718020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 msgr2=0x56260793ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x56260793ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 -- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 msgr2=0x562607930c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607930c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.011+0000 7f1d29400640 1 -- 172.21.15.42:0/1749718020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260759c210 con 0x5626075634d0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d29400640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 0x5626079311c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1d140a95c0 tx=0x7f1d140a5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a18028e3441c03d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d07400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d07400640 1 -- 172.21.15.42:0/1749718020 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14009de0 con 0x5626075634d0 2024-05-27T10:29:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d07400640 1 -- 172.21.15.42:0/1749718020 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1d140dfd20 con 0x5626075634d0 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d18032d80 con 0x5626075634d0 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d07400640 1 -- 172.21.15.42:0/1749718020 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d140dff20 con 0x5626075634d0 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d337aa740 10 client.16992 fetch_fsmap learned FSMap version 38 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562607668620 con 0x5626075634d0 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.012+0000 7f1d07400640 1 -- 172.21.15.42:0/1749718020 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d18032d80 con 0x5626075634d0 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d07400640 1 -- 172.21.15.42:0/1749718020 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f1d14004120 con 0x5626075634d0 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d337aa740 10 client.16992 fetch_fsmap finished waiting for FSMap version 38 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d337aa740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d337aa740 20 client.16992 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x562607542d80 0x562607942e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d337aa740 4 client.16992 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.013+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562607899ae0 con 0x562607542d80 2024-05-27T10:29:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.014+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x562607542d80 0x562607942e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.015+0000 7f1d07400640 10 client.16992 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.015+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x562607542d80 0x562607942e50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba66f8da9345440 server_cookie=9adc70f391882ea in_seq=0 out_seq=0 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.017+0000 7f1d07400640 1 -- 172.21.15.42:0/1749718020 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562607899ae0 con 0x562607542d80 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.017+0000 7f1d07400640 10 client.16992 handle_command_reply: tid=0 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.017+0000 7f1d337aa740 1 client.16992 shutdown 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.017+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 msgr2=0x5626079311c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.017+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 0x5626079311c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1d140a95c0 tx=0x7f1d140a5b70 comp rx=0 tx=0).stop 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 shutdown_connections 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x562607542d80 0x562607942e50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075841b0 0x56260793ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075634d0 0x5626079311c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1749718020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607930c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 >> 172.21.15.42:0/1749718020 conn(0x562607578300 msgr2=0x562607926890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d07400640 0 client.16992 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 shutdown_connections 2024-05-27T10:29:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 1 -- 172.21.15.42:0/1749718020 wait complete. 2024-05-27T10:29:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.018+0000 7f1d337aa740 20 client.16992 trim_cache size 0 max 16384 2024-05-27T10:29:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 Processor -- start 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 -- start start 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x56260792c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x56260793cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075841b0 0x56260793d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b4160 con 0x56260755b2f0 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b19d0 con 0x5626075841b0 2024-05-27T10:29:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.036+0000 7f1d337aa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241af820 con 0x5626075634d0 2024-05-27T10:29:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x56260792c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x56260792c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37588/0 (socket says 172.21.15.42:37588) 2024-05-27T10:29:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 -- 172.21.15.42:0/359465026 learned_addr learned my addr 172.21.15.42:0/359465026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d2a800640 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075841b0 0x56260793d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29400640 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x56260793cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 -- 172.21.15.42:0/359465026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 msgr2=0x56260793cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x56260793cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 -- 172.21.15.42:0/359465026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075841b0 msgr2=0x56260793d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075841b0 0x56260793d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.037+0000 7f1d29e00640 1 -- 172.21.15.42:0/359465026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260793d680 con 0x56260755b2f0 2024-05-27T10:29:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.038+0000 7f1d29e00640 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x56260792c480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009900 tx=0x7f1d1802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e86cee80eda92bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.038+0000 7f1d12600640 1 -- 172.21.15.42:0/359465026 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18052070 con 0x56260755b2f0 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.038+0000 7f1d12600640 1 -- 172.21.15.42:0/359465026 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d18005050 con 0x56260755b2f0 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.038+0000 7f1d12600640 1 -- 172.21.15.42:0/359465026 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1803f040 con 0x56260755b2f0 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 -- 172.21.15.42:0/359465026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 msgr2=0x56260792c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x56260792c480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009900 tx=0x7f1d1802a410 comp rx=0 tx=0).stop 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 -- 172.21.15.42:0/359465026 shutdown_connections 2024-05-27T10:29:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626075841b0 0x56260793d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x56260793cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 --2- 172.21.15.42:0/359465026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56260755b2f0 0x56260792c480 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 -- 172.21.15.42:0/359465026 >> 172.21.15.42:0/359465026 conn(0x562607578300 msgr2=0x562607930f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 -- 172.21.15.42:0/359465026 shutdown_connections 2024-05-27T10:29:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.039+0000 7f1d337aa740 1 -- 172.21.15.42:0/359465026 wait complete. 2024-05-27T10:29:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.040+0000 7f1d337aa740 1 -- start start 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.040+0000 7f1d337aa740 1 Processor -- start 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607936280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x5626079367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 0x5626079a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241afe90 con 0x5626075841b0 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b1300 con 0x56260755b2f0 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b0560 con 0x5626075634d0 2024-05-27T10:29:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d337aa740 10 client.0 _ll_register_callbacks cb 0x7f1d14005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d2a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 0x5626079a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d2a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 0x5626079a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37598/0 (socket says 172.21.15.42:37598) 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d2a800640 1 -- 172.21.15.42:0/1142623772 learned_addr learned my addr 172.21.15.42:0/1142623772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607936280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.041+0000 7f1d29400640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x5626079367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d2a800640 1 -- 172.21.15.42:0/1142623772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 msgr2=0x5626079367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x5626079367c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d2a800640 1 -- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 msgr2=0x562607936280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607936280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d2a800640 1 -- 172.21.15.42:0/1142623772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626079a9a60 con 0x5626075841b0 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607936280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:29:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d2a800640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 0x5626079a9520 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1d200049d0 tx=0x7f1d20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ae25abf7a03b5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d07400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d07400640 1 -- 172.21.15.42:0/1142623772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004d20 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d18005050 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.042+0000 7f1d07400640 1 -- 172.21.15.42:0/1142623772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1d20012950 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d07400640 1 -- 172.21.15.42:0/1142623772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d20012b90 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d337aa740 10 client.16998 fetch_fsmap learned FSMap version 38 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5626079aac70 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d07400640 1 -- 172.21.15.42:0/1142623772 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d18005050 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d07400640 1 -- 172.21.15.42:0/1142623772 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f1d20019040 con 0x5626075841b0 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d337aa740 10 client.16998 fetch_fsmap finished waiting for FSMap version 38 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d337aa740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d337aa740 20 client.16998 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.043+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5626075181c0 0x5626079ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.044+0000 7f1d337aa740 4 client.16998 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:29:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.044+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626078932c0 con 0x5626075181c0 2024-05-27T10:29:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.044+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5626075181c0 0x5626079ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.045+0000 7f1d29e00640 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5626075181c0 0x5626079ad510 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce2f3f5bc74360d server_cookie=9970deee9cb72706 in_seq=0 out_seq=0 2024-05-27T10:29:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.045+0000 7f1d07400640 10 client.16998 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.073+0000 7f1d07400640 1 -- 172.21.15.42:0/1142623772 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5626078932c0 con 0x5626075181c0 2024-05-27T10:29:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.073+0000 7f1d07400640 10 client.16998 handle_command_reply: tid=0 2024-05-27T10:29:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.073+0000 7f1d337aa740 1 client.16998 shutdown 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 msgr2=0x5626079a9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 0x5626079a9520 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1d200049d0 tx=0x7f1d20004a40 comp rx=0 tx=0).stop 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 shutdown_connections 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626075841b0 0x5626079a9520 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5626075181c0 0x5626079ad510 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626075634d0 0x5626079367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 --2- 172.21.15.42:0/1142623772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56260755b2f0 0x562607936280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d07400640 0 client.16998 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 >> 172.21.15.42:0/1142623772 conn(0x562607578300 msgr2=0x562607930c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 shutdown_connections 2024-05-27T10:29:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 1 -- 172.21.15.42:0/1142623772 wait complete. 2024-05-27T10:29:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.074+0000 7f1d337aa740 20 client.16998 trim_cache size 0 max 16384 2024-05-27T10:29:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:29:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:29:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1d0c0b2940 msgr2=0x7f1d0c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1d0c0b2940 0x7f1d0c0b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x562607923bd0 tx=0x7f1d20009500 comp rx=0 tx=0).stop 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 msgr2=0x7f1d241a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d241a2a40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005fd0 tx=0x7f1d1400dde0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 shutdown_connections 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f1d0c0b2940 0x7f1d0c0b4e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d2410d340 0x7f1d2419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d241089e0 0x7f1d241a2a40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 --2- 172.21.15.42:0/1962794942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d24102ab0 0x7f1d241a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.075+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 >> 172.21.15.42:0/1962794942 conn(0x7f1d240fe210 msgr2=0x7f1d24104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.076+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 shutdown_connections 2024-05-27T10:29:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:07.076+0000 7f1d2b200640 1 -- 172.21.15.42:0/1962794942 wait complete. 2024-05-27T10:29:08.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:08 smithi042 ceph-mon[31661]: pgmap v322: 97 pgs: 97 active+clean; 388 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 539 KiB/s wr, 83 op/s 2024-05-27T10:29:08.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:08 smithi104 ceph-mon[34812]: pgmap v322: 97 pgs: 97 active+clean; 388 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 539 KiB/s wr, 83 op/s 2024-05-27T10:29:08.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:08 smithi143 ceph-mon[35545]: pgmap v322: 97 pgs: 97 active+clean; 388 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 539 KiB/s wr, 83 op/s 2024-05-27T10:29:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:10 smithi104 ceph-mon[34812]: pgmap v323: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 993 KiB/s wr, 150 op/s 2024-05-27T10:29:10.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:10 smithi143 ceph-mon[35545]: pgmap v323: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 993 KiB/s wr, 150 op/s 2024-05-27T10:29:10.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:10 smithi042 ceph-mon[31661]: pgmap v323: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 993 KiB/s wr, 150 op/s 2024-05-27T10:29:11.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:11 smithi104 ceph-mon[34812]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T10:29:11.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:11 smithi143 ceph-mon[35545]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T10:29:11.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:11 smithi042 ceph-mon[31661]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T10:29:12.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:12 smithi104 ceph-mon[34812]: pgmap v325: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-05-27T10:29:12.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:12 smithi143 ceph-mon[35545]: pgmap v325: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-05-27T10:29:12.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:12 smithi042 ceph-mon[31661]: pgmap v325: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-05-27T10:29:14.697 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:14 smithi104 ceph-mon[34812]: pgmap v326: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-05-27T10:29:14.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:14 smithi143 ceph-mon[35545]: pgmap v326: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-05-27T10:29:14.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:14 smithi042 ceph-mon[31661]: pgmap v326: 97 pgs: 97 active+clean; 393 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-05-27T10:29:17.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:16 smithi104 ceph-mon[34812]: pgmap v327: 97 pgs: 97 active+clean; 397 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 141 op/s 2024-05-27T10:29:17.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:16 smithi104 ceph-mon[34812]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T10:29:17.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:16 smithi143 ceph-mon[35545]: pgmap v327: 97 pgs: 97 active+clean; 397 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 141 op/s 2024-05-27T10:29:17.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:16 smithi143 ceph-mon[35545]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T10:29:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:16 smithi042 ceph-mon[31661]: pgmap v327: 97 pgs: 97 active+clean; 397 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 141 op/s 2024-05-27T10:29:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:16 smithi042 ceph-mon[31661]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T10:29:18.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:17 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:29:18.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:17 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:29:18.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:17 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:29:19.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:18 smithi104 ceph-mon[34812]: pgmap v329: 97 pgs: 97 active+clean; 397 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 575 KiB/s wr, 76 op/s 2024-05-27T10:29:19.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:18 smithi143 ceph-mon[35545]: pgmap v329: 97 pgs: 97 active+clean; 397 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 575 KiB/s wr, 76 op/s 2024-05-27T10:29:19.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:18 smithi042 ceph-mon[31661]: pgmap v329: 97 pgs: 97 active+clean; 397 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 575 KiB/s wr, 76 op/s 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 -- 172.21.15.42:0/4195546186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 msgr2=0x7fa4b810da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/4195546186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b810da90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009900 tx=0x7fa4a402a480 comp rx=0 tx=0).stop 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 -- 172.21.15.42:0/4195546186 shutdown_connections 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/4195546186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b810e020 0x7fa4b8110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/4195546186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b810da90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/4195546186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4b8104c10 0x7fa4b8105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 -- 172.21.15.42:0/4195546186 >> 172.21.15.42:0/4195546186 conn(0x7fa4b80fe210 msgr2=0x7fa4b8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 -- 172.21.15.42:0/4195546186 shutdown_connections 2024-05-27T10:29:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.315+0000 7fa4bdc00640 1 -- 172.21.15.42:0/4195546186 wait complete. 2024-05-27T10:29:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.316+0000 7fa4bdc00640 1 Processor -- start 2024-05-27T10:29:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 -- start start 2024-05-27T10:29:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4b8104c10 0x7fa4b819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b810e020 0x7fa4b819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8113b80 con 0x7fa4b810b670 2024-05-27T10:29:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8113a00 con 0x7fa4b810e020 2024-05-27T10:29:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.317+0000 7fa4bdc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8113d00 con 0x7fa4b8104c10 2024-05-27T10:29:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57752/0 (socket says 172.21.15.42:57752) 2024-05-27T10:29:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 -- 172.21.15.42:0/1265746069 learned_addr learned my addr 172.21.15.42:0/1265746069 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b810e020 0x7fa4b819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4b8104c10 0x7fa4b819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 -- 172.21.15.42:0/1265746069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4b8104c10 msgr2=0x7fa4b819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4b8104c10 0x7fa4b819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 -- 172.21.15.42:0/1265746069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b810e020 msgr2=0x7fa4b819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.318+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b810e020 0x7fa4b819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.319+0000 7fa4b7400640 1 -- 172.21.15.42:0/1265746069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b819f420 con 0x7fa4b810b670 2024-05-27T10:29:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.319+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b819ab20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa4a402a9a0 tx=0x7fa4a4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.319+0000 7fa4b4c00640 1 -- 172.21.15.42:0/1265746069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a403f070 con 0x7fa4b810b670 2024-05-27T10:29:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.319+0000 7fa4b4c00640 1 -- 172.21.15.42:0/1265746069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a4003e40 con 0x7fa4b810b670 2024-05-27T10:29:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.319+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b819f6b0 con 0x7fa4b810b670 2024-05-27T10:29:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.319+0000 7fa4b4c00640 1 -- 172.21.15.42:0/1265746069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4035030 con 0x7fa4b810b670 2024-05-27T10:29:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.320+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b819fba0 con 0x7fa4b810b670 2024-05-27T10:29:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.323+0000 7fa4c6378740 1 Processor -- start 2024-05-27T10:29:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.323+0000 7fa4c6378740 1 -- start start 2024-05-27T10:29:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe31a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe31fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe324e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4c6378740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56544fda2b90 con 0x56544fa8f1b0 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4c6378740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56544fd92730 con 0x56544fa6e4d0 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4c6378740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56544fd96770 con 0x56544fa662f0 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4b4c00640 1 -- 172.21.15.42:0/1265746069 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa4a4042070 con 0x7fa4b810b670 2024-05-27T10:29:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe31a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe31a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57758/0 (socket says 172.21.15.42:57758) 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4bd200640 1 -- 172.21.15.42:0/3520952767 learned_addr learned my addr 172.21.15.42:0/3520952767 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe31fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4bd200640 1 -- 172.21.15.42:0/3520952767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 msgr2=0x56544fe31fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.324+0000 7fa4bd200640 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe31fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4bd200640 1 -- 172.21.15.42:0/3520952767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 msgr2=0x56544fe324e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4bd200640 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4bd200640 1 -- 172.21.15.42:0/3520952767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544fd2b8b0 con 0x56544fa8f1b0 2024-05-27T10:29:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4bd200640 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe31a60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009f70 tx=0x7fa4b0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8495a982ec7021 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4ad600640 1 -- 172.21.15.42:0/3520952767 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0016070 con 0x56544fa8f1b0 2024-05-27T10:29:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4ad600640 1 -- 172.21.15.42:0/3520952767 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4b0004d20 con 0x56544fa8f1b0 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.325+0000 7fa4ad600640 1 -- 172.21.15.42:0/3520952767 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0012760 con 0x56544fa8f1b0 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.326+0000 7fa4b4c00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa49c0b2950 0x7fa49c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.326+0000 7fa4b4c00640 1 -- 172.21.15.42:0/1265746069 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fa4a4102df0 con 0x7fa4b810b670 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.326+0000 7fa4c6378740 1 -- 172.21.15.42:0/3520952767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 msgr2=0x56544fe31a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.326+0000 7fa4c6378740 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe31a60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009f70 tx=0x7fa4b0010800 comp rx=0 tx=0).stop 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.326+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa49c0b2950 0x7fa49c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 -- 172.21.15.42:0/3520952767 shutdown_connections 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe324e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe31fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 --2- 172.21.15.42:0/3520952767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe31a60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 -- 172.21.15.42:0/3520952767 >> 172.21.15.42:0/3520952767 conn(0x56544fa83300 msgr2=0x56544fe2e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 -- 172.21.15.42:0/3520952767 shutdown_connections 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 -- 172.21.15.42:0/3520952767 wait complete. 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa49c0b2950 0x7fa49c0b4e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x56544fe32e10 tx=0x7fa4a800b390 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 -- start start 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.327+0000 7fa4c6378740 1 Processor -- start 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe3c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe47ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56544fa78790 con 0x56544fa8f1b0 2024-05-27T10:29:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56544fd3abc0 con 0x56544fa6e4d0 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56544fe3eb90 con 0x56544fa662f0 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4c6378740 10 client.0 _ll_register_callbacks cb 0x56544fd18400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe47ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe47ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57760/0 (socket says 172.21.15.42:57760) 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.328+0000 7fa4bd200640 1 -- 172.21.15.42:0/1354783464 learned_addr learned my addr 172.21.15.42:0/1354783464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe3c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4bd200640 1 -- 172.21.15.42:0/1354783464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 msgr2=0x56544fe3bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe3bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4bd200640 1 -- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 msgr2=0x56544fe3c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe3c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4bd200640 1 -- 172.21.15.42:0/1354783464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544faa7210 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe47ef0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0011010 tx=0x7fa4b0005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fb3dc1d53d9d7f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.329+0000 7fa4a2000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4a2000640 1 -- 172.21.15.42:0/1354783464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0016030 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b0004d20 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4a2000640 1 -- 172.21.15.42:0/1354783464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fa4b00054b0 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4a2000640 1 -- 172.21.15.42:0/1354783464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa4b00056b0 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4c6378740 10 client.17010 fetch_fsmap learned FSMap version 39 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56544fb73620 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4a2000640 1 -- 172.21.15.42:0/1354783464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4b0004d20 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.330+0000 7fa4a2000640 1 -- 172.21.15.42:0/1354783464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa4b002c070 con 0x56544fa8f1b0 2024-05-27T10:29:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4c6378740 10 client.17010 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:29:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4c6378740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4c6378740 20 client.17010 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa4dd80 0x56544fe4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4c6378740 4 client.17010 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:29:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56544fda4ae0 con 0x56544fa4dd80 2024-05-27T10:29:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.331+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa4dd80 0x56544fe4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.332+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa4dd80 0x56544fe4de50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ecc32b064f966f3 server_cookie=a9c23f52789040ab in_seq=0 out_seq=0 2024-05-27T10:29:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.332+0000 7fa4a2000640 10 client.17010 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.334+0000 7fa4a2000640 1 -- 172.21.15.42:0/1354783464 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56544fda4ae0 con 0x56544fa4dd80 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.334+0000 7fa4a2000640 10 client.17010 handle_command_reply: tid=0 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.334+0000 7fa4c6378740 1 client.17010 shutdown 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.334+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 msgr2=0x56544fe47ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.334+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe47ef0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0011010 tx=0x7fa4b0005100 comp rx=0 tx=0).stop 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 shutdown_connections 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544fe47ef0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe3c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa4dd80 0x56544fe4de50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1354783464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544fe3bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 >> 172.21.15.42:0/1354783464 conn(0x56544fa83300 msgr2=0x56544fe33780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4a2000640 0 client.17010 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 shutdown_connections 2024-05-27T10:29:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 1 -- 172.21.15.42:0/1354783464 wait complete. 2024-05-27T10:29:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.335+0000 7fa4c6378740 20 client.17010 trim_cache size 0 max 16384 2024-05-27T10:29:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 Processor -- start 2024-05-27T10:29:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 -- start start 2024-05-27T10:29:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 0x56544fe47c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa8f1b0 0x56544fe53530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b2120 con 0x56544fa662f0 2024-05-27T10:29:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81af9c0 con 0x56544fa6e4d0 2024-05-27T10:29:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4c6378740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81ad830 con 0x56544fa8f1b0 2024-05-27T10:29:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 0x56544fe47c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 0x56544fe47c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57762/0 (socket says 172.21.15.42:57762) 2024-05-27T10:29:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.353+0000 7fa4b7e00640 1 -- 172.21.15.42:0/1188432993 learned_addr learned my addr 172.21.15.42:0/1188432993 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7e00640 1 -- 172.21.15.42:0/1188432993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa8f1b0 msgr2=0x56544fe53530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa8f1b0 0x56544fe53530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa8f1b0 0x56544fe53530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7e00640 1 -- 172.21.15.42:0/1188432993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 msgr2=0x56544fe481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7e00640 1 -- 172.21.15.42:0/1188432993 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544fe486e0 con 0x56544fa662f0 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.354+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 0x56544fe47c60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8004650 tx=0x7fa4a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff3623be7a0d46c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.355+0000 7fa4acc00640 1 -- 172.21.15.42:0/1188432993 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8012d60 con 0x56544fa662f0 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.355+0000 7fa4acc00640 1 -- 172.21.15.42:0/1188432993 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8028840 con 0x56544fa662f0 2024-05-27T10:29:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.355+0000 7fa4acc00640 1 -- 172.21.15.42:0/1188432993 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a801b970 con 0x56544fa662f0 2024-05-27T10:29:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 -- 172.21.15.42:0/1188432993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 msgr2=0x56544fe47c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 0x56544fe47c60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8004650 tx=0x7fa4a8015000 comp rx=0 tx=0).stop 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 -- 172.21.15.42:0/1188432993 shutdown_connections 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa8f1b0 0x56544fe53530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544fe481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1188432993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa662f0 0x56544fe47c60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 -- 172.21.15.42:0/1188432993 >> 172.21.15.42:0/1188432993 conn(0x56544fa83300 msgr2=0x56544fe3bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 -- 172.21.15.42:0/1188432993 shutdown_connections 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 -- 172.21.15.42:0/1188432993 wait complete. 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.356+0000 7fa4c6378740 1 -- start start 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 Processor -- start 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544feae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544feaed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544feaf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81ade80 con 0x56544fa8f1b0 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81af2f0 con 0x56544fa6e4d0 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81ae550 con 0x56544fa662f0 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4c6378740 10 client.0 _ll_register_callbacks cb 0x7fa4b0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544feaf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544feaf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57764/0 (socket says 172.21.15.42:57764) 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.357+0000 7fa4bd200640 1 -- 172.21.15.42:0/1438054330 learned_addr learned my addr 172.21.15.42:0/1438054330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544feae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4b7400640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544feaed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4bd200640 1 -- 172.21.15.42:0/1438054330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 msgr2=0x56544feae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544feae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4bd200640 1 -- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 msgr2=0x56544feaed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544feaed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4bd200640 1 -- 172.21.15.42:0/1438054330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544fe42e40 con 0x56544fa8f1b0 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544feaf250 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009ec0 tx=0x7fa4b0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50210d8d137e6a8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4a2000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4a2000640 1 -- 172.21.15.42:0/1438054330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0032070 con 0x56544fa8f1b0 2024-05-27T10:29:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4a2000640 1 -- 172.21.15.42:0/1438054330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fa4b0002030 con 0x56544fa8f1b0 2024-05-27T10:29:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.358+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a8028840 con 0x56544fa8f1b0 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4a2000640 1 -- 172.21.15.42:0/1438054330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa4b003b050 con 0x56544fa8f1b0 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 10 client.17016 fetch_fsmap learned FSMap version 39 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56544fe44040 con 0x56544fa8f1b0 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4a2000640 1 -- 172.21.15.42:0/1438054330 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8028840 con 0x56544fa8f1b0 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4a2000640 1 -- 172.21.15.42:0/1438054330 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa4b0035020 con 0x56544fa8f1b0 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 10 client.17016 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:29:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 20 client.17016 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa231c0 0x56544fec1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 4 client.17016 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.359+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56544fd9e2c0 con 0x56544fa231c0 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.360+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa231c0 0x56544fec1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.361+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa231c0 0x56544fec1010 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa94f3c807d08bfa server_cookie=1f5096c6ca73cc4c in_seq=0 out_seq=0 2024-05-27T10:29:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.361+0000 7fa4a2000640 10 client.17016 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.387+0000 7fa4a2000640 1 -- 172.21.15.42:0/1438054330 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56544fd9e2c0 con 0x56544fa231c0 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4a2000640 10 client.17016 handle_command_reply: tid=0 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 client.17016 shutdown 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 msgr2=0x56544feaf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544feaf250 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009ec0 tx=0x7fa4b0009800 comp rx=0 tx=0).stop 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 shutdown_connections 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56544fa8f1b0 0x56544feaf250 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56544fa6e4d0 0x56544feaed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x56544fa231c0 0x56544fec1010 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 --2- 172.21.15.42:0/1438054330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56544fa662f0 0x56544feae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4a2000640 0 client.17016 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 >> 172.21.15.42:0/1438054330 conn(0x56544fa83300 msgr2=0x56544fe36460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.388+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 shutdown_connections 2024-05-27T10:29:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.389+0000 7fa4c6378740 1 -- 172.21.15.42:0/1438054330 wait complete. 2024-05-27T10:29:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.389+0000 7fa4c6378740 20 client.17016 trim_cache size 0 max 16384 2024-05-27T10:29:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:29:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:29:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:29:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa49c0b2950 msgr2=0x7fa49c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa49c0b2950 0x7fa49c0b4e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x56544fe32e10 tx=0x7fa4a800b390 comp rx=0 tx=0).stop 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 msgr2=0x7fa4b819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b819ab20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa4a402a9a0 tx=0x7fa4a4003c20 comp rx=0 tx=0).stop 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 shutdown_connections 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa49c0b2950 0x7fa49c0b4e60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b810e020 0x7fa4b819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b810b670 0x7fa4b819ab20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1265746069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4b8104c10 0x7fa4b819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 >> 172.21.15.42:0/1265746069 conn(0x7fa4b80fe210 msgr2=0x7fa4b810e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 shutdown_connections 2024-05-27T10:29:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:20.390+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1265746069 wait complete. 2024-05-27T10:29:21.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:20 smithi104 ceph-mon[34812]: pgmap v330: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-05-27T10:29:21.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:20 smithi143 ceph-mon[35545]: pgmap v330: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-05-27T10:29:21.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:20 smithi042 ceph-mon[31661]: pgmap v330: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-05-27T10:29:23.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:22 smithi104 ceph-mon[34812]: pgmap v331: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 116 op/s 2024-05-27T10:29:23.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:22 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:29:23.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:22 smithi143 ceph-mon[35545]: pgmap v331: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 116 op/s 2024-05-27T10:29:23.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:22 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:29:23.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:22 smithi042 ceph-mon[31661]: pgmap v331: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 116 op/s 2024-05-27T10:29:23.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:22 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:29:25.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:24 smithi104 ceph-mon[34812]: pgmap v332: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 116 op/s 2024-05-27T10:29:25.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:24 smithi143 ceph-mon[35545]: pgmap v332: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 116 op/s 2024-05-27T10:29:25.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:24 smithi042 ceph-mon[31661]: pgmap v332: 97 pgs: 97 active+clean; 402 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 116 op/s 2024-05-27T10:29:26.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: pgmap v333: 97 pgs: 97 active+clean; 408 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-05-27T10:29:26.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:26 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: pgmap v333: 97 pgs: 97 active+clean; 408 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-05-27T10:29:26.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:26 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: pgmap v333: 97 pgs: 97 active+clean; 408 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-05-27T10:29:26.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:26.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:26 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:27.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-05-27T10:29:27.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi104 to 176.4M 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: Unable to set osd_memory_target on smithi104 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:29:27.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:29:27.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:27 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:27.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi104 to 176.4M 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: Unable to set osd_memory_target on smithi104 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:29:27.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:29:27.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:27 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi104 to 176.4M 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: Unable to set osd_memory_target on smithi104 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:29:27.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:29:27.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:27 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:29:28.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:28 smithi143 ceph-mon[35545]: pgmap v334: 97 pgs: 97 active+clean; 408 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 986 KiB/s wr, 110 op/s 2024-05-27T10:29:28.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:28 smithi143 ceph-mon[35545]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-05-27T10:29:28.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:28 smithi143 ceph-mon[35545]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T10:29:28.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:28 smithi042 ceph-mon[31661]: pgmap v334: 97 pgs: 97 active+clean; 408 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 986 KiB/s wr, 110 op/s 2024-05-27T10:29:28.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:28 smithi042 ceph-mon[31661]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-05-27T10:29:28.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:28 smithi042 ceph-mon[31661]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T10:29:28.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:28 smithi104 ceph-mon[34812]: pgmap v334: 97 pgs: 97 active+clean; 408 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 986 KiB/s wr, 110 op/s 2024-05-27T10:29:28.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:28 smithi104 ceph-mon[34812]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-05-27T10:29:28.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:28 smithi104 ceph-mon[34812]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-27T10:29:30.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:30 smithi143 ceph-mon[35545]: pgmap v335: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-27T10:29:30.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:30 smithi042 ceph-mon[31661]: pgmap v335: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-27T10:29:30.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:30 smithi104 ceph-mon[34812]: pgmap v335: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-27T10:29:32.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:32 smithi143 ceph-mon[35545]: pgmap v336: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 998 KiB/s wr, 123 op/s 2024-05-27T10:29:32.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:32 smithi042 ceph-mon[31661]: pgmap v336: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 998 KiB/s wr, 123 op/s 2024-05-27T10:29:32.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:32 smithi104 ceph-mon[34812]: pgmap v336: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 998 KiB/s wr, 123 op/s 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 -- 172.21.15.42:0/3706851035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109ad0 msgr2=0x7f93a8103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 --2- 172.21.15.42:0/3706851035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109ad0 0x7f93a8103670 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9398009900 tx=0x7f939802a410 comp rx=0 tx=0).stop 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 -- 172.21.15.42:0/3706851035 shutdown_connections 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 --2- 172.21.15.42:0/3706851035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a8104220 0x7f93a81105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 --2- 172.21.15.42:0/3706851035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109f70 0x7f93a8103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 --2- 172.21.15.42:0/3706851035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109ad0 0x7f93a8103670 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 -- 172.21.15.42:0/3706851035 >> 172.21.15.42:0/3706851035 conn(0x7f93a8079100 msgr2=0x7f93a8079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.630+0000 7f93ad800640 1 -- 172.21.15.42:0/3706851035 shutdown_connections 2024-05-27T10:29:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.631+0000 7f93ad800640 1 -- 172.21.15.42:0/3706851035 wait complete. 2024-05-27T10:29:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 Processor -- start 2024-05-27T10:29:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 -- start start 2024-05-27T10:29:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a8104220 0x7f93a819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109ad0 0x7f93a819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 0x7f93a819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8113b70 con 0x7f93a8109f70 2024-05-27T10:29:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81139f0 con 0x7f93a8104220 2024-05-27T10:29:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.632+0000 7f93ad800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8113cf0 con 0x7f93a8109ad0 2024-05-27T10:29:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 0x7f93a819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 0x7f93a819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35176/0 (socket says 172.21.15.42:35176) 2024-05-27T10:29:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 -- 172.21.15.42:0/683992543 learned_addr learned my addr 172.21.15.42:0/683992543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93a7e00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a8104220 0x7f93a819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93a7400640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109ad0 0x7f93a819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 -- 172.21.15.42:0/683992543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109ad0 msgr2=0x7f93a819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109ad0 0x7f93a819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 -- 172.21.15.42:0/683992543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a8104220 msgr2=0x7f93a819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a8104220 0x7f93a819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93ace00640 1 -- 172.21.15.42:0/683992543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a819f430 con 0x7f93a8109f70 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.633+0000 7f93a7400640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109ad0 0x7f93a819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.634+0000 7f93ace00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 0x7f93a819edf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f93a000eaa0 tx=0x7f93a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.634+0000 7f93a4c00640 1 -- 172.21.15.42:0/683992543 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a000cdb0 con 0x7f93a8109f70 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.634+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a819f720 con 0x7f93a8109f70 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.634+0000 7f93a4c00640 1 -- 172.21.15.42:0/683992543 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a0005020 con 0x7f93a8109f70 2024-05-27T10:29:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.634+0000 7f93a4c00640 1 -- 172.21.15.42:0/683992543 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a00116d0 con 0x7f93a8109f70 2024-05-27T10:29:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.634+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a819fc20 con 0x7f93a8109f70 2024-05-27T10:29:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.637+0000 7f93b5e03740 1 Processor -- start 2024-05-27T10:29:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.638+0000 7f93a4c00640 1 -- 172.21.15.42:0/683992543 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f93a00051d0 con 0x7f93a8109f70 2024-05-27T10:29:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.638+0000 7f93b5e03740 1 -- start start 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x564233258e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x5642332598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93b5e03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5642331c5b90 con 0x564232eb21b0 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93b5e03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642331b5730 con 0x564232e892f0 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93b5e03740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5642331b9770 con 0x564232e914d0 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x564233258e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x564233258e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35178/0 (socket says 172.21.15.42:35178) 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.639+0000 7f93a7400640 1 -- 172.21.15.42:0/4040695381 learned_addr learned my addr 172.21.15.42:0/4040695381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7e00640 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7400640 1 -- 172.21.15.42:0/4040695381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 msgr2=0x5642332598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7400640 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x5642332598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7400640 1 -- 172.21.15.42:0/4040695381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 msgr2=0x5642332593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7400640 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7400640 1 -- 172.21.15.42:0/4040695381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423314e8b0 con 0x564232eb21b0 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a4c00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f938c0b2950 0x7f938c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a7400640 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x564233258e70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f939400bcf0 tx=0x7f9394011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10d2adeaac1a6bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.640+0000 7f93a4c00640 1 -- 172.21.15.42:0/683992543 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f93a00e2850 con 0x7f93a8109f70 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.641+0000 7f939d600640 1 -- 172.21.15.42:0/4040695381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394004030 con 0x564232eb21b0 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.641+0000 7f93a7e00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f938c0b2950 0x7f938c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.641+0000 7f939d600640 1 -- 172.21.15.42:0/4040695381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93940035f0 con 0x564232eb21b0 2024-05-27T10:29:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.641+0000 7f939d600640 1 -- 172.21.15.42:0/4040695381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394012650 con 0x564232eb21b0 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 -- 172.21.15.42:0/4040695381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 msgr2=0x564233258e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x564233258e70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f939400bcf0 tx=0x7f9394011750 comp rx=0 tx=0).stop 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93a7e00640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f938c0b2950 0x7f938c0b4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9398004770 tx=0x7f93980058d0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 -- 172.21.15.42:0/4040695381 shutdown_connections 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x5642332598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 --2- 172.21.15.42:0/4040695381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x564233258e70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 -- 172.21.15.42:0/4040695381 >> 172.21.15.42:0/4040695381 conn(0x564232ea6300 msgr2=0x564233255620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 -- 172.21.15.42:0/4040695381 shutdown_connections 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 -- 172.21.15.42:0/4040695381 wait complete. 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.642+0000 7f93b5e03740 1 -- start start 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 Processor -- start 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x564233263800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564232e9b790 con 0x564232eb21b0 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56423315dbc0 con 0x564232e892f0 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564233265fa0 con 0x564232e914d0 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.643+0000 7f93b5e03740 10 client.0 _ll_register_callbacks cb 0x56423313b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35184/0 (socket says 172.21.15.42:35184) 2024-05-27T10:29:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 -- 172.21.15.42:0/2641547170 learned_addr learned my addr 172.21.15.42:0/2641547170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93a7400640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x564233263800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93a7e00640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 -- 172.21.15.42:0/2641547170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 msgr2=0x564233263800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x564233263800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 -- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 msgr2=0x5642332632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.644+0000 7f93ace00640 1 -- 172.21.15.42:0/2641547170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564232eca210 con 0x564232eb21b0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f93ace00640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f2d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93a00a9a80 tx=0x7f93a0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ebe617c2f12f3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f9392000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f9392000640 1 -- 172.21.15.42:0/2641547170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a001f030 con 0x564232eb21b0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f9392000640 1 -- 172.21.15.42:0/2641547170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f93a00a8d50 con 0x564232eb21b0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93940035f0 con 0x564232eb21b0 2024-05-27T10:29:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f9392000640 1 -- 172.21.15.42:0/2641547170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f93a00adea0 con 0x564232eb21b0 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f93b5e03740 10 client.17028 fetch_fsmap learned FSMap version 39 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564232f96620 con 0x564232eb21b0 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.645+0000 7f9392000640 1 -- 172.21.15.42:0/2641547170 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93940035f0 con 0x564232eb21b0 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f9392000640 1 -- 172.21.15.42:0/2641547170 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f93a000f040 con 0x564232eb21b0 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f93b5e03740 10 client.17028 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f93b5e03740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f93b5e03740 20 client.17028 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f93b5e03740 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e70d80 0x564233275360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f93b5e03740 4 client.17028 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.646+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642331c7ae0 con 0x564232e70d80 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.647+0000 7f93a7e00640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e70d80 0x564233275360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.648+0000 7f93a7e00640 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e70d80 0x564233275360 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62990279cc809100 server_cookie=20e3b9b109d9e533 in_seq=0 out_seq=0 2024-05-27T10:29:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.648+0000 7f9392000640 10 client.17028 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f9392000640 1 -- 172.21.15.42:0/2641547170 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5642331c7ae0 con 0x564232e70d80 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f9392000640 10 client.17028 handle_command_reply: tid=0 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 client.17028 shutdown 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 msgr2=0x56423326f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f2d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93a00a9a80 tx=0x7f93a0005720 comp rx=0 tx=0).stop 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 shutdown_connections 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f2d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e70d80 0x564233275360 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e914d0 0x564233263800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 --2- 172.21.15.42:0/2641547170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e892f0 0x5642332632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f9392000640 0 client.17028 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 >> 172.21.15.42:0/2641547170 conn(0x564232ea6300 msgr2=0x564233258ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 shutdown_connections 2024-05-27T10:29:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 1 -- 172.21.15.42:0/2641547170 wait complete. 2024-05-27T10:29:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.650+0000 7f93b5e03740 20 client.17028 trim_cache size 0 max 16384 2024-05-27T10:29:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 Processor -- start 2024-05-27T10:29:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 -- start start 2024-05-27T10:29:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233259c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x56423324ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81afe40 con 0x564232eb21b0 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81ad6e0 con 0x564232e914d0 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.668+0000 7f93b5e03740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81ab570 con 0x564232e892f0 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35192/0 (socket says 172.21.15.42:35192) 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 -- 172.21.15.42:0/125048639 learned_addr learned my addr 172.21.15.42:0/125048639 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93a7e00640 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233259c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93a7400640 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x56423324ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 -- 172.21.15.42:0/125048639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 msgr2=0x564233259c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233259c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 -- 172.21.15.42:0/125048639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 msgr2=0x56423324ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x56423324ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 -- 172.21.15.42:0/125048639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564233261600 con 0x564232eb21b0 2024-05-27T10:29:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.669+0000 7f93ace00640 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f040 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009270 tx=0x7f93a00a9600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17fc4655071cb2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.670+0000 7f939cc00640 1 -- 172.21.15.42:0/125048639 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a001f070 con 0x564232eb21b0 2024-05-27T10:29:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.670+0000 7f939cc00640 1 -- 172.21.15.42:0/125048639 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a00b2950 con 0x564232eb21b0 2024-05-27T10:29:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.670+0000 7f939cc00640 1 -- 172.21.15.42:0/125048639 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a00a8d50 con 0x564232eb21b0 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 -- 172.21.15.42:0/125048639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 msgr2=0x56423326f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f040 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009270 tx=0x7f93a00a9600 comp rx=0 tx=0).stop 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 -- 172.21.15.42:0/125048639 shutdown_connections 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x56423326f040 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x56423324ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 --2- 172.21.15.42:0/125048639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233259c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 -- 172.21.15.42:0/125048639 >> 172.21.15.42:0/125048639 conn(0x564232ea6300 msgr2=0x56423324f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 -- 172.21.15.42:0/125048639 shutdown_connections 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 -- 172.21.15.42:0/125048639 wait complete. 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 -- start start 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.671+0000 7f93b5e03740 1 Processor -- start 2024-05-27T10:29:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233266a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x564233266f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x5642332d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81abbc0 con 0x564232eb21b0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81ad010 con 0x564232e914d0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81ac270 con 0x564232e892f0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93b5e03740 10 client.0 _ll_register_callbacks cb 0x7f93a00a5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x5642332d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x5642332d59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35206/0 (socket says 172.21.15.42:35206) 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.672+0000 7f93ace00640 1 -- 172.21.15.42:0/1252349958 learned_addr learned my addr 172.21.15.42:0/1252349958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93a7400640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x564233266f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93a7e00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233266a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93ace00640 1 -- 172.21.15.42:0/1252349958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 msgr2=0x564233266a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93ace00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233266a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93ace00640 1 -- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 msgr2=0x564233266f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93ace00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x564233266f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93ace00640 1 -- 172.21.15.42:0/1252349958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642332d5f80 con 0x564232eb21b0 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93a7400640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x564233266f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93a7e00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233266a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93ace00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x5642332d59f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f93a00a9a80 tx=0x7f93a00a8a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14476abe68766ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f9392000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f9392000640 1 -- 172.21.15.42:0/1252349958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a008d020 con 0x564232eb21b0 2024-05-27T10:29:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.673+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a00b2950 con 0x564232eb21b0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f9392000640 1 -- 172.21.15.42:0/1252349958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f93a00a8f60 con 0x564232eb21b0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f9392000640 1 -- 172.21.15.42:0/1252349958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f93a001f050 con 0x564232eb21b0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 10 client.17034 fetch_fsmap learned FSMap version 39 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5642332dd2b0 con 0x564232eb21b0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f9392000640 1 -- 172.21.15.42:0/1252349958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a00b2950 con 0x564232eb21b0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f9392000640 1 -- 172.21.15.42:0/1252349958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f93a0004b60 con 0x564232eb21b0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 10 client.17034 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 20 client.17034 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e461c0 0x5642332df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 4 client.17034 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.674+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56423325d670 con 0x564232e461c0 2024-05-27T10:29:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.675+0000 7f93a7e00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e461c0 0x5642332df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.676+0000 7f93a7e00640 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e461c0 0x5642332df8b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b282a8da762f62f5 server_cookie=f8e3fd1d1c711477 in_seq=0 out_seq=0 2024-05-27T10:29:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.676+0000 7f9392000640 10 client.17034 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.702+0000 7f9392000640 1 -- 172.21.15.42:0/1252349958 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56423325d670 con 0x564232e461c0 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.702+0000 7f9392000640 10 client.17034 handle_command_reply: tid=0 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.702+0000 7f93b5e03740 1 client.17034 shutdown 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.702+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 msgr2=0x5642332d59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.702+0000 7f93b5e03740 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x5642332d59f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f93a00a9a80 tx=0x7f93a00a8a50 comp rx=0 tx=0).stop 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 shutdown_connections 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564232eb21b0 0x5642332d59f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564232e914d0 0x564233266f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x564232e461c0 0x5642332df8b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 --2- 172.21.15.42:0/1252349958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564232e892f0 0x564233266a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f9392000640 0 client.17034 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 >> 172.21.15.42:0/1252349958 conn(0x564232ea6300 msgr2=0x564233264760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 shutdown_connections 2024-05-27T10:29:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 1 -- 172.21.15.42:0/1252349958 wait complete. 2024-05-27T10:29:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.703+0000 7f93b5e03740 20 client.17034 trim_cache size 0 max 16384 2024-05-27T10:29:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:29:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:29:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f938c0b2950 msgr2=0x7f938c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f938c0b2950 0x7f938c0b4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9398004770 tx=0x7f93980058d0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 msgr2=0x7f93a819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 0x7f93a819edf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f93a000eaa0 tx=0x7f93a000ef60 comp rx=0 tx=0).stop 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 shutdown_connections 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f938c0b2950 0x7f938c0b4e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93a8109f70 0x7f93a819edf0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8109ad0 0x7f93a819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 --2- 172.21.15.42:0/683992543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a8104220 0x7f93a819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 >> 172.21.15.42:0/683992543 conn(0x7f93a8079100 msgr2=0x7f93a8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 shutdown_connections 2024-05-27T10:29:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:33.705+0000 7f93ad800640 1 -- 172.21.15.42:0/683992543 wait complete. 2024-05-27T10:29:34.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:34 smithi143 ceph-mon[35545]: pgmap v337: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 998 KiB/s wr, 123 op/s 2024-05-27T10:29:34.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:34 smithi042 ceph-mon[31661]: pgmap v337: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 998 KiB/s wr, 123 op/s 2024-05-27T10:29:34.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:34 smithi104 ceph-mon[34812]: pgmap v337: 97 pgs: 97 active+clean; 413 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 998 KiB/s wr, 123 op/s 2024-05-27T10:29:36.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:36 smithi143 ceph-mon[35545]: pgmap v338: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-05-27T10:29:36.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:36 smithi042 ceph-mon[31661]: pgmap v338: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-05-27T10:29:36.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:36 smithi104 ceph-mon[34812]: pgmap v338: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-05-27T10:29:38.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:38 smithi143 ceph-mon[35545]: pgmap v339: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 120 op/s 2024-05-27T10:29:38.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:38 smithi042 ceph-mon[31661]: pgmap v339: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 120 op/s 2024-05-27T10:29:38.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:38 smithi104 ceph-mon[34812]: pgmap v339: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 120 op/s 2024-05-27T10:29:40.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:40 smithi042 ceph-mon[31661]: pgmap v340: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-27T10:29:40.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:40 smithi104 ceph-mon[34812]: pgmap v340: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-27T10:29:41.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:40 smithi143 ceph-mon[35545]: pgmap v340: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-27T10:29:43.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:42 smithi104 ceph-mon[34812]: pgmap v341: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 904 KiB/s wr, 96 op/s 2024-05-27T10:29:43.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:42 smithi143 ceph-mon[35545]: pgmap v341: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 904 KiB/s wr, 96 op/s 2024-05-27T10:29:43.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:42 smithi042 ceph-mon[31661]: pgmap v341: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 904 KiB/s wr, 96 op/s 2024-05-27T10:29:45.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:44 smithi104 ceph-mon[34812]: pgmap v342: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 904 KiB/s wr, 96 op/s 2024-05-27T10:29:45.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:44 smithi143 ceph-mon[35545]: pgmap v342: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 904 KiB/s wr, 96 op/s 2024-05-27T10:29:45.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:44 smithi042 ceph-mon[31661]: pgmap v342: 97 pgs: 97 active+clean; 424 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 904 KiB/s wr, 96 op/s 2024-05-27T10:29:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 -- 172.21.15.42:0/2369475799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab9410db90 msgr2=0x7fab9410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 --2- 172.21.15.42:0/2369475799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab9410db90 0x7fab9410ffb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009900 tx=0x7fab7c02a490 comp rx=0 tx=0).stop 2024-05-27T10:29:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 -- 172.21.15.42:0/2369475799 shutdown_connections 2024-05-27T10:29:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 --2- 172.21.15.42:0/2369475799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab941104f0 0x7fab94112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 --2- 172.21.15.42:0/2369475799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab9410db90 0x7fab9410ffb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 --2- 172.21.15.42:0/2369475799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab94076510 0x7fab94076920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 -- 172.21.15.42:0/2369475799 >> 172.21.15.42:0/2369475799 conn(0x7fab940ff340 msgr2=0x7fab94101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 -- 172.21.15.42:0/2369475799 shutdown_connections 2024-05-27T10:29:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.948+0000 7fab99a00640 1 -- 172.21.15.42:0/2369475799 wait complete. 2024-05-27T10:29:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 Processor -- start 2024-05-27T10:29:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 -- start start 2024-05-27T10:29:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab94076510 0x7fab941a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab9410db90 0x7fab941a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 0x7fab9419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab94115f30 con 0x7fab941104f0 2024-05-27T10:29:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab94115db0 con 0x7fab94076510 2024-05-27T10:29:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.950+0000 7fab99a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab941160b0 con 0x7fab9410db90 2024-05-27T10:29:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 0x7fab9419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 0x7fab9419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35754/0 (socket says 172.21.15.42:35754) 2024-05-27T10:29:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 -- 172.21.15.42:0/579633375 learned_addr learned my addr 172.21.15.42:0/579633375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab92a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab94076510 0x7fab941a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab8be00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab9410db90 0x7fab941a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 -- 172.21.15.42:0/579633375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab9410db90 msgr2=0x7fab941a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab9410db90 0x7fab941a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 -- 172.21.15.42:0/579633375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab94076510 msgr2=0x7fab941a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab94076510 0x7fab941a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab93400640 1 -- 172.21.15.42:0/579633375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9419cf10 con 0x7fab941104f0 2024-05-27T10:29:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.951+0000 7fab92a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab94076510 0x7fab941a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:29:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.952+0000 7fab93400640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 0x7fab9419c870 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00eaa0 tx=0x7fab8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.952+0000 7fab8b400640 1 -- 172.21.15.42:0/579633375 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00cdb0 con 0x7fab941104f0 2024-05-27T10:29:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.952+0000 7fab8b400640 1 -- 172.21.15.42:0/579633375 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab8c005020 con 0x7fab941104f0 2024-05-27T10:29:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.952+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9419d200 con 0x7fab941104f0 2024-05-27T10:29:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.952+0000 7fab8b400640 1 -- 172.21.15.42:0/579633375 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0116d0 con 0x7fab941104f0 2024-05-27T10:29:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.952+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab941a94e0 con 0x7fab941104f0 2024-05-27T10:29:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.956+0000 7faba09dd740 1 Processor -- start 2024-05-27T10:29:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.956+0000 7fab8b400640 1 -- 172.21.15.42:0/579633375 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fab8c0051d0 con 0x7fab941104f0 2024-05-27T10:29:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.956+0000 7faba09dd740 1 -- start start 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7faba09dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a4b1b0 0x5609d4ded860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7faba09dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4dedda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7faba09dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4dee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7faba09dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5609d4d5eb90 con 0x5609d4a2a4d0 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7faba09dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5609d4d4e730 con 0x5609d4a222f0 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7faba09dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5609d4d52770 con 0x5609d4a4b1b0 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7fab8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4dee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7fab8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4dee2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35768/0 (socket says 172.21.15.42:35768) 2024-05-27T10:29:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7fab8be00640 1 -- 172.21.15.42:0/3345989025 learned_addr learned my addr 172.21.15.42:0/3345989025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7fab92a00640 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4dedda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.957+0000 7fab8be00640 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a4b1b0 0x5609d4ded860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8b400640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fab780b2950 0x7fab780b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8be00640 1 -- 172.21.15.42:0/3345989025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a4b1b0 msgr2=0x5609d4ded860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8be00640 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a4b1b0 0x5609d4ded860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8be00640 1 -- 172.21.15.42:0/3345989025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 msgr2=0x5609d4dedda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8be00640 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4dedda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8be00640 1 -- 172.21.15.42:0/3345989025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d4ce78b0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8b400640 1 -- 172.21.15.42:0/579633375 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fab8c0e28a0 con 0x7fab941104f0 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab93400640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fab780b2950 0x7fab780b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.958+0000 7fab8be00640 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4dee2e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab7c005f70 tx=0x7fab7c02a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4a6dbc39af01e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.959+0000 7fab83e00640 1 -- 172.21.15.42:0/3345989025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c035040 con 0x5609d4a2a4d0 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.959+0000 7fab83e00640 1 -- 172.21.15.42:0/3345989025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab7c0038c0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.959+0000 7fab83e00640 1 -- 172.21.15.42:0/3345989025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c004920 con 0x5609d4a2a4d0 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 -- 172.21.15.42:0/3345989025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 msgr2=0x5609d4dee2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4dee2e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab7c005f70 tx=0x7fab7c02a490 comp rx=0 tx=0).stop 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7fab93400640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fab780b2950 0x7fab780b4e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0092f0 tx=0x7fab8c005b20 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 -- 172.21.15.42:0/3345989025 shutdown_connections 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4dee2e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4dedda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 --2- 172.21.15.42:0/3345989025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a4b1b0 0x5609d4ded860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 -- 172.21.15.42:0/3345989025 >> 172.21.15.42:0/3345989025 conn(0x5609d4a3f300 msgr2=0x5609d4dea010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 -- 172.21.15.42:0/3345989025 shutdown_connections 2024-05-27T10:29:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.960+0000 7faba09dd740 1 -- 172.21.15.42:0/3345989025 wait complete. 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.961+0000 7faba09dd740 1 -- start start 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.961+0000 7faba09dd740 1 Processor -- start 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4df7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4e03cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5609d4a34790 con 0x5609d4a4b1b0 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5609d4cf6bc0 con 0x5609d4a222f0 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5609d4dfa930 con 0x5609d4a2a4d0 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7faba09dd740 10 client.0 _ll_register_callbacks cb 0x5609d4cd4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7fab8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4e03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7fab8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4e03cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35774/0 (socket says 172.21.15.42:35774) 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7fab8be00640 1 -- 172.21.15.42:0/2114222442 learned_addr learned my addr 172.21.15.42:0/2114222442 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7fab8be00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.962+0000 7fab92a00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4df7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab8be00640 1 -- 172.21.15.42:0/2114222442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 msgr2=0x5609d4df7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab8be00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4df7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab8be00640 1 -- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 msgr2=0x5609d4df7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab8be00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab8be00640 1 -- 172.21.15.42:0/2114222442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d4a63210 con 0x5609d4a4b1b0 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab8be00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4e03cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fab7c004190 tx=0x7fab7c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d667358bca3f8ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab80c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab80c00640 1 -- 172.21.15.42:0/2114222442 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c04e020 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7fab80c00640 1 -- 172.21.15.42:0/2114222442 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fab7c0046c0 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.963+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7c0038c0 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7fab80c00640 1 -- 172.21.15.42:0/2114222442 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab7c035040 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7faba09dd740 10 client.17046 fetch_fsmap learned FSMap version 39 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609d4b2f620 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7fab80c00640 1 -- 172.21.15.42:0/2114222442 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab7c0038c0 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7fab80c00640 1 -- 172.21.15.42:0/2114222442 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fab7c03a040 con 0x5609d4a4b1b0 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7faba09dd740 10 client.17046 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7faba09dd740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.964+0000 7faba09dd740 20 client.17046 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.965+0000 7faba09dd740 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d4a09d80 0x5609d4e09ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.965+0000 7faba09dd740 4 client.17046 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:29:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.965+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609d4d60ae0 con 0x5609d4a09d80 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.965+0000 7fab92a00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d4a09d80 0x5609d4e09ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.966+0000 7fab92a00640 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d4a09d80 0x5609d4e09ae0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c6ce005ce33b9d server_cookie=b76a3a8d60ae1ebb in_seq=0 out_seq=0 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.966+0000 7fab80c00640 10 client.17046 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7fab80c00640 1 -- 172.21.15.42:0/2114222442 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5609d4d60ae0 con 0x5609d4a09d80 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7fab80c00640 10 client.17046 handle_command_reply: tid=0 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 client.17046 shutdown 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 msgr2=0x5609d4e03cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4e03cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fab7c004190 tx=0x7fab7c005e30 comp rx=0 tx=0).stop 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 shutdown_connections 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4e03cc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d4a09d80 0x5609d4e09ae0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4df7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 --2- 172.21.15.42:0/2114222442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 >> 172.21.15.42:0/2114222442 conn(0x5609d4a3f300 msgr2=0x5609d4def580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.968+0000 7fab80c00640 0 client.17046 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.969+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 shutdown_connections 2024-05-27T10:29:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.969+0000 7faba09dd740 1 -- 172.21.15.42:0/2114222442 wait complete. 2024-05-27T10:29:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.969+0000 7faba09dd740 20 client.17046 trim_cache size 0 max 16384 2024-05-27T10:29:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.986+0000 7faba09dd740 1 Processor -- start 2024-05-27T10:29:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.986+0000 7faba09dd740 1 -- start start 2024-05-27T10:29:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7faba09dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7faba09dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4e03c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7faba09dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4df1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7faba09dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab941b44a0 con 0x5609d4a4b1b0 2024-05-27T10:29:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7faba09dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab941b1d40 con 0x5609d4a222f0 2024-05-27T10:29:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7faba09dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab941afbb0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4df1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4df1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35782/0 (socket says 172.21.15.42:35782) 2024-05-27T10:29:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab8be00640 1 -- 172.21.15.42:0/1360679355 learned_addr learned my addr 172.21.15.42:0/1360679355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab8be00640 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab8be00640 1 -- 172.21.15.42:0/1360679355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 msgr2=0x5609d4e03c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab92a00640 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4e03c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.987+0000 7fab8be00640 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4e03c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab8be00640 1 -- 172.21.15.42:0/1360679355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 msgr2=0x5609d4df3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab8be00640 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab8be00640 1 -- 172.21.15.42:0/1360679355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d4deec40 con 0x5609d4a4b1b0 2024-05-27T10:29:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab8be00640 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4df1480 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fab7c04c040 tx=0x7fab7c02a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b097624803100324 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab83400640 1 -- 172.21.15.42:0/1360679355 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c04e040 con 0x5609d4a4b1b0 2024-05-27T10:29:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab83400640 1 -- 172.21.15.42:0/1360679355 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab7c05d070 con 0x5609d4a4b1b0 2024-05-27T10:29:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.988+0000 7fab83400640 1 -- 172.21.15.42:0/1360679355 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c03d480 con 0x5609d4a4b1b0 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 -- 172.21.15.42:0/1360679355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 msgr2=0x5609d4df1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4df1480 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fab7c04c040 tx=0x7fab7c02a490 comp rx=0 tx=0).stop 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 -- 172.21.15.42:0/1360679355 shutdown_connections 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a4b1b0 0x5609d4df1480 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a2a4d0 0x5609d4e03c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 --2- 172.21.15.42:0/1360679355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a222f0 0x5609d4df3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 -- 172.21.15.42:0/1360679355 >> 172.21.15.42:0/1360679355 conn(0x5609d4a3f300 msgr2=0x5609d4df9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 -- 172.21.15.42:0/1360679355 shutdown_connections 2024-05-27T10:29:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.989+0000 7faba09dd740 1 -- 172.21.15.42:0/1360679355 wait complete. 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 -- start start 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 Processor -- start 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a222f0 0x5609d4e09c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4e0a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a4b1b0 0x5609d4e78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab941b0200 con 0x5609d4a2a4d0 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab941b1670 con 0x5609d4a4b1b0 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab941b08d0 con 0x5609d4a222f0 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.990+0000 7faba09dd740 10 client.0 _ll_register_callbacks cb 0x7fab8400b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4e0a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4e0a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35788/0 (socket says 172.21.15.42:35788) 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 -- 172.21.15.42:0/2420743324 learned_addr learned my addr 172.21.15.42:0/2420743324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:29:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab8be00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a222f0 0x5609d4e09c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab8be00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a4b1b0 0x5609d4e78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 -- 172.21.15.42:0/2420743324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a222f0 msgr2=0x5609d4e09c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a222f0 0x5609d4e09c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 -- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a4b1b0 msgr2=0x5609d4e78730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a4b1b0 0x5609d4e78730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 -- 172.21.15.42:0/2420743324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d4e0aaa0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab8be00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a4b1b0 0x5609d4e78730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab92a00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4e0a1c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x5609d4deb350 tx=0x7fab840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76def592f41ef7ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.991+0000 7fab80c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.992+0000 7fab80c00640 1 -- 172.21.15.42:0/2420743324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8401d940 con 0x5609d4a2a4d0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.992+0000 7fab80c00640 1 -- 172.21.15.42:0/2420743324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fab8401dae0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.992+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7c05d070 con 0x5609d4a2a4d0 2024-05-27T10:29:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.992+0000 7fab80c00640 1 -- 172.21.15.42:0/2420743324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab8402c050 con 0x5609d4a2a4d0 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.992+0000 7faba09dd740 10 client.17052 fetch_fsmap learned FSMap version 39 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.992+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609d4e79da0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7fab80c00640 1 -- 172.21.15.42:0/2420743324 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab7c05d070 con 0x5609d4a2a4d0 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7fab80c00640 1 -- 172.21.15.42:0/2420743324 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fab840103d0 con 0x5609d4a2a4d0 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7faba09dd740 10 client.17052 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7faba09dd740 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7faba09dd740 20 client.17052 populate_metadata read hostname 'smithi042' 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7faba09dd740 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d49df1c0 0x5609d4e7c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7faba09dd740 4 client.17052 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.993+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609d4d5a2c0 con 0x5609d49df1c0 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.994+0000 7fab8be00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d49df1c0 0x5609d4e7c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.995+0000 7fab80c00640 10 client.17052 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:46.995+0000 7fab8be00640 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d49df1c0 0x5609d4e7c5c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70205307f3d6d392 server_cookie=90fec4cc485be8e3 in_seq=0 out_seq=0 2024-05-27T10:29:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.046+0000 7fab80c00640 1 -- 172.21.15.42:0/2420743324 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5609d4d5a2c0 con 0x5609d49df1c0 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.046+0000 7fab80c00640 10 client.17052 handle_command_reply: tid=0 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.046+0000 7faba09dd740 1 client.17052 shutdown 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 msgr2=0x5609d4e0a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4e0a1c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x5609d4deb350 tx=0x7fab840118d0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 shutdown_connections 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609d4a4b1b0 0x5609d4e78730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d4a2a4d0 0x5609d4e0a1c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5609d49df1c0 0x5609d4e7c5c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 --2- 172.21.15.42:0/2420743324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5609d4a222f0 0x5609d4e09c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 >> 172.21.15.42:0/2420743324 conn(0x5609d4a3f300 msgr2=0x5609d4df90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7fab80c00640 0 client.17052 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:29:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 shutdown_connections 2024-05-27T10:29:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 1 -- 172.21.15.42:0/2420743324 wait complete. 2024-05-27T10:29:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.047+0000 7faba09dd740 20 client.17052 trim_cache size 0 max 16384 2024-05-27T10:29:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:29:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:29:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fab780b2950 msgr2=0x7fab780b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fab780b2950 0x7fab780b4e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0092f0 tx=0x7fab8c005b20 comp rx=0 tx=0).stop 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 msgr2=0x7fab9419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 0x7fab9419c870 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00eaa0 tx=0x7fab8c00ef60 comp rx=0 tx=0).stop 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 shutdown_connections 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fab780b2950 0x7fab780b4e60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab941104f0 0x7fab9419c870 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab9410db90 0x7fab941a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 --2- 172.21.15.42:0/579633375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab94076510 0x7fab941a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:29:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 >> 172.21.15.42:0/579633375 conn(0x7fab940ff340 msgr2=0x7fab94110a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:29:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 shutdown_connections 2024-05-27T10:29:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:29:47.048+0000 7fab99a00640 1 -- 172.21.15.42:0/579633375 wait complete. 2024-05-27T10:29:47.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:46 smithi104 ceph-mon[34812]: pgmap v343: 97 pgs: 97 active+clean; 427 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-05-27T10:29:47.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:46 smithi143 ceph-mon[35545]: pgmap v343: 97 pgs: 97 active+clean; 427 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-05-27T10:29:47.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:46 smithi042 ceph-mon[31661]: pgmap v343: 97 pgs: 97 active+clean; 427 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-05-27T10:29:49.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:48 smithi143 ceph-mon[35545]: pgmap v344: 97 pgs: 97 active+clean; 427 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 731 KiB/s wr, 86 op/s 2024-05-27T10:29:49.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:48 smithi104 ceph-mon[34812]: pgmap v344: 97 pgs: 97 active+clean; 427 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 731 KiB/s wr, 86 op/s 2024-05-27T10:29:49.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:48 smithi042 ceph-mon[31661]: pgmap v344: 97 pgs: 97 active+clean; 427 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 731 KiB/s wr, 86 op/s 2024-05-27T10:29:51.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:50 smithi104 ceph-mon[34812]: pgmap v345: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-27T10:29:51.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:50 smithi143 ceph-mon[35545]: pgmap v345: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-27T10:29:51.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:50 smithi042 ceph-mon[31661]: pgmap v345: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-27T10:29:53.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:52 smithi104 ceph-mon[34812]: pgmap v346: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 661 KiB/s wr, 107 op/s 2024-05-27T10:29:53.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:52 smithi143 ceph-mon[35545]: pgmap v346: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 661 KiB/s wr, 107 op/s 2024-05-27T10:29:53.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:52 smithi042 ceph-mon[31661]: pgmap v346: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 661 KiB/s wr, 107 op/s 2024-05-27T10:29:55.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:54 smithi143 ceph-mon[35545]: pgmap v347: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 661 KiB/s wr, 107 op/s 2024-05-27T10:29:55.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:54 smithi042 ceph-mon[31661]: pgmap v347: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 661 KiB/s wr, 107 op/s 2024-05-27T10:29:55.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:54 smithi104 ceph-mon[34812]: pgmap v347: 97 pgs: 97 active+clean; 415 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 661 KiB/s wr, 107 op/s 2024-05-27T10:29:56.727 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-05-27T10:29:56.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-27T10:29:57.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:56 smithi042 ceph-mon[31661]: pgmap v348: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 954 KiB/s wr, 155 op/s 2024-05-27T10:29:57.226 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:56 smithi104 ceph-mon[34812]: pgmap v348: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 954 KiB/s wr, 155 op/s 2024-05-27T10:29:57.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:56 smithi143 ceph-mon[35545]: pgmap v348: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 954 KiB/s wr, 155 op/s 2024-05-27T10:29:57.937 DEBUG:teuthology.parallel:result is None 2024-05-27T10:29:57.938 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T10:29:58.049 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T10:29:58.049 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-27T10:29:58.059 INFO:tasks.quiescer:joining Quiescers 2024-05-27T10:29:58.060 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-27T10:29:58.060 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T10:29:58.558 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:29:58.960 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:29:58 smithi042 ceph-mon[31661]: pgmap v349: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 660 KiB/s wr, 110 op/s 2024-05-27T10:29:59.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:29:58 smithi143 ceph-mon[35545]: pgmap v349: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 660 KiB/s wr, 110 op/s 2024-05-27T10:29:59.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:29:58 smithi104 ceph-mon[34812]: pgmap v349: 97 pgs: 97 active+clean; 419 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 660 KiB/s wr, 110 op/s 2024-05-27T10:30:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 -- 172.21.15.42:0/1597486619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e010d9c0 msgr2=0x7f78e010fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 --2- 172.21.15.42:0/1597486619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e010d9c0 0x7f78e010fde0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c802a480 comp rx=0 tx=0).stop 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 -- 172.21.15.42:0/1597486619 shutdown_connections 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 --2- 172.21.15.42:0/1597486619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0110320 0x7f78e0112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 --2- 172.21.15.42:0/1597486619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e010d9c0 0x7f78e010fde0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 --2- 172.21.15.42:0/1597486619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78e0075420 0x7f78e0075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 -- 172.21.15.42:0/1597486619 >> 172.21.15.42:0/1597486619 conn(0x7f78e0079100 msgr2=0x7f78e0079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 -- 172.21.15.42:0/1597486619 shutdown_connections 2024-05-27T10:30:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.292+0000 7f78e5a00640 1 -- 172.21.15.42:0/1597486619 wait complete. 2024-05-27T10:30:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.294+0000 7f78e5a00640 1 Processor -- start 2024-05-27T10:30:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.294+0000 7f78e5a00640 1 -- start start 2024-05-27T10:30:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78e5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 0x7f78e019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78e5a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78e010d9c0 0x7f78e019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78e5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0110320 0x7f78e01a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78e5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0115d40 con 0x7f78e0075420 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78e5a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0115bc0 con 0x7f78e010d9c0 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78e5a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0115ec0 con 0x7f78e0110320 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 0x7f78e019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 0x7f78e019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51700/0 (socket says 172.21.15.42:51700) 2024-05-27T10:30:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.295+0000 7f78dfe00640 1 -- 172.21.15.42:0/3594292082 learned_addr learned my addr 172.21.15.42:0/3594292082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78e5000640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0110320 0x7f78e01a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78d7e00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78e010d9c0 0x7f78e019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78dfe00640 1 -- 172.21.15.42:0/3594292082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0110320 msgr2=0x7f78e01a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78dfe00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0110320 0x7f78e01a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78dfe00640 1 -- 172.21.15.42:0/3594292082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78e010d9c0 msgr2=0x7f78e019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78dfe00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78e010d9c0 0x7f78e019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78dfe00640 1 -- 172.21.15.42:0/3594292082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e01a1660 con 0x7f78e0075420 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.296+0000 7f78dfe00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 0x7f78e019c800 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f78d000e9f0 tx=0x7f78d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.297+0000 7f78dd600640 1 -- 172.21.15.42:0/3594292082 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d000cd80 con 0x7f78e0075420 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.297+0000 7f78dd600640 1 -- 172.21.15.42:0/3594292082 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d00108c0 con 0x7f78e0075420 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.297+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01a1950 con 0x7f78e0075420 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.297+0000 7f78dd600640 1 -- 172.21.15.42:0/3594292082 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00187c0 con 0x7f78e0075420 2024-05-27T10:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.297+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01a92d0 con 0x7f78e0075420 2024-05-27T10:30:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.300+0000 7f78edf65740 1 Processor -- start 2024-05-27T10:30:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 -- start start 2024-05-27T10:30:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf349a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef7e2f0 0x55eecf349fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf34a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eecf2bab90 con 0x55eecefa71b0 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eecf2aa730 con 0x55eecef7e2f0 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78edf65740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eecf2ae770 con 0x55eecef864d0 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78e5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf349a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78e5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf349a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51708/0 (socket says 172.21.15.42:51708) 2024-05-27T10:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.301+0000 7f78e5000640 1 -- 172.21.15.42:0/4159526943 learned_addr learned my addr 172.21.15.42:0/4159526943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78d7e00640 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf34a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78e5000640 1 -- 172.21.15.42:0/4159526943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 msgr2=0x55eecf34a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78e5000640 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf34a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78e5000640 1 -- 172.21.15.42:0/4159526943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef7e2f0 msgr2=0x55eecf349fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T10:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78e5000640 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef7e2f0 0x55eecf349fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78e5000640 1 -- 172.21.15.42:0/4159526943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eecf2438b0 con 0x55eecefa71b0 2024-05-27T10:30:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.302+0000 7f78e5000640 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf349a60 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009e60 tx=0x7f78d8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37f8a312697ca79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.303+0000 7f78d5600640 1 -- 172.21.15.42:0/4159526943 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8016070 con 0x55eecefa71b0 2024-05-27T10:30:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.303+0000 7f78d5600640 1 -- 172.21.15.42:0/4159526943 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d8004d20 con 0x55eecefa71b0 2024-05-27T10:30:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.303+0000 7f78d5600640 1 -- 172.21.15.42:0/4159526943 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80126b0 con 0x55eecefa71b0 2024-05-27T10:30:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.303+0000 7f78dd600640 1 -- 172.21.15.42:0/3594292082 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f78d0010430 con 0x7f78e0075420 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 -- 172.21.15.42:0/4159526943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 msgr2=0x55eecf349a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf349a60 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009e60 tx=0x7f78d8010740 comp rx=0 tx=0).stop 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 -- 172.21.15.42:0/4159526943 shutdown_connections 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf34a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef7e2f0 0x55eecf349fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 --2- 172.21.15.42:0/4159526943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf349a60 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 -- 172.21.15.42:0/4159526943 >> 172.21.15.42:0/4159526943 conn(0x55eecef9b300 msgr2=0x55eecf346210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 -- 172.21.15.42:0/4159526943 shutdown_connections 2024-05-27T10:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.304+0000 7f78edf65740 1 -- 172.21.15.42:0/4159526943 wait complete. 2024-05-27T10:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.305+0000 7f78edf65740 1 -- start start 2024-05-27T10:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.305+0000 7f78edf65740 1 Processor -- start 2024-05-27T10:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.305+0000 7f78dd600640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78c40b28e0 0x7f78c40b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78dd600640 1 -- 172.21.15.42:0/3594292082 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f78d00e2a90 con 0x7f78e0075420 2024-05-27T10:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78e5000640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78c40b28e0 0x7f78c40b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf353e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef864d0 0x55eecf3543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf35fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eecef90790 con 0x55eecefa71b0 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eecf252bc0 con 0x55eecef864d0 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eecf356b90 con 0x55eecef7e2f0 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.306+0000 7f78edf65740 10 client.0 _ll_register_callbacks cb 0x55eecf230400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf35fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf35fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51724/0 (socket says 172.21.15.42:51724) 2024-05-27T10:30:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 -- 172.21.15.42:0/1931863478 learned_addr learned my addr 172.21.15.42:0/1931863478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78dfe00640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf353e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78d7e00640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef864d0 0x55eecf3543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78c40b28e0 0x7f78c40b4df0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78d8011010 tx=0x7f78d8005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 -- 172.21.15.42:0/1931863478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 msgr2=0x55eecf353e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf353e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 -- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef864d0 msgr2=0x55eecf3543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef864d0 0x55eecf3543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.307+0000 7f78e5000640 1 -- 172.21.15.42:0/1931863478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eecefbf210 con 0x55eecefa71b0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78e5000640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf35fec0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f78d8004cf0 tx=0x7f78d8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaade9b4351c1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78ce000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78ce000640 1 -- 172.21.15.42:0/1931863478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8035060 con 0x55eecefa71b0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78ce000640 1 -- 172.21.15.42:0/1931863478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f78d8020030 con 0x55eecefa71b0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d8004d20 con 0x55eecefa71b0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78ce000640 1 -- 172.21.15.42:0/1931863478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78d8016050 con 0x55eecefa71b0 2024-05-27T10:30:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78edf65740 10 client.17064 fetch_fsmap learned FSMap version 39 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.308+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eecf08b620 con 0x55eecefa71b0 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78ce000640 1 -- 172.21.15.42:0/1931863478 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d8004d20 con 0x55eecefa71b0 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78ce000640 1 -- 172.21.15.42:0/1931863478 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f78d800f040 con 0x55eecefa71b0 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78edf65740 10 client.17064 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78edf65740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78edf65740 20 client.17064 populate_metadata read hostname 'smithi042' 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78edf65740 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef65d80 0x55eecf365dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78edf65740 4 client.17064 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:30:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.309+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eecf2bcae0 con 0x55eecef65d80 2024-05-27T10:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.310+0000 7f78dfe00640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef65d80 0x55eecf365dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.311+0000 7f78dfe00640 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef65d80 0x55eecf365dc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b193f01788a230a2 server_cookie=726fb7f2bf6e42df in_seq=0 out_seq=0 2024-05-27T10:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.311+0000 7f78ce000640 10 client.17064 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78ce000640 1 -- 172.21.15.42:0/1931863478 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eecf2bcae0 con 0x55eecef65d80 2024-05-27T10:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78ce000640 10 client.17064 handle_command_reply: tid=0 2024-05-27T10:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 client.17064 shutdown 2024-05-27T10:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 msgr2=0x55eecf35fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf35fec0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f78d8004cf0 tx=0x7f78d8010740 comp rx=0 tx=0).stop 2024-05-27T10:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 shutdown_connections 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecefa71b0 0x55eecf35fec0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecef864d0 0x55eecf3543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef65d80 0x55eecf365dc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 --2- 172.21.15.42:0/1931863478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf353e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 >> 172.21.15.42:0/1931863478 conn(0x55eecef9b300 msgr2=0x55eecf349830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.314+0000 7f78ce000640 0 client.17064 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.315+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 shutdown_connections 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.315+0000 7f78edf65740 1 -- 172.21.15.42:0/1931863478 wait complete. 2024-05-27T10:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.315+0000 7f78edf65740 20 client.17064 trim_cache size 0 max 16384 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 Processor -- start 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 -- start start 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 0x55eecf349f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf35fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf360170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01b43f0 con 0x55eecef7e2f0 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01b1c90 con 0x55eecefa71b0 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.334+0000 7f78edf65740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01afb00 con 0x55eecef864d0 2024-05-27T10:30:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 0x55eecf349f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 0x55eecf349f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51736/0 (socket says 172.21.15.42:51736) 2024-05-27T10:30:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 -- 172.21.15.42:0/2222477054 learned_addr learned my addr 172.21.15.42:0/2222477054 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78e5000640 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf360170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78d7e00640 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf35fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 -- 172.21.15.42:0/2222477054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 msgr2=0x55eecf35fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf35fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 -- 172.21.15.42:0/2222477054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 msgr2=0x55eecf360170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf360170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.335+0000 7f78dfe00640 1 -- 172.21.15.42:0/2222477054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eecf3521b0 con 0x55eecef7e2f0 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.336+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 0x55eecf349f00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f78d00a9a30 tx=0x7f78d0010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225fd8a37b90f052 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.336+0000 7f78d4c00640 1 -- 172.21.15.42:0/2222477054 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00b2b90 con 0x55eecef7e2f0 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.336+0000 7f78d4c00640 1 -- 172.21.15.42:0/2222477054 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d00c0070 con 0x55eecef7e2f0 2024-05-27T10:30:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.336+0000 7f78d4c00640 1 -- 172.21.15.42:0/2222477054 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00a56a0 con 0x55eecef7e2f0 2024-05-27T10:30:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 -- 172.21.15.42:0/2222477054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 msgr2=0x55eecf349f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 0x55eecf349f00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f78d00a9a30 tx=0x7f78d0010cc0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 -- 172.21.15.42:0/2222477054 shutdown_connections 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf360170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef864d0 0x55eecf35fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 --2- 172.21.15.42:0/2222477054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef7e2f0 0x55eecf349f00 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 -- 172.21.15.42:0/2222477054 >> 172.21.15.42:0/2222477054 conn(0x55eecef9b300 msgr2=0x55eecf3497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 -- 172.21.15.42:0/2222477054 shutdown_connections 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.337+0000 7f78edf65740 1 -- 172.21.15.42:0/2222477054 wait complete. 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.338+0000 7f78edf65740 1 -- start start 2024-05-27T10:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.338+0000 7f78edf65740 1 Processor -- start 2024-05-27T10:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf358e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 0x55eecf359390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf3598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01b0150 con 0x55eecef864d0 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01b15c0 con 0x55eecefa71b0 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01b0820 con 0x55eecef7e2f0 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.339+0000 7f78edf65740 10 client.0 _ll_register_callbacks cb 0x7f78d8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 0x55eecf359390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 0x55eecf359390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51738/0 (socket says 172.21.15.42:51738) 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 -- 172.21.15.42:0/2034800070 learned_addr learned my addr 172.21.15.42:0/2034800070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf358e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78e5000640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf3598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 -- 172.21.15.42:0/2034800070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 msgr2=0x55eecf358e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf358e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 -- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 msgr2=0x55eecf3598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf3598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78d7e00640 1 -- 172.21.15.42:0/2034800070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eecf3cc4f0 con 0x55eecef864d0 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.340+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf358e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.341+0000 7f78d7e00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 0x55eecf359390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f78c8005ab0 tx=0x7f78c80316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa55e3ac506d6c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.341+0000 7f78ce000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.341+0000 7f78ce000640 1 -- 172.21.15.42:0/2034800070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8035040 con 0x55eecef864d0 2024-05-27T10:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.341+0000 7f78ce000640 1 -- 172.21.15.42:0/2034800070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f78c8004240 con 0x55eecef864d0 2024-05-27T10:30:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.341+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d00c0070 con 0x55eecef864d0 2024-05-27T10:30:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.342+0000 7f78ce000640 1 -- 172.21.15.42:0/2034800070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78c8004440 con 0x55eecef864d0 2024-05-27T10:30:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.342+0000 7f78edf65740 10 client.17070 fetch_fsmap learned FSMap version 39 2024-05-27T10:30:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.342+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eecf3cd780 con 0x55eecef864d0 2024-05-27T10:30:00.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31657]: 2024-05-27T10:29:59.998+0000 7fd90f000700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-27T10:30:00.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31657]: 2024-05-27T10:29:59.998+0000 7fd90f000700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-27T10:30:00.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31657]: 2024-05-27T10:29:59.998+0000 7fd90f000700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2024-05-27T10:30:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.343+0000 7f78ce000640 1 -- 172.21.15.42:0/2034800070 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d00c0070 con 0x55eecef864d0 2024-05-27T10:30:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78ce000640 1 -- 172.21.15.42:0/2034800070 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f78c8004c60 con 0x55eecef864d0 2024-05-27T10:30:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78edf65740 10 client.17070 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T10:30:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78edf65740 10 client.17070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T10:30:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78edf65740 20 client.17070 populate_metadata read hostname 'smithi042' 2024-05-27T10:30:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78edf65740 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef3b1c0 0x55eecf3cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78edf65740 4 client.17070 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T10:30:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.344+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eecf2b62c0 con 0x55eecef3b1c0 2024-05-27T10:30:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.345+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef3b1c0 0x55eecf3cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.346+0000 7f78ce000640 10 client.17070 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.346+0000 7f78dfe00640 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef3b1c0 0x55eecf3cff40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4566013d424c7f4 server_cookie=e23135b310b7d8e5 in_seq=0 out_seq=0 2024-05-27T10:30:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78ce000640 1 -- 172.21.15.42:0/2034800070 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eecf2b62c0 con 0x55eecef3b1c0 2024-05-27T10:30:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78ce000640 10 client.17070 handle_command_reply: tid=0 2024-05-27T10:30:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 client.17070 shutdown 2024-05-27T10:30:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 msgr2=0x55eecf359390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 0x55eecf359390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f78c8005ab0 tx=0x7f78c80316a0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 shutdown_connections 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eecefa71b0 0x55eecf3598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eecef864d0 0x55eecf359390 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x55eecef3b1c0 0x55eecf3cff40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 --2- 172.21.15.42:0/2034800070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eecef7e2f0 0x55eecf358e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 >> 172.21.15.42:0/2034800070 conn(0x55eecef9b300 msgr2=0x55eecf34e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.387+0000 7f78ce000640 0 client.17070 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.388+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 shutdown_connections 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.388+0000 7f78edf65740 1 -- 172.21.15.42:0/2034800070 wait complete. 2024-05-27T10:30:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.388+0000 7f78edf65740 20 client.17070 trim_cache size 0 max 16384 2024-05-27T10:30:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-05-27T10:30:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-05-27T10:30:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78c40b28e0 msgr2=0x7f78c40b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78c40b28e0 0x7f78c40b4df0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78d8011010 tx=0x7f78d8005fd0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 msgr2=0x7f78e019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 0x7f78e019c800 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f78d000e9f0 tx=0x7f78d000eeb0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 shutdown_connections 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f78c40b28e0 0x7f78c40b4df0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0110320 0x7f78e01a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78e010d9c0 0x7f78e019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 --2- 172.21.15.42:0/3594292082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78e0075420 0x7f78e019c800 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 >> 172.21.15.42:0/3594292082 conn(0x7f78e0079100 msgr2=0x7f78e01022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 shutdown_connections 2024-05-27T10:30:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-05-27T10:30:00.389+0000 7f78e5a00640 1 -- 172.21.15.42:0/3594292082 wait complete. 2024-05-27T10:30:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.124+0000 7fd7d3e00700 1 -- 172.21.15.42:0/4126814373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 msgr2=0x7fd7d410abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.124+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/4126814373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d410abc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009990 tx=0x7fd7c0009ca0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.125 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-mon[31661]: pgmap v350: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 937 KiB/s wr, 164 op/s 2024-05-27T10:30:01.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-mon[31661]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-27T10:30:01.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-mon[31661]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-27T10:30:01.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:00 smithi042 ceph-mon[31661]: mds.h(mds.1): Metadata damage detected 2024-05-27T10:30:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 -- 172.21.15.42:0/4126814373 shutdown_connections 2024-05-27T10:30:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/4126814373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d410b100 0x7fd7d4117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/4126814373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d410f6b0 0x7fd7d410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/4126814373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d410abc0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 -- 172.21.15.42:0/4126814373 >> 172.21.15.42:0/4126814373 conn(0x7fd7d4076af0 msgr2=0x7fd7d4076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 -- 172.21.15.42:0/4126814373 shutdown_connections 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.125+0000 7fd7d3e00700 1 -- 172.21.15.42:0/4126814373 wait complete. 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.126+0000 7fd7d3e00700 1 Processor -- start 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.126+0000 7fd7d3e00700 1 -- start start 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d410b100 0x7fd7d41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d410f6b0 0x7fd7d41a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d411a7a0 con 0x7fd7d410a7b0 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d411a620 con 0x7fd7d410f6b0 2024-05-27T10:30:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d411a920 con 0x7fd7d410b100 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d2a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d41a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51750/0 (socket says 172.21.15.42:51750) 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d2a00700 1 -- 172.21.15.42:0/3618454791 learned_addr learned my addr 172.21.15.42:0/3618454791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d2000700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d410b100 0x7fd7d41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.127+0000 7fd7d3400700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d410f6b0 0x7fd7d41a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7d2a00700 1 -- 172.21.15.42:0/3618454791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d410b100 msgr2=0x7fd7d41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7d2a00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d410b100 0x7fd7d41a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7d2a00700 1 -- 172.21.15.42:0/3618454791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d410f6b0 msgr2=0x7fd7d41a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:01.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7d2a00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d410f6b0 0x7fd7d41a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7d2a00700 1 -- 172.21.15.42:0/3618454791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d41a6040 con 0x7fd7d410a7b0 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7d2a00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d41a1170 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400d950 tx=0x7fd7c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.128+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4016070 con 0x7fd7d410a7b0 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.129+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7c4009dd0 con 0x7fd7d410a7b0 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.129+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400f3c0 con 0x7fd7d410a7b0 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.129+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d41a6330 con 0x7fd7d410a7b0 2024-05-27T10:30:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.129+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41adc50 con 0x7fd7d410a7b0 2024-05-27T10:30:01.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.130+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d40512b0 con 0x7fd7d410a7b0 2024-05-27T10:30:01.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.134+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd7c4019070 con 0x7fd7d410a7b0 2024-05-27T10:30:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.136+0000 7fd7cb400700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7b80b2a70 0x7fd7b80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.136+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fd7c40e21a0 con 0x7fd7d410a7b0 2024-05-27T10:30:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.136+0000 7fd7d2000700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7b80b2a70 0x7fd7b80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.137+0000 7fd7d2000700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7b80b2a70 0x7fd7b80b4f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4077ff0 tx=0x7fd7c0005fd0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.139+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7c40a9b40 con 0x7fd7d410a7b0 2024-05-27T10:30:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:00 smithi143 ceph-mon[35545]: pgmap v350: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 937 KiB/s wr, 164 op/s 2024-05-27T10:30:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:00 smithi143 ceph-mon[35545]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-27T10:30:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:00 smithi143 ceph-mon[35545]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-27T10:30:01.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:00 smithi143 ceph-mon[35545]: mds.h(mds.1): Metadata damage detected 2024-05-27T10:30:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.434+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7d402e5e0 con 0x7fd7d410a7b0 2024-05-27T10:30:01.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.437+0000 7fd7cb400700 1 -- 172.21.15.42:0/3618454791 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11877 (secure 0 0 0) 0x7fd7c40a9440 con 0x7fd7d410a7b0 2024-05-27T10:30:01.436 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:30:01.437 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/781814611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":781814611},{"type":"v1","addr":"172.21.15.42:6835","nonce":781814611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1839056289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1839056289},{"type":"v1","addr":"172.21.15.42:6837","nonce":1839056289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/4181502432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":4181502432},{"type":"v1","addr":"172.21.15.42:6839","nonce":4181502432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2889621451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2889621451},{"type":"v1","addr":"172.21.15.42:6841","nonce":2889621451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3395724894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3395724894},{"type":"v1","addr":"172.21.15.143:6833","nonce":3395724894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3892720784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3892720784},{"type":"v1","addr":"172.21.15.143:6835","nonce":3892720784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T10:14:18.127208+0000","modified":"2024-05-27T10:29:16.068172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34466,"mds_2":34442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/4166809554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4166809554},{"type":"v1","addr":"172.21.15.143:6837","nonce":4166809554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/517039305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":517039305},{"type":"v1","addr":"172.21.15.143:6839","nonce":517039305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.104:6835/3501163811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3501163811},{"type":"v1","addr":"172.21.15.104:6835","nonce":3501163811}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":126,"addr":"172.21.15.104:6837/219118322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":219118322},{"type":"v1","addr":"172.21.15.104:6837","nonce":219118322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6839/3864142797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3864142797},{"type":"v1","addr":"172.21.15.104:6839","nonce":3864142797}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_36359":{"gid":36359,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/755631209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":755631209},{"type":"v1","addr":"172.21.15.104:6833","nonce":755631209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34499,34442,34466]},"id":1}]} 2024-05-27T10:30:01.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7b80b2a70 msgr2=0x7fd7b80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:01.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7b80b2a70 0x7fd7b80b4f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4077ff0 tx=0x7fd7c0005fd0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 msgr2=0x7fd7d41a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:01.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d41a1170 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400d950 tx=0x7fd7c400dc60 comp rx=0 tx=0).stop 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 shutdown_connections 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd7b80b2a70 0x7fd7b80b4f80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d410f6b0 0x7fd7d41a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7d410a7b0 0x7fd7d41a1170 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 --2- 172.21.15.42:0/3618454791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d410b100 0x7fd7d41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 >> 172.21.15.42:0/3618454791 conn(0x7fd7d4076af0 msgr2=0x7fd7d41088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:01.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.440+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 shutdown_connections 2024-05-27T10:30:01.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:01.441+0000 7fd7d3e00700 1 -- 172.21.15.42:0/3618454791 wait complete. 2024-05-27T10:30:01.441 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 39 2024-05-27T10:30:01.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:00 smithi104 ceph-mon[34812]: pgmap v350: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 937 KiB/s wr, 164 op/s 2024-05-27T10:30:01.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:00 smithi104 ceph-mon[34812]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-27T10:30:01.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:00 smithi104 ceph-mon[34812]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-27T10:30:01.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:00 smithi104 ceph-mon[34812]: mds.h(mds.1): Metadata damage detected 2024-05-27T10:30:02.209 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:01 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/3618454791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:30:02.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:01 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/3618454791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:30:02.410 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34499 quiesce db --cancel --all' 2024-05-27T10:30:02.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.34499 quiesce db --cancel --all 2024-05-27T10:30:02.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:01 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/3618454791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T10:30:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:30:03.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:02 smithi143 ceph-mon[35545]: pgmap v351: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 570 KiB/s wr, 101 op/s 2024-05-27T10:30:03.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:02 smithi143 ceph-mon[35545]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:30:03.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:02 smithi042 ceph-mon[31661]: pgmap v351: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 570 KiB/s wr, 101 op/s 2024-05-27T10:30:03.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:02 smithi042 ceph-mon[31661]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:30:03.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:02 smithi104 ceph-mon[34812]: pgmap v351: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 570 KiB/s wr, 101 op/s 2024-05-27T10:30:03.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:02 smithi104 ceph-mon[34812]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T10:30:05.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:04 smithi042 ceph-mon[31661]: pgmap v352: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 570 KiB/s wr, 101 op/s 2024-05-27T10:30:05.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:04 smithi143 ceph-mon[35545]: pgmap v352: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 570 KiB/s wr, 101 op/s 2024-05-27T10:30:05.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:04 smithi104 ceph-mon[34812]: pgmap v352: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 570 KiB/s wr, 101 op/s 2024-05-27T10:30:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.544+0000 7fd073e00700 1 -- 172.21.15.42:0/2028252790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c10c5a0 msgr2=0x7fd06c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.544+0000 7fd073e00700 1 --2- 172.21.15.42:0/2028252790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c10c5a0 0x7fd06c10c9b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098f0 tx=0x7fd05c009c00 comp rx=0 tx=0).stop 2024-05-27T10:30:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.545+0000 7fd073e00700 1 -- 172.21.15.42:0/2028252790 shutdown_connections 2024-05-27T10:30:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.545+0000 7fd073e00700 1 --2- 172.21.15.42:0/2028252790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106eb0 0x7fd06c113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.545+0000 7fd073e00700 1 --2- 172.21.15.42:0/2028252790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c10c5a0 0x7fd06c10c9b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.545+0000 7fd073e00700 1 --2- 172.21.15.42:0/2028252790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c106560 0x7fd06c106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.545+0000 7fd073e00700 1 -- 172.21.15.42:0/2028252790 >> 172.21.15.42:0/2028252790 conn(0x7fd06c102040 msgr2=0x7fd06c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:05.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.546+0000 7fd073e00700 1 -- 172.21.15.42:0/2028252790 shutdown_connections 2024-05-27T10:30:05.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.546+0000 7fd073e00700 1 -- 172.21.15.42:0/2028252790 wait complete. 2024-05-27T10:30:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.548+0000 7fd073e00700 1 Processor -- start 2024-05-27T10:30:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.548+0000 7fd073e00700 1 -- start start 2024-05-27T10:30:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.549+0000 7fd073e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106560 0x7fd06c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd073e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c106eb0 0x7fd06c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd073e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 0x7fd06c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd073e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c116600 con 0x7fd06c106560 2024-05-27T10:30:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd073e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c116480 con 0x7fd06c10c5a0 2024-05-27T10:30:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd073e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c116780 con 0x7fd06c106eb0 2024-05-27T10:30:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd071e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 0x7fd06c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.551+0000 7fd071400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106560 0x7fd06c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.552+0000 7fd071e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 0x7fd06c1a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:55856/0 (socket says 172.21.15.42:55856) 2024-05-27T10:30:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.552+0000 7fd071e00700 1 -- 172.21.15.42:0/1318789266 learned_addr learned my addr 172.21.15.42:0/1318789266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.552+0000 7fd070a00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c106eb0 0x7fd06c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd070a00700 1 -- 172.21.15.42:0/1318789266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 msgr2=0x7fd06c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd070a00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 0x7fd06c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd070a00700 1 -- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106560 msgr2=0x7fd06c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd070a00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106560 0x7fd06c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd070a00700 1 -- 172.21.15.42:0/1318789266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c1a1ea0 con 0x7fd06c106eb0 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd071400700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106560 0x7fd06c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.553+0000 7fd071e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 0x7fd06c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T10:30:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.554+0000 7fd070a00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c106eb0 0x7fd06c19d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd06c075b70 tx=0x7fd05800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:05.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.555+0000 7fd06a000700 1 -- 172.21.15.42:0/1318789266 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800cbd0 con 0x7fd06c106eb0 2024-05-27T10:30:05.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.555+0000 7fd06a000700 1 -- 172.21.15.42:0/1318789266 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd05800ee80 con 0x7fd06c106eb0 2024-05-27T10:30:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.555+0000 7fd06a000700 1 -- 172.21.15.42:0/1318789266 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058018680 con 0x7fd06c106eb0 2024-05-27T10:30:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.555+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1a2190 con 0x7fd06c106eb0 2024-05-27T10:30:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.555+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1a9ab0 con 0x7fd06c106eb0 2024-05-27T10:30:05.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.567+0000 7fd06a000700 1 -- 172.21.15.42:0/1318789266 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd058010430 con 0x7fd06c106eb0 2024-05-27T10:30:05.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.567+0000 7fd0828deb80 1 Processor -- start 2024-05-27T10:30:05.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.568+0000 7fd0828deb80 1 -- start start 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436dd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436d7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd0828deb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5614435fbeb0 con 0x56144354eeb0 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd0828deb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5614435eba50 con 0x56144363aa60 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd0828deb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5614435efa90 con 0x5614436d6d60 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd06a000700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0500b2c00 0x7fd0500b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd071400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd06a000700 1 -- 172.21.15.42:0/1318789266 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7fd0580e2570 con 0x7fd06c106eb0 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.571+0000 7fd071400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436dd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36708/0 (socket says 172.21.15.42:36708) 2024-05-27T10:30:05.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.572+0000 7fd071400700 1 -- 172.21.15.42:0/3313599146 learned_addr learned my addr 172.21.15.42:0/3313599146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:05.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.572+0000 7fd071e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0500b2c00 0x7fd0500b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.572+0000 7fd071e00700 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.572+0000 7fd070a00700 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071400700 1 -- 172.21.15.42:0/3313599146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 msgr2=0x5614436d7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071400700 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436d7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071400700 1 -- 172.21.15.42:0/3313599146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 msgr2=0x5614436dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071400700 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071400700 1 -- 172.21.15.42:0/3313599146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614435b4490 con 0x56144354eeb0 2024-05-27T10:30:05.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071e00700 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:30:05.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.573+0000 7fd071400700 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436dd210 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098f0 tx=0x7fd05c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcf4dced2c130e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:05.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.574+0000 7fd071e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0500b2c00 0x7fd0500b5110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd064009e10 tx=0x7fd0640099e0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:05.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.575+0000 7fd062a00700 1 -- 172.21.15.42:0/3313599146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c027070 con 0x56144354eeb0 2024-05-27T10:30:05.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.575+0000 7fd062a00700 1 -- 172.21.15.42:0/3313599146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd05c00fe20 con 0x56144354eeb0 2024-05-27T10:30:05.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.575+0000 7fd062a00700 1 -- 172.21.15.42:0/3313599146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c027070 con 0x56144354eeb0 2024-05-27T10:30:05.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.576+0000 7fd0828deb80 1 -- 172.21.15.42:0/3313599146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 msgr2=0x5614436dd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.576+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436dd210 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098f0 tx=0x7fd05c00fc30 comp rx=0 tx=0).stop 2024-05-27T10:30:05.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 -- 172.21.15.42:0/3313599146 shutdown_connections 2024-05-27T10:30:05.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436d7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3313599146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436dd210 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 -- 172.21.15.42:0/3313599146 >> 172.21.15.42:0/3313599146 conn(0x5614435e8400 msgr2=0x5614436c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:05.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 -- 172.21.15.42:0/3313599146 shutdown_connections 2024-05-27T10:30:05.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.577+0000 7fd0828deb80 1 -- 172.21.15.42:0/3313599146 wait complete. 2024-05-27T10:30:05.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.578+0000 7fd0828deb80 1 -- start start 2024-05-27T10:30:05.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.578+0000 7fd0828deb80 1 Processor -- start 2024-05-27T10:30:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436e1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5614434f2010 con 0x56144354eeb0 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56144358c7d0 con 0x56144363aa60 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5614436d7af0 con 0x5614436d6d60 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.579+0000 7fd0828deb80 10 client.0 _ll_register_callbacks cb 0x561443595ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.580+0000 7fd071400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.580+0000 7fd071400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36718/0 (socket says 172.21.15.42:36718) 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.580+0000 7fd071400700 1 -- 172.21.15.42:0/3875961068 learned_addr learned my addr 172.21.15.42:0/3875961068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.580+0000 7fd070a00700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.580+0000 7fd071e00700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.580+0000 7fd071400700 1 -- 172.21.15.42:0/3875961068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 msgr2=0x5614436e1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.581+0000 7fd071400700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436e1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.581+0000 7fd071400700 1 -- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 msgr2=0x5614436e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.581+0000 7fd071400700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.581+0000 7fd071400700 1 -- 172.21.15.42:0/3875961068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56144353b940 con 0x56144354eeb0 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.581+0000 7fd071400700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436e0990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098c0 tx=0x7fd05c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1748d83a81b4c9f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.581+0000 7fd057400700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-27T10:30:05.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.582+0000 7fd057400700 1 -- 172.21.15.42:0/3875961068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c027070 con 0x56144354eeb0 2024-05-27T10:30:05.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.582+0000 7fd057400700 1 -- 172.21.15.42:0/3875961068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7fd05c0050b0 con 0x56144354eeb0 2024-05-27T10:30:05.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.582+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c00fe20 con 0x56144354eeb0 2024-05-27T10:30:05.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.582+0000 7fd057400700 1 -- 172.21.15.42:0/3875961068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd05c0052b0 con 0x56144354eeb0 2024-05-27T10:30:05.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.583+0000 7fd0828deb80 10 client.17088 fetch_fsmap learned FSMap version 40 2024-05-27T10:30:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.583+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5614435bc0d0 con 0x56144354eeb0 2024-05-27T10:30:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.583+0000 7fd057400700 1 -- 172.21.15.42:0/3875961068 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd05c00fe20 con 0x56144354eeb0 2024-05-27T10:30:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.583+0000 7fd057400700 1 -- 172.21.15.42:0/3875961068 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fd05c021070 con 0x56144354eeb0 2024-05-27T10:30:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd0828deb80 10 client.17088 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T10:30:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd0828deb80 10 client.17088 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:30:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd0828deb80 20 client.17088 populate_metadata read hostname 'smithi042' 2024-05-27T10:30:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436f05b0 0x5614436f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd0828deb80 4 client.17088 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T10:30:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614435fde00 con 0x5614436f05b0 2024-05-27T10:30:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.584+0000 7fd070a00700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436f05b0 0x5614436f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.586+0000 7fd070a00700 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436f05b0 0x5614436f29d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b58f85cd6cf79d54 server_cookie=8d9b7b77ff22757c in_seq=0 out_seq=0 2024-05-27T10:30:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.586+0000 7fd057400700 10 client.17088 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:05.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.587+0000 7fd057400700 1 -- 172.21.15.42:0/3875961068 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5614435fde00 con 0x5614436f05b0 2024-05-27T10:30:05.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.587+0000 7fd057400700 10 client.17088 handle_command_reply: tid=0 2024-05-27T10:30:05.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.587+0000 7fd0828deb80 1 client.17088 shutdown 2024-05-27T10:30:05.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 msgr2=0x5614436e0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436e0990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098c0 tx=0x7fd05c0039e0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 shutdown_connections 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436f05b0 0x5614436f29d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614436d6d60 0x5614436e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144354eeb0 0x5614436e0990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3875961068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x5614436e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 >> 172.21.15.42:0/3875961068 conn(0x5614435e8400 msgr2=0x5614436d5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.588+0000 7fd057400700 0 client.17088 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.589+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 shutdown_connections 2024-05-27T10:30:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.589+0000 7fd0828deb80 1 -- 172.21.15.42:0/3875961068 wait complete. 2024-05-27T10:30:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.589+0000 7fd0828deb80 20 client.17088 trim_cache size 0 max 16384 2024-05-27T10:30:05.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.633+0000 7fd0828deb80 1 Processor -- start 2024-05-27T10:30:05.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.633+0000 7fd0828deb80 1 -- start start 2024-05-27T10:30:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614435e8400 0x5614436ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 0x5614436e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144363aa60 0x5614436dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd0828deb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c00ffb0 con 0x56144363aa60 2024-05-27T10:30:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd0828deb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c00d7e0 con 0x56144354eeb0 2024-05-27T10:30:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd0828deb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c00b630 con 0x5614435e8400 2024-05-27T10:30:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd070a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 0x5614436e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd071e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144363aa60 0x5614436dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd070a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 0x5614436e2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:55876/0 (socket says 172.21.15.42:55876) 2024-05-27T10:30:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd070a00700 1 -- 172.21.15.42:0/3246694492 learned_addr learned my addr 172.21.15.42:0/3246694492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.634+0000 7fd071400700 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614435e8400 0x5614436ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.635+0000 7fd070a00700 1 -- 172.21.15.42:0/3246694492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614435e8400 msgr2=0x5614436ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.635+0000 7fd070a00700 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614435e8400 0x5614436ecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.635+0000 7fd070a00700 1 -- 172.21.15.42:0/3246694492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144363aa60 msgr2=0x5614436dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.635+0000 7fd070a00700 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144363aa60 0x5614436dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.635+0000 7fd070a00700 1 -- 172.21.15.42:0/3246694492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614436c58b0 con 0x56144354eeb0 2024-05-27T10:30:05.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.636+0000 7fd070a00700 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 0x5614436e2e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0580a5890 tx=0x7fd0580e4940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c3f233fd96eea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:05.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.636+0000 7fd062000700 1 -- 172.21.15.42:0/3246694492 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0580a9f10 con 0x56144354eeb0 2024-05-27T10:30:05.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.637+0000 7fd062000700 1 -- 172.21.15.42:0/3246694492 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0580a9810 con 0x56144354eeb0 2024-05-27T10:30:05.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.637+0000 7fd062000700 1 -- 172.21.15.42:0/3246694492 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0580a9f10 con 0x56144354eeb0 2024-05-27T10:30:05.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.638+0000 7fd0828deb80 1 -- 172.21.15.42:0/3246694492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 msgr2=0x5614436e2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.638+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 0x5614436e2e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0580a5890 tx=0x7fd0580e4940 comp rx=0 tx=0).stop 2024-05-27T10:30:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.640+0000 7fd0828deb80 1 -- 172.21.15.42:0/3246694492 shutdown_connections 2024-05-27T10:30:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.640+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56144363aa60 0x5614436dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.640+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5614435e8400 0x5614436ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.640+0000 7fd0828deb80 1 --2- 172.21.15.42:0/3246694492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144354eeb0 0x5614436e2e30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.640+0000 7fd0828deb80 1 -- 172.21.15.42:0/3246694492 >> 172.21.15.42:0/3246694492 conn(0x5614436dd4c0 msgr2=0x5614436d63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.641+0000 7fd0828deb80 1 -- 172.21.15.42:0/3246694492 shutdown_connections 2024-05-27T10:30:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.641+0000 7fd0828deb80 1 -- 172.21.15.42:0/3246694492 wait complete. 2024-05-27T10:30:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.642+0000 7fd0828deb80 1 -- start start 2024-05-27T10:30:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.642+0000 7fd0828deb80 1 Processor -- start 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56144354eeb0 0x5614436f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614435e8400 0x5614436f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x561443759e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c00bca0 con 0x5614435e8400 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c00d110 con 0x56144363aa60 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c00c370 con 0x56144354eeb0 2024-05-27T10:30:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.643+0000 7fd0828deb80 10 client.0 _ll_register_callbacks cb 0x561443595ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T10:30:05.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.644+0000 7fd071e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x561443759e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.644+0000 7fd071e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x561443759e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.42:55880/0 (socket says 172.21.15.42:55880) 2024-05-27T10:30:05.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.644+0000 7fd071e00700 1 -- 172.21.15.42:0/532010654 learned_addr learned my addr 172.21.15.42:0/532010654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:05.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.644+0000 7fd070a00700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614435e8400 0x5614436f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.644+0000 7fd071400700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56144354eeb0 0x5614436f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.645+0000 7fd071e00700 1 -- 172.21.15.42:0/532010654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56144354eeb0 msgr2=0x5614436f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.645+0000 7fd071e00700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56144354eeb0 0x5614436f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.645+0000 7fd071e00700 1 -- 172.21.15.42:0/532010654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614435e8400 msgr2=0x5614436f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T10:30:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.645+0000 7fd071e00700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614435e8400 0x5614436f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.645+0000 7fd071e00700 1 -- 172.21.15.42:0/532010654 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614436e1780 con 0x56144363aa60 2024-05-27T10:30:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.646+0000 7fd071e00700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x561443759e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd064019450 tx=0x7fd064004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c468f6407c766a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:05.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.646+0000 7fd057400700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-27T10:30:05.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.646+0000 7fd057400700 1 -- 172.21.15.42:0/532010654 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06402b070 con 0x56144363aa60 2024-05-27T10:30:05.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.647+0000 7fd057400700 1 -- 172.21.15.42:0/532010654 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7fd064027420 con 0x56144363aa60 2024-05-27T10:30:05.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.647+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0580a9810 con 0x56144363aa60 2024-05-27T10:30:05.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.648+0000 7fd057400700 1 -- 172.21.15.42:0/532010654 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd064027620 con 0x56144363aa60 2024-05-27T10:30:05.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.649+0000 7fd0828deb80 10 client.36557 fetch_fsmap learned FSMap version 40 2024-05-27T10:30:05.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.649+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56144375b280 con 0x56144363aa60 2024-05-27T10:30:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.649+0000 7fd057400700 1 -- 172.21.15.42:0/532010654 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0580a9810 con 0x56144363aa60 2024-05-27T10:30:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.649+0000 7fd057400700 1 -- 172.21.15.42:0/532010654 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fd064037020 con 0x56144363aa60 2024-05-27T10:30:05.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.650+0000 7fd0828deb80 10 client.36557 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T10:30:05.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.650+0000 7fd0828deb80 10 client.36557 resolve_mds: validated gid 34499 aka daemon mds.b 2024-05-27T10:30:05.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.650+0000 7fd0828deb80 20 client.36557 populate_metadata read hostname 'smithi042' 2024-05-27T10:30:05.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.650+0000 7fd0828deb80 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436d6d60 0x56144375da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:05.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.651+0000 7fd0828deb80 4 client.36557 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-27T10:30:05.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.651+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 --> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5614436c1780 con 0x5614436d6d60 2024-05-27T10:30:05.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.651+0000 7fd071400700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436d6d60 0x56144375da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:05.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.653+0000 7fd057400700 10 client.36557 ms_handle_connect on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:05.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.653+0000 7fd071400700 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436d6d60 0x56144375da50 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3865a9b4533c712 server_cookie=400af5ce1ef4bfd6 in_seq=0 out_seq=0 2024-05-27T10:30:05.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.653+0000 7fd057400700 1 -- 172.21.15.42:0/532010654 <== mds.0 v2:172.21.15.104:6838/3864142797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5614436c1780 con 0x5614436d6d60 2024-05-27T10:30:05.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.653+0000 7fd057400700 10 client.36557 handle_command_reply: tid=0 2024-05-27T10:30:05.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd0828deb80 1 client.36557 shutdown 2024-05-27T10:30:05.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 msgr2=0x561443759e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd0828deb80 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x561443759e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd064019450 tx=0x7fd064004ad0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 shutdown_connections 2024-05-27T10:30:05.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd0828deb80 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:6838/3864142797,v1:172.21.15.104:6839/3864142797] conn(0x5614436d6d60 0x56144375da50 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd0828deb80 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56144363aa60 0x561443759e20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.654+0000 7fd057400700 0 client.36557 ms_handle_reset on v2:172.21.15.104:6838/3864142797 2024-05-27T10:30:05.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.655+0000 7fd0828deb80 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56144354eeb0 0x5614436f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.655+0000 7fd0828deb80 1 --2- 172.21.15.42:0/532010654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614435e8400 0x5614436f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.655+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 >> 172.21.15.42:0/532010654 conn(0x5614436dd4c0 msgr2=0x5614436df150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.655+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 shutdown_connections 2024-05-27T10:30:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.655+0000 7fd0828deb80 1 -- 172.21.15.42:0/532010654 wait complete. 2024-05-27T10:30:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.656+0000 7fd0828deb80 20 client.36557 trim_cache size 0 max 16384 2024-05-27T10:30:05.657 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:30:05.657 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 40, 2024-05-27T10:30:05.657 INFO:teuthology.orchestra.run.smithi042.stdout: "leader": 34499, 2024-05-27T10:30:05.657 INFO:teuthology.orchestra.run.smithi042.stdout: "set_version": 32, 2024-05-27T10:30:05.657 INFO:teuthology.orchestra.run.smithi042.stdout: "sets": {} 2024-05-27T10:30:05.657 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:30:05.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.658+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0500b2c00 msgr2=0x7fd0500b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.658+0000 7fd073e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0500b2c00 0x7fd0500b5110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd064009e10 tx=0x7fd0640099e0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.658+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c106eb0 msgr2=0x7fd06c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:05.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.658+0000 7fd073e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c106eb0 0x7fd06c19d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd06c075b70 tx=0x7fd05800eca0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.659+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 shutdown_connections 2024-05-27T10:30:05.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.659+0000 7fd073e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fd0500b2c00 0x7fd0500b5110 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.659+0000 7fd073e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd06c10c5a0 0x7fd06c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.659+0000 7fd073e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd06c106560 0x7fd06c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.659+0000 7fd073e00700 1 --2- 172.21.15.42:0/1318789266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd06c106eb0 0x7fd06c19d570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:05.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.659+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 >> 172.21.15.42:0/1318789266 conn(0x7fd06c102040 msgr2=0x7fd06c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:05.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.660+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 shutdown_connections 2024-05-27T10:30:05.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:05.661+0000 7fd073e00700 1 -- 172.21.15.42:0/1318789266 wait complete. 2024-05-27T10:30:06.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:06 smithi143 ceph-mon[35545]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T10:30:06.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:06 smithi143 ceph-mon[35545]: pgmap v354: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 95 op/s 2024-05-27T10:30:06.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:06 smithi042 ceph-mon[31661]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T10:30:06.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:06 smithi042 ceph-mon[31661]: pgmap v354: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 95 op/s 2024-05-27T10:30:06.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:06 smithi104 ceph-mon[34812]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T10:30:06.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:06 smithi104 ceph-mon[34812]: pgmap v354: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 95 op/s 2024-05-27T10:30:07.067 INFO:tasks.quiescer:done joining Quiescers 2024-05-27T10:30:07.068 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-27T10:30:07.078 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:30:07.107 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-27T10:30:07.117 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-27T10:30:07.117 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T10:30:07.118 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:30:07.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-27T10:30:07.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-27T10:30:07.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 -- 172.21.15.42:0/992366335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 msgr2=0x7ff1f8116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 --2- 172.21.15.42:0/992366335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f8116b60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009ca0 tx=0x7ff1f002a8c0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 -- 172.21.15.42:0/992366335 shutdown_connections 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 --2- 172.21.15.42:0/992366335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f8116b60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 --2- 172.21.15.42:0/992366335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f810e900 0x7ff1f810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 --2- 172.21.15.42:0/992366335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810a390 0x7ff1f810a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 -- 172.21.15.42:0/992366335 >> 172.21.15.42:0/992366335 conn(0x7ff1f8076070 msgr2=0x7ff1f8076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 -- 172.21.15.42:0/992366335 shutdown_connections 2024-05-27T10:30:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.628+0000 7ff1fca00640 1 -- 172.21.15.42:0/992366335 wait complete. 2024-05-27T10:30:07.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.629+0000 7ff1fca00640 1 Processor -- start 2024-05-27T10:30:07.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.629+0000 7ff1fca00640 1 -- start start 2024-05-27T10:30:07.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1fca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f810a390 0x7ff1f81a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:07.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1fca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810e900 0x7ff1f81a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:07.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1fca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f81a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:07.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1fca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f811a200 con 0x7ff1f810f3c0 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1fca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f811a080 con 0x7ff1f810a390 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1fca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f811a380 con 0x7ff1f810e900 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1f7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f81a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1f7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f81a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36756/0 (socket says 172.21.15.42:36756) 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.630+0000 7ff1f7200640 1 -- 172.21.15.42:0/1679264861 learned_addr learned my addr 172.21.15.42:0/1679264861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f6800640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f810a390 0x7ff1f81a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f5e00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810e900 0x7ff1f81a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f7200640 1 -- 172.21.15.42:0/1679264861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810e900 msgr2=0x7ff1f81a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f7200640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810e900 0x7ff1f81a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f7200640 1 -- 172.21.15.42:0/1679264861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f810a390 msgr2=0x7ff1f81a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f7200640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f810a390 0x7ff1f81a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f7200640 1 -- 172.21.15.42:0/1679264861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a11a0 con 0x7ff1f810f3c0 2024-05-27T10:30:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f5e00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810e900 0x7ff1f81a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T10:30:07.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1f7200640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f81a0b00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005cc0 tx=0x7ff1f0032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:07.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.631+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0032e00 con 0x7ff1f810f3c0 2024-05-27T10:30:07.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.632+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f00044d0 con 0x7ff1f810f3c0 2024-05-27T10:30:07.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.632+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a1430 con 0x7ff1f810f3c0 2024-05-27T10:30:07.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.632+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0030b70 con 0x7ff1f810f3c0 2024-05-27T10:30:07.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.632+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81ad770 con 0x7ff1f810f3c0 2024-05-27T10:30:07.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.633+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8050860 con 0x7ff1f810f3c0 2024-05-27T10:30:07.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.636+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff1f0030d10 con 0x7ff1f810f3c0 2024-05-27T10:30:07.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.637+0000 7ff1ef400640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff1dc0b2950 0x7ff1dc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:07.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.638+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7ff1f0103cf0 con 0x7ff1f810f3c0 2024-05-27T10:30:07.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.638+0000 7ff1f6800640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff1dc0b2950 0x7ff1dc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:07.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.639+0000 7ff1f6800640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff1dc0b2950 0x7ff1dc0b4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005fd0 tx=0x7ff1e8005850 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:07.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.640+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1f00cb7a0 con 0x7ff1f810f3c0 2024-05-27T10:30:07.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.876+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff1f8063100 con 0x7ff1dc0b2950 2024-05-27T10:30:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.895+0000 7ff1ef400640 1 -- 172.21.15.42:0/1679264861 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ff1f8063100 con 0x7ff1dc0b2950 2024-05-27T10:30:07.895 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T10:19:03", "first": "2024-05-27T10:20:00", "last": "2024-05-27T10:30:00", "last_pruned": "2024-05-27T10:30:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff1dc0b2950 msgr2=0x7ff1dc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff1dc0b2950 0x7ff1dc0b4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005fd0 tx=0x7ff1e8005850 comp rx=0 tx=0).stop 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 msgr2=0x7ff1f81a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f81a0b00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005cc0 tx=0x7ff1f0032a10 comp rx=0 tx=0).stop 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 shutdown_connections 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff1dc0b2950 0x7ff1dc0b4e60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f810f3c0 0x7ff1f81a0b00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1f810e900 0x7ff1f81a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 --2- 172.21.15.42:0/1679264861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f810a390 0x7ff1f81a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 >> 172.21.15.42:0/1679264861 conn(0x7ff1f8076070 msgr2=0x7ff1f8107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 shutdown_connections 2024-05-27T10:30:07.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:07.899+0000 7ff1fca00640 1 -- 172.21.15.42:0/1679264861 wait complete. 2024-05-27T10:30:07.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 -- 172.21.15.42:0/2622643696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010fb50 msgr2=0x7f523010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 --2- 172.21.15.42:0/2622643696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010fb50 0x7f523010ff80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5218009900 tx=0x7f521802a3f0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 -- 172.21.15.42:0/2622643696 shutdown_connections 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 --2- 172.21.15.42:0/2622643696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230110610 0x7f5230117db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 --2- 172.21.15.42:0/2622643696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010fb50 0x7f523010ff80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 --2- 172.21.15.42:0/2622643696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f523010b5d0 0x7f523010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 -- 172.21.15.42:0/2622643696 >> 172.21.15.42:0/2622643696 conn(0x7f523006eb10 msgr2=0x7f5230070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 -- 172.21.15.42:0/2622643696 shutdown_connections 2024-05-27T10:30:08.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.203+0000 7f5234e00640 1 -- 172.21.15.42:0/2622643696 wait complete. 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.205+0000 7f5234e00640 1 Processor -- start 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.205+0000 7f5234e00640 1 -- start start 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 0x7f52301a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f523010fb50 0x7f52301a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230110610 0x7f52301a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f523011b3f0 con 0x7f523010b5d0 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f523011b270 con 0x7f523010fb50 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f523011b570 con 0x7f5230110610 2024-05-27T10:30:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 0x7f52301a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 0x7f52301a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36768/0 (socket says 172.21.15.42:36768) 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 -- 172.21.15.42:0/1400238388 learned_addr learned my addr 172.21.15.42:0/1400238388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522d400640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f523010fb50 0x7f52301a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522e800640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230110610 0x7f52301a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 -- 172.21.15.42:0/1400238388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230110610 msgr2=0x7f52301a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230110610 0x7f52301a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 -- 172.21.15.42:0/1400238388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f523010fb50 msgr2=0x7f52301a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f523010fb50 0x7f52301a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 -- 172.21.15.42:0/1400238388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301a9fa0 con 0x7f523010b5d0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f522de00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 0x7f52301a5140 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f522000eab0 tx=0x7f522000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522000ce40 con 0x7f523010b5d0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5220011900 con 0x7f523010b5d0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220010990 con 0x7f523010b5d0 2024-05-27T10:30:08.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301aa290 con 0x7f523010b5d0 2024-05-27T10:30:08.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.206+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301aa760 con 0x7f523010b5d0 2024-05-27T10:30:08.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.208+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5230050860 con 0x7f523010b5d0 2024-05-27T10:30:08.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.210+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5220011470 con 0x7f523010b5d0 2024-05-27T10:30:08.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.214+0000 7f5226a00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52140b2950 0x7f52140b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.214+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7f52200e2d00 con 0x7f523010b5d0 2024-05-27T10:30:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.214+0000 7f522d400640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52140b2950 0x7f52140b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.214+0000 7f522d400640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52140b2950 0x7f52140b4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52180057c0 tx=0x7f5218000bc0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:08.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.216+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5220016030 con 0x7f523010b5d0 2024-05-27T10:30:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.453+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 --> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5230116fc0 con 0x7f52140b2950 2024-05-27T10:30:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.472+0000 7f5226a00640 1 -- 172.21.15.42:0/1400238388 <== mgr.34892 v2:172.21.15.42:6800/3616349127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5230116fc0 con 0x7f52140b2950 2024-05-27T10:30:08.472 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52140b2950 msgr2=0x7f52140b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52140b2950 0x7f52140b4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52180057c0 tx=0x7f5218000bc0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 msgr2=0x7f52301a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 0x7f52301a5140 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f522000eab0 tx=0x7f522000ef70 comp rx=0 tx=0).stop 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 shutdown_connections 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f52140b2950 0x7f52140b4e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230110610 0x7f52301a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f523010fb50 0x7f52301a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 --2- 172.21.15.42:0/1400238388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f523010b5d0 0x7f52301a5140 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.475+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 >> 172.21.15.42:0/1400238388 conn(0x7f523006eb10 msgr2=0x7f52301168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.476+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 shutdown_connections 2024-05-27T10:30:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.476+0000 7f5234e00640 1 -- 172.21.15.42:0/1400238388 wait complete. 2024-05-27T10:30:08.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:08 smithi042 ceph-mon[31661]: pgmap v355: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 95 op/s 2024-05-27T10:30:08.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:08 smithi042 ceph-mon[31661]: from='client.17100 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:30:08.503 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-27T10:30:08.514 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-27T10:30:08.514 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-27T10:30:08.515 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-27T10:30:08.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T10:30:08.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:08 smithi143 ceph-mon[35545]: pgmap v355: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 95 op/s 2024-05-27T10:30:08.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:08 smithi143 ceph-mon[35545]: from='client.17100 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:30:08.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:08 smithi104 ceph-mon[34812]: pgmap v355: 97 pgs: 97 active+clean; 422 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 95 op/s 2024-05-27T10:30:08.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:08 smithi104 ceph-mon[34812]: from='client.17100 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:30:08.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 -- 172.21.15.42:0/1731924462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 msgr2=0x7fb11c10a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 --2- 172.21.15.42:0/1731924462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c10a790 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009900 tx=0x7fb10c02a410 comp rx=0 tx=0).stop 2024-05-27T10:30:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 -- 172.21.15.42:0/1731924462 shutdown_connections 2024-05-27T10:30:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 --2- 172.21.15.42:0/1731924462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb11c10f330 0x7fb11c116ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 --2- 172.21.15.42:0/1731924462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c10e870 0x7fb11c10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 --2- 172.21.15.42:0/1731924462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c10a790 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 -- 172.21.15.42:0/1731924462 >> 172.21.15.42:0/1731924462 conn(0x7fb11c076000 msgr2=0x7fb11c076410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:08.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 -- 172.21.15.42:0/1731924462 shutdown_connections 2024-05-27T10:30:08.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 -- 172.21.15.42:0/1731924462 wait complete. 2024-05-27T10:30:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 Processor -- start 2024-05-27T10:30:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.837+0000 7fb121000640 1 -- start start 2024-05-27T10:30:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb121000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c1a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb121000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb11c10e870 0x7fb11c1a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb121000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c10f330 0x7fb11c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb121000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c11a110 con 0x7fb11c10a380 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb121000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c119f90 con 0x7fb11c10f330 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb121000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c11a290 con 0x7fb11c10e870 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb11b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c1a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb11b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c1a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36780/0 (socket says 172.21.15.42:36780) 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb11b400640 1 -- 172.21.15.42:0/637645434 learned_addr learned my addr 172.21.15.42:0/637645434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.838+0000 7fb11aa00640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb11c10e870 0x7fb11c1a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11be00640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c10f330 0x7fb11c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11b400640 1 -- 172.21.15.42:0/637645434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb11c10e870 msgr2=0x7fb11c1a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11b400640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb11c10e870 0x7fb11c1a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11b400640 1 -- 172.21.15.42:0/637645434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c10f330 msgr2=0x7fb11c1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11b400640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c10f330 0x7fb11c1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11b400640 1 -- 172.21.15.42:0/637645434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c1a10d0 con 0x7fb11c10a380 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb11b400640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c1a6820 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009230 tx=0x7fb10c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c03f030 con 0x7fb11c10a380 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.839+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb10c0043d0 con 0x7fb11c10a380 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.840+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c035070 con 0x7fb11c10a380 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.840+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c1a1360 con 0x7fb11c10a380 2024-05-27T10:30:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.840+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c1a1800 con 0x7fb11c10a380 2024-05-27T10:30:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.841+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c06a150 con 0x7fb11c10a380 2024-05-27T10:30:08.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.845+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb10c042030 con 0x7fb11c10a380 2024-05-27T10:30:08.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.848+0000 7fb113e00640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb1000b2950 0x7fb1000b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:08.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.849+0000 7fb11aa00640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb1000b2950 0x7fb1000b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:08.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.849+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7fb10c102f00 con 0x7fb11c10a380 2024-05-27T10:30:08.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.850+0000 7fb11aa00640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb1000b2950 0x7fb1000b4e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb11c108fe0 tx=0x7fb108005f90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:08.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:08.850+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb10c103380 con 0x7fb11c10a380 2024-05-27T10:30:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.069+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb11c1ad700 con 0x7fb11c10a380 2024-05-27T10:30:09.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.071+0000 7fb113e00640 1 -- 172.21.15.42:0/637645434 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+23097 (secure 0 0 0) 0x7fb10c0ca8a0 con 0x7fb11c10a380 2024-05-27T10:30:09.071 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:30:09.071 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":92,"fsid":"ff1a20ea-1c0f-11ef-bc9b-c7b262605968","created":"2024-05-27T10:02:27.892323+0000","modified":"2024-05-27T10:30:04.994668+0000","last_up_change":"2024-05-27T10:12:45.387184+0000","last_in_change":"2024-05-27T10:12:19.823767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T10:07:06.385341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T10:14:07.735206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T10:14:12.919933+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":92,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f20f91e-7aa0-41e1-bff3-c8a8c7e24321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6803","nonce":3193665133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6805","nonce":3193665133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6809","nonce":3193665133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3193665133},{"type":"v1","addr":"172.21.15.42:6807","nonce":3193665133}]},"public_addr":"172.21.15.42:6803/3193665133","cluster_addr":"172.21.15.42:6805/3193665133","heartbeat_back_addr":"172.21.15.42:6809/3193665133","heartbeat_front_addr":"172.21.15.42:6807/3193665133","state":["exists","up"]},{"osd":1,"uuid":"7d3ff39d-4276-43ec-8f9f-41a4b970afaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6801","nonce":2575126270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6803","nonce":2575126270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6807","nonce":2575126270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2575126270},{"type":"v1","addr":"172.21.15.104:6805","nonce":2575126270}]},"public_addr":"172.21.15.104:6801/2575126270","cluster_addr":"172.21.15.104:6803/2575126270","heartbeat_back_addr":"172.21.15.104:6807/2575126270","heartbeat_front_addr":"172.21.15.104:6805/2575126270","state":["exists","up"]},{"osd":2,"uuid":"72adfe76-c970-4f68-baea-1d04215301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6801","nonce":793524838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6803","nonce":793524838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6807","nonce":793524838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":793524838},{"type":"v1","addr":"172.21.15.143:6805","nonce":793524838}]},"public_addr":"172.21.15.143:6801/793524838","cluster_addr":"172.21.15.143:6803/793524838","heartbeat_back_addr":"172.21.15.143:6807/793524838","heartbeat_front_addr":"172.21.15.143:6805/793524838","state":["exists","up"]},{"osd":3,"uuid":"11c122a0-b307-4f56-8109-52475745da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6811","nonce":2776846288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6813","nonce":2776846288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6817","nonce":2776846288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2776846288},{"type":"v1","addr":"172.21.15.42:6815","nonce":2776846288}]},"public_addr":"172.21.15.42:6811/2776846288","cluster_addr":"172.21.15.42:6813/2776846288","heartbeat_back_addr":"172.21.15.42:6817/2776846288","heartbeat_front_addr":"172.21.15.42:6815/2776846288","state":["exists","up"]},{"osd":4,"uuid":"78330de0-74fb-49b7-8de6-d694077e8ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6809","nonce":3588429935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6811","nonce":3588429935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6815","nonce":3588429935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3588429935},{"type":"v1","addr":"172.21.15.104:6813","nonce":3588429935}]},"public_addr":"172.21.15.104:6809/3588429935","cluster_addr":"172.21.15.104:6811/3588429935","heartbeat_back_addr":"172.21.15.104:6815/3588429935","heartbeat_front_addr":"172.21.15.104:6813/3588429935","state":["exists","up"]},{"osd":5,"uuid":"6aba47e2-0c53-46c1-bd90-d7cf86d14f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6809","nonce":658500194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6811","nonce":658500194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6815","nonce":658500194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":658500194},{"type":"v1","addr":"172.21.15.143:6813","nonce":658500194}]},"public_addr":"172.21.15.143:6809/658500194","cluster_addr":"172.21.15.143:6811/658500194","heartbeat_back_addr":"172.21.15.143:6815/658500194","heartbeat_front_addr":"172.21.15.143:6813/658500194","state":["exists","up"]},{"osd":6,"uuid":"1cc50666-edea-46b9-a6c0-0d8c27799fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6819","nonce":2880152722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6821","nonce":2880152722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6825","nonce":2880152722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2880152722},{"type":"v1","addr":"172.21.15.42:6823","nonce":2880152722}]},"public_addr":"172.21.15.42:6819/2880152722","cluster_addr":"172.21.15.42:6821/2880152722","heartbeat_back_addr":"172.21.15.42:6825/2880152722","heartbeat_front_addr":"172.21.15.42:6823/2880152722","state":["exists","up"]},{"osd":7,"uuid":"a8c58d76-682d-48b2-a195-ab8ec1b47d3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6817","nonce":2687473765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6819","nonce":2687473765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6823","nonce":2687473765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2687473765},{"type":"v1","addr":"172.21.15.104:6821","nonce":2687473765}]},"public_addr":"172.21.15.104:6817/2687473765","cluster_addr":"172.21.15.104:6819/2687473765","heartbeat_back_addr":"172.21.15.104:6823/2687473765","heartbeat_front_addr":"172.21.15.104:6821/2687473765","state":["exists","up"]},{"osd":8,"uuid":"e59bbe26-23f9-4728-957e-7ca5e03e6bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6817","nonce":3963187375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6819","nonce":3963187375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6823","nonce":3963187375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":3963187375},{"type":"v1","addr":"172.21.15.143:6821","nonce":3963187375}]},"public_addr":"172.21.15.143:6817/3963187375","cluster_addr":"172.21.15.143:6819/3963187375","heartbeat_back_addr":"172.21.15.143:6823/3963187375","heartbeat_front_addr":"172.21.15.143:6821/3963187375","state":["exists","up"]},{"osd":9,"uuid":"951d4d17-d8c3-495b-8295-667f0bfaaef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6827","nonce":2335274703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6829","nonce":2335274703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6833","nonce":2335274703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2335274703},{"type":"v1","addr":"172.21.15.42:6831","nonce":2335274703}]},"public_addr":"172.21.15.42:6827/2335274703","cluster_addr":"172.21.15.42:6829/2335274703","heartbeat_back_addr":"172.21.15.42:6833/2335274703","heartbeat_front_addr":"172.21.15.42:6831/2335274703","state":["exists","up"]},{"osd":10,"uuid":"d114e0d1-848d-453c-a61d-65262d9c404e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6825","nonce":1167441183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6827","nonce":1167441183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6831","nonce":1167441183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1167441183},{"type":"v1","addr":"172.21.15.104:6829","nonce":1167441183}]},"public_addr":"172.21.15.104:6825/1167441183","cluster_addr":"172.21.15.104:6827/1167441183","heartbeat_back_addr":"172.21.15.104:6831/1167441183","heartbeat_front_addr":"172.21.15.104:6829/1167441183","state":["exists","up"]},{"osd":11,"uuid":"a925b882-f488-4723-a316-04230d44cfb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6825","nonce":1326686581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6827","nonce":1326686581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6831","nonce":1326686581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1326686581},{"type":"v1","addr":"172.21.15.143:6829","nonce":1326686581}]},"public_addr":"172.21.15.143:6825/1326686581","cluster_addr":"172.21.15.143:6827/1326686581","heartbeat_back_addr":"172.21.15.143:6831/1326686581","heartbeat_front_addr":"172.21.15.143:6829/1326686581","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:05:51.968270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:06:28.649014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:02.644281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:07:40.830403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:18.807274+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:08:55.158887+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:09:34.954542+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:13.236639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:10:49.824959+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:11:29.360451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:07.128532+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T10:12:43.644854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1953650070":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/4102007980":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/2381100937":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/2062230330":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6801/3098170734":"2024-05-28T10:03:40.179244+0000","172.21.15.42:0/1584316049":"2024-05-28T10:03:40.179244+0000","172.21.15.42:6801/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2667703980":"2024-05-28T10:18:42.838173+0000","172.21.15.42:6800/3950108791":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2970531755":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/1310078254":"2024-05-28T10:18:42.838173+0000","172.21.15.42:0/274704256":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/4159042453":"2024-05-28T10:03:03.499453+0000","172.21.15.42:0/2156747670":"2024-05-28T10:03:03.499453+0000","172.21.15.42:6800/3098170734":"2024-05-28T10:03:40.179244+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T10:30:09.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.074+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb1000b2950 msgr2=0x7fb1000b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.074+0000 7fb121000640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb1000b2950 0x7fb1000b4e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb11c108fe0 tx=0x7fb108005f90 comp rx=0 tx=0).stop 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 msgr2=0x7fb11c1a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c1a6820 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009230 tx=0x7fb10c0319a0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 shutdown_connections 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fb1000b2950 0x7fb1000b4e60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c10f330 0x7fb11c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb11c10e870 0x7fb11c1a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 --2- 172.21.15.42:0/637645434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c10a380 0x7fb11c1a6820 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 >> 172.21.15.42:0/637645434 conn(0x7fb11c076000 msgr2=0x7fb11c107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 shutdown_connections 2024-05-27T10:30:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.075+0000 7fb121000640 1 -- 172.21.15.42:0/637645434 wait complete. 2024-05-27T10:30:09.102 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T10:07:06.385341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T10:14:07.735206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T10:14:12.919933+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 92, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T10:30:09.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 -- 172.21.15.42:0/1822601451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 msgr2=0x7ff7c0112790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/1822601451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c0112790 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009ca0 tx=0x7ff7b802a8b0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 -- 172.21.15.42:0/1822601451 shutdown_connections 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/1822601451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c0112790 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/1822601451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c010c110 0x7ff7c0105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/1822601451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c010bc70 0x7ff7c0105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 -- 172.21.15.42:0/1822601451 >> 172.21.15.42:0/1822601451 conn(0x7ff7c007d590 msgr2=0x7ff7c007d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 -- 172.21.15.42:0/1822601451 shutdown_connections 2024-05-27T10:30:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.399+0000 7ff7bfe00640 1 -- 172.21.15.42:0/1822601451 wait complete. 2024-05-27T10:30:09.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.400+0000 7ff7bfe00640 1 Processor -- start 2024-05-27T10:30:09.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.400+0000 7ff7bfe00640 1 -- start start 2024-05-27T10:30:09.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c010bc70 0x7ff7c019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c010c110 0x7ff7c01a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0115d60 con 0x7ff7c01063c0 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bfe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0115be0 con 0x7ff7c010bc70 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bfe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0115ee0 con 0x7ff7c010c110 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36800/0 (socket says 172.21.15.42:36800) 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.401+0000 7ff7bea00640 1 -- 172.21.15.42:0/80564044 learned_addr learned my addr 172.21.15.42:0/80564044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7be000640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c010bc70 0x7ff7c019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bf400640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c010c110 0x7ff7c01a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bea00640 1 -- 172.21.15.42:0/80564044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c010c110 msgr2=0x7ff7c01a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bea00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c010c110 0x7ff7c01a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bea00640 1 -- 172.21.15.42:0/80564044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c010bc70 msgr2=0x7ff7c019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bea00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c010bc70 0x7ff7c019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bea00640 1 -- 172.21.15.42:0/80564044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01a1500 con 0x7ff7c01063c0 2024-05-27T10:30:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.402+0000 7ff7bea00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c019c720 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000c9a0 tx=0x7ff7b000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.403+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0014070 con 0x7ff7c01063c0 2024-05-27T10:30:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.403+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01a17f0 con 0x7ff7c01063c0 2024-05-27T10:30:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.403+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7b0004440 con 0x7ff7c01063c0 2024-05-27T10:30:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.403+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a1d40 con 0x7ff7c01063c0 2024-05-27T10:30:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.403+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0003d80 con 0x7ff7c01063c0 2024-05-27T10:30:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.404+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0106f50 con 0x7ff7c01063c0 2024-05-27T10:30:09.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.407+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff7b0017070 con 0x7ff7c01063c0 2024-05-27T10:30:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.409+0000 7ff7c4a00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff7a40b2950 0x7ff7a40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.409+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7ff7b00e2960 con 0x7ff7c01063c0 2024-05-27T10:30:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.409+0000 7ff7be000640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff7a40b2950 0x7ff7a40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.410+0000 7ff7be000640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff7a40b2950 0x7ff7a40b4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7c006a100 tx=0x7ff7ac005ee0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.411+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7b001e020 con 0x7ff7c01063c0 2024-05-27T10:30:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:09 smithi143 ceph-mon[35545]: from='client.17106 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:30:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:09 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/637645434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:30:09.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:09 smithi042 ceph-mon[31661]: from='client.17106 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:30:09.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:09 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/637645434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:30:09.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.623+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7c010ddb0 con 0x7ff7c01063c0 2024-05-27T10:30:09.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.624+0000 7ff7c4a00640 1 -- 172.21.15.42:0/80564044 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7ff7b00aa300 con 0x7ff7c01063c0 2024-05-27T10:30:09.624 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-27T10:30:09.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff7a40b2950 msgr2=0x7ff7a40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff7a40b2950 0x7ff7a40b4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7c006a100 tx=0x7ff7ac005ee0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 msgr2=0x7ff7c019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c019c720 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000c9a0 tx=0x7ff7b000ce60 comp rx=0 tx=0).stop 2024-05-27T10:30:09.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 shutdown_connections 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7ff7a40b2950 0x7ff7a40b4e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c010c110 0x7ff7c01a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c010bc70 0x7ff7c019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 --2- 172.21.15.42:0/80564044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7c01063c0 0x7ff7c019c720 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.627+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 >> 172.21.15.42:0/80564044 conn(0x7ff7c007d590 msgr2=0x7ff7c0102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.628+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 shutdown_connections 2024-05-27T10:30:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.628+0000 7ff7bfe00640 1 -- 172.21.15.42:0/80564044 wait complete. 2024-05-27T10:30:09.656 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T10:30:09.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:09 smithi104 ceph-mon[34812]: from='client.17106 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T10:30:09.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:09 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/637645434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 -- 172.21.15.42:0/2806656154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0109c30 msgr2=0x7f99f01037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 --2- 172.21.15.42:0/2806656154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0109c30 0x7f99f01037d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009ca0 tx=0x7f99e002a8c0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 -- 172.21.15.42:0/2806656154 shutdown_connections 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 --2- 172.21.15.42:0/2806656154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f0104380 0x7f99f0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 --2- 172.21.15.42:0/2806656154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99f010a0d0 0x7f99f0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 --2- 172.21.15.42:0/2806656154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0109c30 0x7f99f01037d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 -- 172.21.15.42:0/2806656154 >> 172.21.15.42:0/2806656154 conn(0x7f99f00ff330 msgr2=0x7f99f0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 -- 172.21.15.42:0/2806656154 shutdown_connections 2024-05-27T10:30:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 -- 172.21.15.42:0/2806656154 wait complete. 2024-05-27T10:30:09.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 Processor -- start 2024-05-27T10:30:09.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.952+0000 7f99f5000640 1 -- start start 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99f5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 0x7f99f019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99f5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99f0109c30 0x7f99f019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99f5000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f010a0d0 0x7f99f01a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99f5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0113d20 con 0x7f99f0104380 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99f5000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0113ba0 con 0x7f99f010a0d0 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99f5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0113ea0 con 0x7f99f0109c30 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99ede00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 0x7f99f019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99ede00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 0x7f99f019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36818/0 (socket says 172.21.15.42:36818) 2024-05-27T10:30:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99ede00640 1 -- 172.21.15.42:0/2196085836 learned_addr learned my addr 172.21.15.42:0/2196085836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.953+0000 7f99ee800640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f010a0d0 0x7f99f01a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ed400640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99f0109c30 0x7f99f019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ede00640 1 -- 172.21.15.42:0/2196085836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99f0109c30 msgr2=0x7f99f019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ede00640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99f0109c30 0x7f99f019dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ede00640 1 -- 172.21.15.42:0/2196085836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f010a0d0 msgr2=0x7f99f01a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ede00640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f010a0d0 0x7f99f01a8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ede00640 1 -- 172.21.15.42:0/2196085836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f01a8bc0 con 0x7f99f0104380 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99ede00640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 0x7f99f019d7e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f99e0032a10 tx=0x7f99e0032a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0032e60 con 0x7f99f0104380 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e00044d0 con 0x7f99f0104380 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0030c00 con 0x7f99f0104380 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f01a8df0 con 0x7f99f0104380 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.954+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01a9300 con 0x7f99f0104380 2024-05-27T10:30:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.956+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f006a280 con 0x7f99f0104380 2024-05-27T10:30:09.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.958+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f99e0004040 con 0x7f99f0104380 2024-05-27T10:30:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.959+0000 7f99e6a00640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f99d40b29e0 0x7f99d40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.960+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7f99e0103af0 con 0x7f99f0104380 2024-05-27T10:30:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.960+0000 7f99ed400640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f99d40b29e0 0x7f99d40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:09.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.961+0000 7f99ed400640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f99d40b29e0 0x7f99d40b4ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99f01027b0 tx=0x7f99d8005c90 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:09.964+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99e00098b0 con 0x7f99f0104380 2024-05-27T10:30:10.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.178+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99f01a9530 con 0x7f99f0104380 2024-05-27T10:30:10.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.181+0000 7f99e6a00640 1 -- 172.21.15.42:0/2196085836 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f99e00cb5a0 con 0x7f99f0104380 2024-05-27T10:30:10.179 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-27T10:30:10.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f99d40b29e0 msgr2=0x7f99d40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f99d40b29e0 0x7f99d40b4ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99f01027b0 tx=0x7f99d8005c90 comp rx=0 tx=0).stop 2024-05-27T10:30:10.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 msgr2=0x7f99f019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 0x7f99f019d7e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f99e0032a10 tx=0x7f99e0032a40 comp rx=0 tx=0).stop 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 shutdown_connections 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f99d40b29e0 0x7f99d40b4ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f010a0d0 0x7f99f01a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99f0109c30 0x7f99f019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 --2- 172.21.15.42:0/2196085836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f0104380 0x7f99f019d7e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 >> 172.21.15.42:0/2196085836 conn(0x7f99f00ff330 msgr2=0x7f99f0076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 shutdown_connections 2024-05-27T10:30:10.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.183+0000 7f99f5000640 1 -- 172.21.15.42:0/2196085836 wait complete. 2024-05-27T10:30:10.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T10:30:10.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 -- 172.21.15.42:0/1718752790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f387410bc60 msgr2=0x7f3874105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 --2- 172.21.15.42:0/1718752790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f387410bc60 0x7f3874105800 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3864009ca0 tx=0x7f386402a8c0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 -- 172.21.15.42:0/1718752790 shutdown_connections 2024-05-27T10:30:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 --2- 172.21.15.42:0/1718752790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38741063b0 0x7f3874112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 --2- 172.21.15.42:0/1718752790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410c100 0x7f3874105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 --2- 172.21.15.42:0/1718752790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f387410bc60 0x7f3874105800 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 -- 172.21.15.42:0/1718752790 >> 172.21.15.42:0/1718752790 conn(0x7f3874101360 msgr2=0x7f3874103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 -- 172.21.15.42:0/1718752790 shutdown_connections 2024-05-27T10:30:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.508+0000 7f3879200640 1 -- 172.21.15.42:0/1718752790 wait complete. 2024-05-27T10:30:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.509+0000 7f3879200640 1 Processor -- start 2024-05-27T10:30:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.509+0000 7f3879200640 1 -- start start 2024-05-27T10:30:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3879200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 0x7f387419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3879200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410bc60 0x7f387419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3879200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f387410c100 0x7f38741a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3879200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3874115d50 con 0x7f38741063b0 2024-05-27T10:30:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3879200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3874115bd0 con 0x7f387410c100 2024-05-27T10:30:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3879200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3874115ed0 con 0x7f387410bc60 2024-05-27T10:30:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3873400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 0x7f387419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3873400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 0x7f387419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36832/0 (socket says 172.21.15.42:36832) 2024-05-27T10:30:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3873400640 1 -- 172.21.15.42:0/555124052 learned_addr learned my addr 172.21.15.42:0/555124052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f3873e00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f387410c100 0x7f38741a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.510+0000 7f386aa00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410bc60 0x7f387419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873400640 1 -- 172.21.15.42:0/555124052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410bc60 msgr2=0x7f387419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873400640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410bc60 0x7f387419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873400640 1 -- 172.21.15.42:0/555124052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f387410c100 msgr2=0x7f38741a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873400640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f387410c100 0x7f38741a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873400640 1 -- 172.21.15.42:0/555124052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38741a1630 con 0x7f38741063b0 2024-05-27T10:30:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f386aa00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410bc60 0x7f387419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873e00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f387410c100 0x7f38741a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3873400640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 0x7f387419c7d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3864000f80 tx=0x7f3864032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.511+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864032e00 con 0x7f38741063b0 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.512+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38640044d0 con 0x7f38741063b0 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.512+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38741a18c0 con 0x7f38741063b0 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.512+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864030b80 con 0x7f38741063b0 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.512+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38741a92a0 con 0x7f38741063b0 2024-05-27T10:30:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.513+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387406a150 con 0x7f38741063b0 2024-05-27T10:30:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.515+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3864004680 con 0x7f38741063b0 2024-05-27T10:30:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.517+0000 7f3870c00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f38580b29d0 0x7f38580b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.517+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8528+0+0 (secure 0 0 0) 0x7f3864103c10 con 0x7f38741063b0 2024-05-27T10:30:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.517+0000 7f386aa00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f38580b29d0 0x7f38580b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.518+0000 7f386aa00640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f38580b29d0 0x7f38580b4ee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f385c005fd0 tx=0x7f385c005850 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.519+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38640cb6c0 con 0x7f38741063b0 2024-05-27T10:30:10.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:10 smithi143 ceph-mon[35545]: pgmap v356: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 37 op/s 2024-05-27T10:30:10.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:10 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/80564044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:30:10.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:10 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2196085836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T10:30:10.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:10 smithi042 ceph-mon[31661]: pgmap v356: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 37 op/s 2024-05-27T10:30:10.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:10 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/80564044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:30:10.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:10 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2196085836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T10:30:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:10 smithi104 ceph-mon[34812]: pgmap v356: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 37 op/s 2024-05-27T10:30:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:10 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/80564044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T10:30:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:10 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2196085836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T10:30:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.736+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38741a9470 con 0x7f38741063b0 2024-05-27T10:30:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.737+0000 7f3870c00640 1 -- 172.21.15.42:0/555124052 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f38640098b0 con 0x7f38741063b0 2024-05-27T10:30:10.735 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f38580b29d0 msgr2=0x7f38580b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f38580b29d0 0x7f38580b4ee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f385c005fd0 tx=0x7f385c005850 comp rx=0 tx=0).stop 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 msgr2=0x7f387419c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 0x7f387419c7d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3864000f80 tx=0x7f3864032a10 comp rx=0 tx=0).stop 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 shutdown_connections 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f38580b29d0 0x7f38580b4ee0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f387410c100 0x7f38741a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387410bc60 0x7f387419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 --2- 172.21.15.42:0/555124052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38741063b0 0x7f387419c7d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 >> 172.21.15.42:0/555124052 conn(0x7f3874101360 msgr2=0x7f3874101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 shutdown_connections 2024-05-27T10:30:10.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:10.740+0000 7f3879200640 1 -- 172.21.15.42:0/555124052 wait complete. 2024-05-27T10:30:10.769 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-27T10:30:11.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 -- 172.21.15.42:0/644165211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 msgr2=0x7f61b0116ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:11.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 --2- 172.21.15.42:0/644165211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b0116ad0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ca0 tx=0x7f61a802a8c0 comp rx=0 tx=0).stop 2024-05-27T10:30:11.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 -- 172.21.15.42:0/644165211 shutdown_connections 2024-05-27T10:30:11.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 --2- 172.21.15.42:0/644165211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b0116ad0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:11.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 --2- 172.21.15.42:0/644165211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010e870 0x7f61b010eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:11.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 --2- 172.21.15.42:0/644165211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010a2e0 0x7f61b010a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.073+0000 7f61b5e00640 1 -- 172.21.15.42:0/644165211 >> 172.21.15.42:0/644165211 conn(0x7f61b0076c80 msgr2=0x7f61b00770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.074+0000 7f61b5e00640 1 -- 172.21.15.42:0/644165211 shutdown_connections 2024-05-27T10:30:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.074+0000 7f61b5e00640 1 -- 172.21.15.42:0/644165211 wait complete. 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 Processor -- start 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 -- start start 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010a2e0 0x7f61b01a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010e870 0x7f61b01a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61b011a140 con 0x7f61b010f330 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0119fc0 con 0x7f61b010e870 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.075+0000 7f61b5e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f61b011a2c0 con 0x7f61b010a2e0 2024-05-27T10:30:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b01a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36856/0 (socket says 172.21.15.42:36856) 2024-05-27T10:30:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 -- 172.21.15.42:0/485357190 learned_addr learned my addr 172.21.15.42:0/485357190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:30:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61a7e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010e870 0x7f61b01a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b4a00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010a2e0 0x7f61b01a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 -- 172.21.15.42:0/485357190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010a2e0 msgr2=0x7f61b01a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010a2e0 0x7f61b01a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 -- 172.21.15.42:0/485357190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010e870 msgr2=0x7f61b01a6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010e870 0x7f61b01a6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 -- 172.21.15.42:0/485357190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b01a1000 con 0x7f61b010f330 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.076+0000 7f61b5400640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b01a0960 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005cc0 tx=0x7f61a8032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.077+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8032e00 con 0x7f61b010f330 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.077+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61a80044d0 con 0x7f61b010f330 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.077+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b01a1290 con 0x7f61b010f330 2024-05-27T10:30:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.077+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8030b70 con 0x7f61b010f330 2024-05-27T10:30:11.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.077+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01ad5d0 con 0x7f61b010f330 2024-05-27T10:30:11.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.078+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b006a150 con 0x7f61b010f330 2024-05-27T10:30:11.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.080+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f61a8004680 con 0x7f61b010f330 2024-05-27T10:30:11.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.083+0000 7f61ae000640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f61900b2950 0x7f61900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:30:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.083+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f61a8103ae0 con 0x7f61b010f330 2024-05-27T10:30:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.083+0000 7f61b4a00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f61900b2950 0x7f61900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:30:11.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.084+0000 7f61b4a00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f61900b2950 0x7f61900b4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6198005fd0 tx=0x7f6198005850 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:30:11.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.089+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61a80cb590 con 0x7f61b010f330 2024-05-27T10:30:11.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.357+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f61b0112150 con 0x7f61b010f330 2024-05-27T10:30:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.378+0000 7f61ae000640 1 -- 172.21.15.42:0/485357190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816172 (secure 0 0 0) 0x7f61a80cae90 con 0x7f61b010f330 2024-05-27T10:30:11.390 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-27T10:30:11.390 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:30:11.390 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 26, 2024-05-27T10:30:11.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.390 INFO:teuthology.orchestra.run.smithi042.stdout: "active_gid": 34892, 2024-05-27T10:30:11.390 INFO:teuthology.orchestra.run.smithi042.stdout: "active_name": "x", 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addrs": { 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3616349127 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3616349127 2024-05-27T10:30:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addr": "172.21.15.42:6801/3616349127", 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "active_change": "2024-05-27T10:18:42.838849+0000", 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "active_mgr_features": 4540701547738038271, 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "standbys": [ 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 24910, 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "z", 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-05-27T10:30:11.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.395 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.397 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.402 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-05-27T10:30:11.405 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.409 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T10:30:11.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T10:30:11.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T10:30:11.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-05-27T10:30:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T10:30:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-05-27T10:30:11.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.419 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-05-27T10:30:11.425 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-05-27T10:30:11.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-05-27T10:30:11.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-05-27T10:30:11.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T10:30:11.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-05-27T10:30:11.438 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T10:30:11.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T10:30:11.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.447 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-05-27T10:30:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-05-27T10:30:11.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-05-27T10:30:11.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.454 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-05-27T10:30:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.460 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.465 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-05-27T10:30:11.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-05-27T10:30:11.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-05-27T10:30:11.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-05-27T10:30:11.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-05-27T10:30:11.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.472 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.476 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.477 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-05-27T10:30:11.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-05-27T10:30:11.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.487 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-05-27T10:30:11.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-05-27T10:30:11.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-05-27T10:30:11.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-05-27T10:30:11.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.495 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.496 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T10:30:11.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.502 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T10:30:11.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T10:30:11.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T10:30:11.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T10:30:11.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T10:30:11.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.517 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.521 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.527 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-05-27T10:30:11.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T10:30:11.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T10:30:11.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.540 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T10:30:11.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.542 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T10:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.553 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-05-27T10:30:11.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-05-27T10:30:11.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-05-27T10:30:11.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-05-27T10:30:11.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.562 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.563 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.564 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.565 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-05-27T10:30:11.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.571 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.573 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.574 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.576 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.578 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-05-27T10:30:11.579 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-05-27T10:30:11.580 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.581 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.584 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.586 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-05-27T10:30:11.587 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-05-27T10:30:11.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-05-27T10:30:11.589 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.592 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.595 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T10:30:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-05-27T10:30:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-05-27T10:30:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-05-27T10:30:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-05-27T10:30:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T10:30:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-05-27T10:30:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-05-27T10:30:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.689 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.692 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.698 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.702 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.705 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-05-27T10:30:11.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.710 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T10:30:11.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.714 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.716 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.718 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:11.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-05-27T10:30:11.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-05-27T10:30:11.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-05-27T10:30:11.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.741 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.743 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.745 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T10:30:11.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.747 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-05-27T10:30:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.753 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.755 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:11.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.758 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.759 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.764 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-05-27T10:30:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.788 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-05-27T10:30:11.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-05-27T10:30:11.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smb", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.798 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "update_orchestration": { 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "update_orchestration", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.808 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.812 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.813 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.814 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.815 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-05-27T10:30:11.818 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.819 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:11.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.828 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.835 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-05-27T10:30:11.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-05-27T10:30:11.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-05-27T10:30:11.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-05-27T10:30:11.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-05-27T10:30:11.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.848 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.858 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.865 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.867 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.877 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.878 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.881 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-05-27T10:30:11.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 34883, 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "y", 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.890 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.891 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.892 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-05-27T10:30:11.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-05-27T10:30:11.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-05-27T10:30:11.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-05-27T10:30:11.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.898 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.899 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-05-27T10:30:11.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.916 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-05-27T10:30:11.919 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.921 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-05-27T10:30:11.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-05-27T10:30:11.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-05-27T10:30:11.928 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.932 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T10:30:11.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.935 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T10:30:11.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T10:30:11.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-05-27T10:30:11.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-05-27T10:30:11.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.949 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-05-27T10:30:11.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-05-27T10:30:11.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-05-27T10:30:11.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:11.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-05-27T10:30:11.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-05-27T10:30:11.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.966 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.967 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T10:30:11.972 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T10:30:11.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-05-27T10:30:11.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.975 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-05-27T10:30:11.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-05-27T10:30:11.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-05-27T10:30:11.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-05-27T10:30:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.981 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.987 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.988 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.989 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.994 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.995 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:11.997 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:11.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-05-27T10:30:11.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.001 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T10:30:12.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T10:30:12.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T10:30:12.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.014 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T10:30:12.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.017 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.022 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.024 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-05-27T10:30:12.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.029 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.030 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T10:30:12.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T10:30:12.032 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.033 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.034 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.041 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.044 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T10:30:12.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T10:30:12.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T10:30:12.048 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T10:30:12.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.050 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.051 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.052 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.053 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.057 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.058 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.060 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-05-27T10:30:12.065 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.068 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.072 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.074 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.077 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T10:30:12.083 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T10:30:12.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T10:30:12.085 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.088 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.089 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.092 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.095 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-05-27T10:30:12.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-05-27T10:30:12.099 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.100 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.101 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.103 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.104 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-05-27T10:30:12.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.110 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.112 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-05-27T10:30:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.115 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.116 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.119 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.120 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.121 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.122 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.123 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.124 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.125 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.129 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-05-27T10:30:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.131 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.137 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.139 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.140 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.143 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.144 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-05-27T10:30:12.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.151 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.153 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.154 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-05-27T10:30:12.157 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.158 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.159 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.161 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.162 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.163 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.164 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.165 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.167 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.172 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.173 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.176 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.177 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.178 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.179 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.181 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.183 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-05-27T10:30:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-05-27T10:30:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-05-27T10:30:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T10:30:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-05-27T10:30:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-05-27T10:30:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-05-27T10:30:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-05-27T10:30:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-05-27T10:30:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smb", 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "update_orchestration": { 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "update_orchestration", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.305 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.306 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.315 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.316 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.318 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.330 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-05-27T10:30:12.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-05-27T10:30:12.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.333 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.334 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.335 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-05-27T10:30:12.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.338 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.340 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.342 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.343 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.347 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.350 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.354 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.357 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.358 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.360 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.362 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.364 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.365 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.373 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.374 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-05-27T10:30:12.377 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard", 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "restful", 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_schedule" 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.381 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.382 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.383 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-05-27T10:30:12.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.387 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.390 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.392 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.393 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.394 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.397 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.398 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T10:30:12.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T10:30:12.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.403 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.406 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-05-27T10:30:12.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T10:30:12.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T10:30:12.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-05-27T10:30:12.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T10:30:12.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.425 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T10:30:12.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.434 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-05-27T10:30:12.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-05-27T10:30:12.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.439 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T10:30:12.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T10:30:12.443 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T10:30:12.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-05-27T10:30:12.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-05-27T10:30:12.450 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.453 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T10:30:12.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-05-27T10:30:12.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.466 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.468 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.471 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-05-27T10:30:12.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-05-27T10:30:12.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-05-27T10:30:12.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-05-27T10:30:12.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-05-27T10:30:12.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.479 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-05-27T10:30:12.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-05-27T10:30:12.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.485 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.488 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.489 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T10:30:12.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T10:30:12.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T10:30:12.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-05-27T10:30:12.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.497 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.498 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.499 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.510 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.513 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.514 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-05-27T10:30:12.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T10:30:12.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T10:30:12.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T10:30:12.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T10:30:12.526 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T10:30:12.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T10:30:12.528 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T10:30:12.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.530 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.531 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.535 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-05-27T10:30:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-05-27T10:30:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-05-27T10:30:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-05-27T10:30:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-05-27T10:30:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-05-27T10:30:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-05-27T10:30:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-05-27T10:30:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-05-27T10:30:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-05-27T10:30:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-05-27T10:30:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-05-27T10:30:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-05-27T10:30:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-05-27T10:30:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-05-27T10:30:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.663 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.665 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.667 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.670 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.679 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.683 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.687 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-05-27T10:30:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-05-27T10:30:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T10:30:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-05-27T10:30:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-05-27T10:30:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T10:30:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smb", 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "update_orchestration": { 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "update_orchestration", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-05-27T10:30:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-05-27T10:30:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-05-27T10:30:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-05-27T10:30:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-05-27T10:30:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard": "https://172.21.15.42:8443/" 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "always_on_modules": { 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "octopus": [ 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-05-27T10:30:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "pacific": [ 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-05-27T10:30:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "quincy": [ 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-05-27T10:30:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "reef": [ 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-05-27T10:30:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "squid": [ 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "last_failure_osd_epoch": 82, 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "active_clients": [ 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 123627288 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "libcephsqlite", 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 26767850 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:30:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1204644796 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2014290995 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1953166657 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-05-27T10:30:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550542167 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-27T10:30:12.875 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:30:12.876 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:11 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/555124052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T10:30:12.876 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:11 smithi104 ceph-mon[34812]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T10:30:12.876 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:12 smithi104 ceph-mon[34812]: pgmap v358: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 101 KiB/s wr, 46 op/s 2024-05-27T10:30:12.876 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:12 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/485357190' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T10:30:12.877 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:11 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/555124052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T10:30:12.877 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:11 smithi143 ceph-mon[35545]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T10:30:12.877 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:12 smithi143 ceph-mon[35545]: pgmap v358: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 101 KiB/s wr, 46 op/s 2024-05-27T10:30:12.877 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:12 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/485357190' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f61900b2950 msgr2=0x7f61900b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f61900b2950 0x7f61900b4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6198005fd0 tx=0x7f6198005850 comp rx=0 tx=0).stop 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 msgr2=0x7f61b01a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b01a0960 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005cc0 tx=0x7f61a8032a10 comp rx=0 tx=0).stop 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 shutdown_connections 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f61900b2950 0x7f61900b4e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:12.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b010f330 0x7f61b01a0960 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:12.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b010e870 0x7f61b01a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:12.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 --2- 172.21.15.42:0/485357190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61b010a2e0 0x7f61b01a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:30:12.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 >> 172.21.15.42:0/485357190 conn(0x7f61b0076c80 msgr2=0x7f61b0107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:30:12.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 shutdown_connections 2024-05-27T10:30:12.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:30:11.389+0000 7f61b5e00640 1 -- 172.21.15.42:0/485357190 wait complete. 2024-05-27T10:30:12.934 DEBUG:tasks.check_counter:Getting stats from a 2024-05-27T10:30:12.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-27T10:30:12.934 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:11 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/555124052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T10:30:12.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:11 smithi042 ceph-mon[31661]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T10:30:12.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:12 smithi042 ceph-mon[31661]: pgmap v358: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 101 KiB/s wr, 46 op/s 2024-05-27T10:30:12.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:12 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/485357190' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T10:30:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:30:14.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:14 smithi143 ceph-mon[35545]: pgmap v359: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:30:14.579 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:14 smithi042 ceph-mon[31661]: pgmap v359: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:30:14.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:14 smithi104 ceph-mon[34812]: pgmap v359: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.005081031, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000065664, 2024-05-27T10:30:16.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001479719, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001902489, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000634163 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.129 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 274, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 263, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1725760, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 162320, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.121669708, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.038355943, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.105555328, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011582425, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 262, 2024-05-27T10:30:16.130 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.023268290, 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000088810 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.131 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1725760, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 162320 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 201, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 192768, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.037769756, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.026048548, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005706679, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:16.132 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 198, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.027430123, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000138535 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 192768 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-05-27T10:30:16.133 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:16.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:16.135 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148754, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 31, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:16.136 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:16.137 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-05-27T10:30:16.138 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 276, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1687092, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 276, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 276, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1687092, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:16.139 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:30:16.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:16 smithi143 ceph-mon[35545]: pgmap v360: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-27T10:30:16.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:16 smithi042 ceph-mon[31661]: pgmap v360: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-27T10:30:16.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:16 smithi104 ceph-mon[34812]: pgmap v360: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-27T10:30:17.423 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-05-27T10:30:17.423 DEBUG:tasks.check_counter:Getting stats from d 2024-05-27T10:30:17.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-27T10:30:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:30:18.323 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:18 smithi042 ceph-mon[31661]: pgmap v361: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-27T10:30:18.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:18 smithi143 ceph-mon[35545]: pgmap v361: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-27T10:30:18.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:18 smithi104 ceph-mon[34812]: pgmap v361: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-27T10:30:20.370 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:20 smithi042 ceph-mon[31661]: pgmap v362: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-27T10:30:20.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:20 smithi143 ceph-mon[35545]: pgmap v362: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-27T10:30:20.640 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:30:20.640 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:20.640 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-05-27T10:30:20.640 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-05-27T10:30:20.640 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-05-27T10:30:20.640 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.003985837, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000872508, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 208, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-05-27T10:30:20.642 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 194000, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.038094827, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.026456338, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004784702, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 205, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.037093003, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000180941 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 194000 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 288, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 275, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1731568, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 168832, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-05-27T10:30:20.644 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.127753797, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.039727107, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.112099360, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011820699, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 275, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.025377078, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000092280 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:20.645 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1731568, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 168832 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:20.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:20.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:20.648 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148754, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 31, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-05-27T10:30:20.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-05-27T10:30:20.650 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 286, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1688495, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 286, 2024-05-27T10:30:20.651 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 286, 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1688495, 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:20.652 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:30:20.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:20 smithi104 ceph-mon[34812]: pgmap v362: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-27T10:30:21.660 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-27T10:30:21.660 DEBUG:tasks.check_counter:Getting stats from g 2024-05-27T10:30:21.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-27T10:30:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:30:22.535 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:22 smithi042 ceph-mon[31661]: pgmap v363: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 997 B/s wr, 6 op/s 2024-05-27T10:30:22.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:22 smithi143 ceph-mon[35545]: pgmap v363: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 997 B/s wr, 6 op/s 2024-05-27T10:30:22.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:22 smithi104 ceph-mon[34812]: pgmap v363: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 997 B/s wr, 6 op/s 2024-05-27T10:30:24.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:24 smithi143 ceph-mon[35545]: pgmap v364: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-05-27T10:30:24.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:24 smithi042 ceph-mon[31661]: pgmap v364: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-05-27T10:30:24.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:24 smithi104 ceph-mon[34812]: pgmap v364: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-05-27T10:30:24.888 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:30:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-05-27T10:30:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.005265407, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000069202, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001537872, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-05-27T10:30:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001889964, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000629988 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 279, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 268, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1727216, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 165184, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.114166314, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.038823400, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.089302232, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011015971, 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 267, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.023576641, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000088302 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1727216, 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 165184 2024-05-27T10:30:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 204, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 193296, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.035626825, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025399874, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004052701, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 201, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.031318287, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000155812 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T10:30:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 193296 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148714, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-05-27T10:30:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-05-27T10:30:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 281, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1687845, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 281, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 281, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1687845, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:24.902 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-27T10:30:25.846 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-05-27T10:30:25.847 DEBUG:tasks.check_counter:Getting stats from j 2024-05-27T10:30:25.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-27T10:30:26.331 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:30:26.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:26 smithi143 ceph-mon[35545]: pgmap v365: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-27T10:30:26.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:26 smithi042 ceph-mon[31661]: pgmap v365: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-27T10:30:26.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:26 smithi104 ceph-mon[34812]: pgmap v365: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-27T10:30:28.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:28 smithi042 ceph-mon[31661]: pgmap v366: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:28.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:28 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:30:28.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:28 smithi143 ceph-mon[35545]: pgmap v366: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:28.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:28 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:30:28.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:28 smithi104 ceph-mon[34812]: pgmap v366: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:28.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:28 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T10:30:28.969 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-27T10:30:28.969 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:28.969 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 287, 2024-05-27T10:30:28.969 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 276, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1729392, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 170288, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.118901082, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.040425290, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.087712867, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011741549, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 275, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.025073926, 2024-05-27T10:30:28.970 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000091177 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1729392, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 170288 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 6, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 213, 2024-05-27T10:30:28.971 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 3312, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 194976, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.040789838, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.028300043, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005883473, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 211, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.027761359, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000131570 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:28.972 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 194976 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-05-27T10:30:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.000778716, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000649884, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:28.975 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:28.976 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:28.977 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148714, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-05-27T10:30:28.978 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-05-27T10:30:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 289, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1688885, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 289, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 289, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1688885, 2024-05-27T10:30:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-05-27T10:30:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-05-27T10:30:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-05-27T10:30:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-05-27T10:30:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-27T10:30:28.981 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-27T10:30:29.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:29 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:30:29.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:29 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:30:29.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:29 smithi143 ceph-mon[35545]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:30:29.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:29 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:30:29.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:29 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:30:29.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:29 smithi042 ceph-mon[31661]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:30:29.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:29 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T10:30:29.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:29 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T10:30:29.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:29 smithi104 ceph-mon[34812]: from='mgr.34892 172.21.15.42:0/1243279194' entity='mgr.x' 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-27T10:30:30.245 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-27T10:30:30.245 DEBUG:tasks.check_counter:Getting stats from b 2024-05-27T10:30:30.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-27T10:30:30.702 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:30:31.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:30 smithi104 ceph-mon[34812]: pgmap v367: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:31.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:30 smithi143 ceph-mon[35545]: pgmap v367: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:31.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:30 smithi042 ceph-mon[31661]: pgmap v367: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:33.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:32 smithi104 ceph-mon[34812]: pgmap v368: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:33.272 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-27T10:30:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2234, 2024-05-27T10:30:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1366, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2523216, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 4828024, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 176, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.715472521, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.181385879, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.392715801, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.117118040, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1362, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.477106056, 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000350298 2024-05-27T10:30:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1474, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000505596, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000343 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2319792, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 313264 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 20985, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 20833, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 6720209, 2024-05-27T10:30:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 6067823, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 16, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 4.432335107, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 1.844080257, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 3.018845640, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.134570383, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 20834, 2024-05-27T10:30:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.892523601, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000090838 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 39619, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.019418590, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000490 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 461328 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 19022, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 12588, 2024-05-27T10:30:33.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 9054914, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 8074459, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 4.116653849, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 2.092629626, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 2.515878802, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.131527393, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 12586, 2024-05-27T10:30:33.277 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.752929907, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000139276 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 27807, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012457554, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000448 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 42560, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 1936 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-27T10:30:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2563, 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.831976994, 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001104946 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 520, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.089041698, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000171234 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 15517, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 2958, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2958, 2024-05-27T10:30:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 39.490285731, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.013350333 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 12559, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 4, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 1, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 355, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 1, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 84, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 9, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 65, 2024-05-27T10:30:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 10, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 71, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 870, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 3, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 3, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 3, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 20531, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 5511, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 12505, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 216, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-05-27T10:30:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 1, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 11, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 497, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 3, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 235, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 50, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-05-27T10:30:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 50, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 33, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 4, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 29, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 17, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 4, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 671, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 749, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T10:30:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 2337, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 229, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 32, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 1356 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 62, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 1598, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 855, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 1238, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 62, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 3, 2024-05-27T10:30:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 294, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 291, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 8, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 524, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 4, 2024-05-27T10:30:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 11, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 1, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 3779, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 3766, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 3766, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 13, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-05-27T10:30:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 85, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 52, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 51, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 51, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 1, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 1, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 1, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 10750526, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 10816060, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 858, 2024-05-27T10:30:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4.909617731, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005722165 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 51, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 797, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 746, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 23, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 1023, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 1000, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 84, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 1441, 2024-05-27T10:30:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 1357, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 3, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 382, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 379, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 132848, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 231824 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 17163, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 729, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 15517, 2024-05-27T10:30:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 38, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 2233, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 250, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 20.637436114, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.082549744 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 470, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.988400812, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006358299 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1686, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 5.432458821, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003222098 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-05-27T10:30:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.005804246, 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000527658 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 15, 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.055226590, 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003681772 2024-05-27T10:30:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.301886279, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.118353298 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 16, 2024-05-27T10:30:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.014626046, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000914127 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 22, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.148684936, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006758406 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 168, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4.582751923, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.027278285 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.079987994, 2024-05-27T10:30:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.019996998 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.858684617, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.571736923 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-05-27T10:30:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001138364, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001138364 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-05-27T10:30:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 8, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.058542520, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.007317815 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-05-27T10:30:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.002262406, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002262406 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 290, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.322394063, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004559979 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 6, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 4, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 438, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 219, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 669, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 360, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 360, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 327735, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 113, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7936, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 165, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 270276, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 1407, 2024-05-27T10:30:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 2849, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 8114733, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 37, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 2849, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2849, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 44.407487612, 2024-05-27T10:30:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.015587043 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2849, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4440 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 2849, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 78, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 2771, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 321, 2024-05-27T10:30:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 391, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 19, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 873, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 186, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 1, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 805, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 59, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 782, 2024-05-27T10:30:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-05-27T10:30:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 93, 2024-05-27T10:30:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 14, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 913, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 10, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 80 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-05-27T10:30:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 630 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 10, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 10, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 291, 2024-05-27T10:30:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 291, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 42237, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 14421454, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 42237, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 42237, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 14421454, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2849, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 8147120, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2849, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 8147120, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2849, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 2849, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2849, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2849, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 291, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 32455, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 291, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 15, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 32455, 2024-05-27T10:30:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55a16beb49c0": { 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 3779, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 6621756, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 3779, 2024-05-27T10:30:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 858, 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 6621756, 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:33.321 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-27T10:30:33.322 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:32 smithi143 ceph-mon[35545]: pgmap v368: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:33.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:32 smithi042 ceph-mon[31661]: pgmap v368: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=62 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=1598 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-05-27T10:30:34.333 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-05-27T10:30:34.334 DEBUG:tasks.check_counter:Getting stats from e 2024-05-27T10:30:34.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-27T10:30:34.758 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:30:35.197 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:35 smithi104 ceph-mon[34812]: pgmap v369: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:35.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:35 smithi143 ceph-mon[35545]: pgmap v369: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:35.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:35 smithi042 ceph-mon[31661]: pgmap v369: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:37.195 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:37 smithi104 ceph-mon[34812]: pgmap v370: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 31693, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 18053, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 10430835, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 10339623, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 18, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 4.490780124, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 1.869643209, 2024-05-27T10:30:37.200 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 3.411113833, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.040452442, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18051, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.225017083, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000067864 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 47214, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.019585958, 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000414 2024-05-27T10:30:37.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 736 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 862, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 974, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1967978, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 2669318, 2024-05-27T10:30:37.202 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.251245483, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.092337771, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.149866323, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.037779459, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 974, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.138265551, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000141956 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.203 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1835344, 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 523056 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 15678, 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 11149, 2024-05-27T10:30:37.204 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 8137471, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 3814524, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 19, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 3.479368316, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 1.847526492, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 2.119469773, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.020116762, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11146, 2024-05-27T10:30:37.205 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.890630497, 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000079905 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 25584, 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012744604, 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000498 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 5712, 2024-05-27T10:30:37.206 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 157936 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1708, 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.129149270, 2024-05-27T10:30:37.207 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001246574 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1004, 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.143315678, 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000142744 2024-05-27T10:30:37.208 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 14137, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 5153, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5153, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 65.091842701, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.012631834 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 8984, 2024-05-27T10:30:37.209 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 20, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 1, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 18, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 3, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 5, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 10, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 13, 2024-05-27T10:30:37.210 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 1146, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 1, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 1, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 4, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 22095, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 8860, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 8982, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 114, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-05-27T10:30:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 9, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 846, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 2, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 330, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 694, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 14165281, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 3, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-05-27T10:30:37.212 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 2, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 848, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 1008, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 16715, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 265, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 48, 2024-05-27T10:30:37.213 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 1769 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 178, 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 1185, 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 704, 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 746, 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 280, 2024-05-27T10:30:37.214 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 429, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 429, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-05-27T10:30:37.215 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 8, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 1370, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 9, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 1, 2024-05-27T10:30:37.216 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 6980, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 6980, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-05-27T10:30:37.217 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 9, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 9, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 10876710, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 762, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.496505107, 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004588589 2024-05-27T10:30:37.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 23, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 963, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 940, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 22, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 636, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 614, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 18, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 1621, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 1603, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 1, 2024-05-27T10:30:37.219 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 436, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 435, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 132820, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223632 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 16040, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 13, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 14137, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 27, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 1133, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 265, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 49.410102020, 2024-05-27T10:30:37.220 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.186453215 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.002312283, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000578070 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-05-27T10:30:37.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4260, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 7.190678927, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001687952 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.224 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.225 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.226 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 66, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.050981272, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000772443 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 13, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020511415, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001577801 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 265, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 7.195281329, 2024-05-27T10:30:37.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.027152005 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-05-27T10:30:37.228 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-05-27T10:30:37.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 280, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.221975455, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004364198 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-05-27T10:30:37.232 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 4, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 740, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 370, 2024-05-27T10:30:37.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 672, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:37.234 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:37.235 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:37.236 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 296151, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 103, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7936, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 165, 2024-05-27T10:30:37.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 152981, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 506, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-05-27T10:30:37.238 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 1579, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 6893207, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 29, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 1579, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1579, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 8.088906622, 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005122803 2024-05-27T10:30:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1579, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1750 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 1579, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 79, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 1500, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-05-27T10:30:37.240 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 79, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 841, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 73, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 1, 2024-05-27T10:30:37.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 442, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-05-27T10:30:37.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-05-27T10:30:37.243 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 93, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-05-27T10:30:37.244 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 14, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 212, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-05-27T10:30:37.245 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 72 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-05-27T10:30:37.246 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 1370 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 10, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 10, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 429, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 429, 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-05-27T10:30:37.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 48229, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 16173770, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 48229, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 48229, 2024-05-27T10:30:37.248 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 16173770, 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-05-27T10:30:37.249 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 1579, 2024-05-27T10:30:37.250 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 6947088, 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1579, 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 6947088, 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-05-27T10:30:37.251 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 1579, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 1579, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1579, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1579, 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:37.252 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 429, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 53881, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 429, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:37.253 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 79, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 53881, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x556faf3ee8f0": { 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:37.254 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 6980, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 6682406, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 6980, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 762, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 6682406, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:37.255 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-27T10:30:37.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:37 smithi143 ceph-mon[35545]: pgmap v370: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:37.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:37 smithi042 ceph-mon[31661]: pgmap v370: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:38.193 INFO:tasks.check_counter:Daemon mds.e mds.exported=9 2024-05-27T10:30:38.193 INFO:tasks.check_counter:Daemon mds.e mds.imported=2 2024-05-27T10:30:38.193 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=178 2024-05-27T10:30:38.193 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=1185 2024-05-27T10:30:38.193 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=3 2024-05-27T10:30:38.194 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-27T10:30:38.194 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T10:30:38.194 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=1 2024-05-27T10:30:38.194 DEBUG:tasks.check_counter:Getting stats from h 2024-05-27T10:30:38.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-27T10:30:38.645 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:30:39.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:39 smithi104 ceph-mon[34812]: pgmap v371: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:39.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:39 smithi143 ceph-mon[35545]: pgmap v371: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:39.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:39 smithi042 ceph-mon[31661]: pgmap v371: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:40.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:40 smithi042 ceph-mon[31661]: pgmap v372: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:40.395 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:40 smithi104 ceph-mon[34812]: pgmap v372: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:40.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:40 smithi143 ceph-mon[35545]: pgmap v372: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:41.132 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-27T10:30:41.132 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:41.132 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 605067, 2024-05-27T10:30:41.132 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 239596, 2024-05-27T10:30:41.132 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 208141154, 2024-05-27T10:30:41.132 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 243058348, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 24, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 53.557647866, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 23.846057766, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 42.838235542, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.431145932, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 239592, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 14.469064197, 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000060390 2024-05-27T10:30:41.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 769706, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.284188653, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000369 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1856, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 1712 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 18281, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 24748, 2024-05-27T10:30:41.134 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 6478648, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 106758755, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 5.352824609, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 2.830210081, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 2.282834070, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.482524255, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 24749, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.532014235, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000142713 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:41.135 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 27352, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.015646339, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000572 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1836848, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 508416 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 20584, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 25087, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 7211342, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 161882075, 2024-05-27T10:30:41.136 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 14, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 5.188042213, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 2.589131422, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 1.913170502, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.749584100, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 25085, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4.410436567, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000175819 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 21714, 2024-05-27T10:30:41.137 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.010262054, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000472 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 7168, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 149248 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:41.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 12392, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 105.848659917, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.008541693 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 19715, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 9.476329278, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000480665 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.139 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 126025, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 123363, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 123363, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2228.379975436, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.018063600 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 2662, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 2, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 667, 2024-05-27T10:30:41.140 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 47, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 21041, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 1089, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 19873, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 79, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 19483, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 860, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 3, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 3, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 21, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 225044, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 134187, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 2631, 2024-05-27T10:30:41.141 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 5, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 14, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 5, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 565, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 20, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 1343, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-05-27T10:30:41.142 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 48, 2024-05-27T10:30:41.143 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-05-27T10:30:41.143 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 694, 2024-05-27T10:30:41.143 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 14165281, 2024-05-27T10:30:41.143 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 3, 2024-05-27T10:30:41.143 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 48, 2024-05-27T10:30:41.143 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-05-27T10:30:41.144 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T10:30:41.144 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-05-27T10:30:41.144 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 3, 2024-05-27T10:30:41.144 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T10:30:41.144 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 3, 2024-05-27T10:30:41.144 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 45, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 14, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 35023, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 25214, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 5, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 470468, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 11822, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 321, 2024-05-27T10:30:41.145 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 66611 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 990, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 17, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 11, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 12, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 1654, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 64, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-05-27T10:30:41.146 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 11476, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 11412, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 8, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 29126, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T10:30:41.147 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 9, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 47, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 206009, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 101284, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 98321, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 107688, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 2963, 2024-05-27T10:30:41.148 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 243, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 115, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 112, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 131, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 9, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 3, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 149851692, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 369446051, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7605, 2024-05-27T10:30:41.149 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 127.993240582, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.016830143 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 10124, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 21858, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 11734, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 406, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 492, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 86, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 21041, 2024-05-27T10:30:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 41912, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 20871, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 3, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 11741, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 11738, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 202708, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223632 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 132688, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 1810, 2024-05-27T10:30:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 126025, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 30, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 4308, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11609, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 518.426407271, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.044657283 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 499, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.983017241, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005977990 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.152 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-05-27T10:30:41.153 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 85431, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1052.512406560, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.012320029 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.154 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-05-27T10:30:41.155 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.156 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.156 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 67, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.686770369, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.010250304 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.157 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4222, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 163.155980883, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.038644239 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.158 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 709, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.468501645, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002071229 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11313, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 398.563827338, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.035230604 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.159 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 72, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 31.256929905, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.434124026 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-05-27T10:30:41.160 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-05-27T10:30:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.008961226, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004480613 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 9439, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 59.317172998, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006284264 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-05-27T10:30:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 5, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 4, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 16303, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 16303, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 720, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 817505, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 234, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7936, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 165, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 37095148, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 221185, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-27T10:30:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 27267, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 380318394, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 32, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 27267, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 27267, 2024-05-27T10:30:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 281.241751156, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.010314363 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 27267, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 39635 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 27267, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 704, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 26563, 2024-05-27T10:30:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 227, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 5267, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 654, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 8259, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 351, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 47, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 11569, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-05-27T10:30:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 50, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 10534, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 5, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-05-27T10:30:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-05-27T10:30:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 93, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 14, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 2065, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 4, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 603 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-05-27T10:30:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 29436 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 18, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 10, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 11436, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 11412, 2024-05-27T10:30:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 643928, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 163855610, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 643928, 2024-05-27T10:30:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 643928, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 163855610, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 27267, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 381740406, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 27267, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 381740406, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 27267, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 27267, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 27267, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 27267, 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 11412, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1422044, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 11412, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 654, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1422044, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x558dd3918b60": { 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 206009, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 365251747, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 206009, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 7605, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 365251747, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:41.183 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-27T10:30:42.226 INFO:tasks.check_counter:Daemon mds.h mds.exported=5 2024-05-27T10:30:42.226 INFO:tasks.check_counter:Daemon mds.h mds.imported=20 2024-05-27T10:30:42.226 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=990 2024-05-27T10:30:42.227 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=17 2024-05-27T10:30:42.227 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=3 2024-05-27T10:30:42.227 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-27T10:30:42.227 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T10:30:42.227 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=47 2024-05-27T10:30:42.227 DEBUG:tasks.check_counter:Getting stats from k 2024-05-27T10:30:42.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-27T10:30:42.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:42 smithi104 ceph-mon[34812]: pgmap v373: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:42.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:42 smithi143 ceph-mon[35545]: pgmap v373: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:42.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:42 smithi042 ceph-mon[31661]: pgmap v373: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:42.683 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T10:30:44.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:44 smithi104 ceph-mon[34812]: pgmap v374: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:44.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:44 smithi143 ceph-mon[35545]: pgmap v374: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:44.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:44 smithi042 ceph-mon[31661]: pgmap v374: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:45.192 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 157, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 157, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 58087, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 49112, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 2, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.044561048, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.017899526, 2024-05-27T10:30:45.193 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.010888183, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.009425977, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 156, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020566248, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000131834 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 213, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 208, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 237676, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 76670, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.056934595, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.017358791, 2024-05-27T10:30:45.195 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.035271408, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.013375147, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 207, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.024880219, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000120194 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.196 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 198208, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 27632 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 182, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 215, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1043082, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 168727, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.049812778, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.016514794, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.018384583, 2024-05-27T10:30:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.010201850, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 213, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.019836894, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000093130 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-27T10:30:45.198 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 111168 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 779, 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.048042319, 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000061671 2024-05-27T10:30:45.199 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-05-27T10:30:45.200 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-05-27T10:30:45.201 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 146, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 58, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 88, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 63, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 2, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-05-27T10:30:45.202 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-05-27T10:30:45.203 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-05-27T10:30:45.204 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-05-27T10:30:45.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-05-27T10:30:45.206 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-05-27T10:30:45.207 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 200, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-05-27T10:30:45.208 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 9, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 10750526, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 10816060, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 10807718, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.209 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 200 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 39, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 189, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 150, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 11, 2024-05-27T10:30:45.210 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 16, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 5, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 146, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 295, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 149, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 119512, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215440 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-05-27T10:30:45.211 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.212 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.213 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.214 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.215 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.216 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.217 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.218 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.218 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-05-27T10:30:45.219 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-05-27T10:30:45.220 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-05-27T10:30:45.221 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.222 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.223 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.224 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.225 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.226 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 0, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 0, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-05-27T10:30:45.227 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2392, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:45.228 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:45.229 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:45.230 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 188439, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 45, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7936, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 165, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:45.231 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 244916, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 1243, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 488, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 26, 2024-05-27T10:30:45.232 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 4, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 488, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 488, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.641992603, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001315558 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 488, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 489 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 488, 2024-05-27T10:30:45.233 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 488, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 300, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 185, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-05-27T10:30:45.234 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 2, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T10:30:45.235 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-05-27T10:30:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 93, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 4, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 5, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 5, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-05-27T10:30:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-05-27T10:30:45.238 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 548, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1283575, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 548, 2024-05-27T10:30:45.239 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 548, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1283575, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 488, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 995437, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 488, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 995437, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 488, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 488, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 488, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 488, 2024-05-27T10:30:45.242 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:45.243 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.244 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55fa1df1d2b0": { 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-27T10:30:45.245 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-27T10:30:45.246 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T10:30:46.226 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-27T10:30:46.226 DEBUG:tasks.check_counter:Getting stats from c 2024-05-27T10:30:46.227 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-27T10:30:46.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:46 smithi104 ceph-mon[34812]: pgmap v375: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:46.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:46 smithi143 ceph-mon[35545]: pgmap v375: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:46.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:46 smithi042 ceph-mon[31661]: pgmap v375: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:46.681 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:30:48.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:48 smithi143 ceph-mon[35545]: pgmap v376: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:48.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:48 smithi104 ceph-mon[34812]: pgmap v376: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:48.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:48 smithi042 ceph-mon[31661]: pgmap v376: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:49.146 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-27T10:30:49.146 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 848, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 848, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 4417511, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 268904, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 2, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.206836131, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.062531195, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.069334096, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.050017359, 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:49.147 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 847, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.085904192, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000101421 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.148 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 903, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1113, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 2713866, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 744459, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.247523962, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.085814100, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.067584657, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.065249594, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1110, 2024-05-27T10:30:49.149 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.121349233, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000109323 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 457936 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:49.150 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1505, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1478, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 2197640, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 550400, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.412425477, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.134959950, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.187794493, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.094154102, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1477, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.160677958, 2024-05-27T10:30:49.151 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000108786 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1819616, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 173888 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-27T10:30:49.152 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4801, 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.294884348, 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000061421 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-27T10:30:49.153 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.154 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 437, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 437, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 437, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-05-27T10:30:49.155 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-05-27T10:30:49.156 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T10:30:49.157 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-05-27T10:30:49.158 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-05-27T10:30:49.159 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-05-27T10:30:49.160 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 6980, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-05-27T10:30:49.161 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 9, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 10876710, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 10872257, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.162 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 6980 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 11, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 841, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 830, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 9, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 33, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 24, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 437, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 1399, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 962, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-05-27T10:30:49.163 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 121796, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215440 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.164 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-05-27T10:30:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.166 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.168 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-05-27T10:30:49.169 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.169 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.169 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.169 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.169 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.170 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.171 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.172 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-05-27T10:30:49.173 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.174 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.175 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.176 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 2, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 4, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 2, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 3, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 3245, 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-05-27T10:30:49.177 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:49.178 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:49.179 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 200056, 2024-05-27T10:30:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 59, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 7936, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 165, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:49.181 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 358488, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 1881, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 2929, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-05-27T10:30:49.182 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 2929, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2929, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 3.507907539, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001197646 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2929, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2930 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 2929, 2024-05-27T10:30:49.183 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 2929, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 1839, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 1088, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-05-27T10:30:49.184 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T10:30:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-05-27T10:30:49.186 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 93, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 12, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 6, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 7, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 1, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-05-27T10:30:49.187 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-05-27T10:30:49.188 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 3252, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 9016957, 2024-05-27T10:30:49.189 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 3252, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 3252, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 9016957, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-05-27T10:30:49.190 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2929, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 6682411, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2929, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 6682411, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.191 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2929, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 2929, 2024-05-27T10:30:49.192 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2929, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2929, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:49.193 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x559398464ea0": { 2024-05-27T10:30:49.194 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:49.195 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:49.196 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.196 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:49.196 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-27T10:30:50.024 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T10:30:50.025 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-27T10:30:50.025 DEBUG:tasks.check_counter:Getting stats from f 2024-05-27T10:30:50.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-27T10:30:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:30:50.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:50 smithi143 ceph-mon[35545]: pgmap v377: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:50.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:50 smithi042 ceph-mon[31661]: pgmap v377: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:50.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:50 smithi104 ceph-mon[34812]: pgmap v377: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:52.430 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:52 smithi143 ceph-mon[35545]: pgmap v378: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:52.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:52 smithi042 ceph-mon[31661]: pgmap v378: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:52.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:52 smithi104 ceph-mon[34812]: pgmap v378: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:52.917 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-27T10:30:52.917 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:52.917 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 294, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 281, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1733200, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 172672, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 0, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.141513189, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.043488209, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.114541263, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.014620631, 2024-05-27T10:30:52.918 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 281, 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.031099597, 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000110674 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:52.922 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1733200, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 172672 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 2, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 213, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 320, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 129280, 2024-05-27T10:30:52.923 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.030615182, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.019765707, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.004173004, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 210, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.042271608, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000201293 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:52.924 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 129280 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 0, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 2, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 0, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 0, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 2, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-05-27T10:30:52.925 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.000603727, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.000466986, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:52.926 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 0, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:52.927 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:52.928 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:52.929 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 148744, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 30, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-27T10:30:52.930 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 928, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 2, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:52.931 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 292, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 1689275, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 292, 2024-05-27T10:30:52.932 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 292, 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1689275, 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:52.933 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-27T10:30:53.732 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-05-27T10:30:53.732 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-05-27T10:30:53.732 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-05-27T10:30:53.732 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-05-27T10:30:53.732 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-05-27T10:30:53.732 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-05-27T10:30:53.733 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-05-27T10:30:53.733 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-05-27T10:30:53.733 DEBUG:tasks.check_counter:Getting stats from i 2024-05-27T10:30:53.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-27T10:30:54.176 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:30:54.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:54 smithi143 ceph-mon[35545]: pgmap v379: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:54.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:54 smithi042 ceph-mon[31661]: pgmap v379: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:54.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:54 smithi104 ceph-mon[34812]: pgmap v379: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:56.456 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:56 smithi143 ceph-mon[35545]: pgmap v380: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:56.460 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-27T10:30:56.460 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1723, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1696, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 73909400, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 619724, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.681684716, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.152497467, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.427687709, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.105190431, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1695, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.184170300, 2024-05-27T10:30:56.461 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000108655 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1819616, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 173888 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 445, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 652, 2024-05-27T10:30:56.462 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 100784797, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 631344, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 9, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 7, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.422012599, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.054657214, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.324036529, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.026828283, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 650, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.089725240, 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000138038 2024-05-27T10:30:56.463 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 491680 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1727, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1729, 2024-05-27T10:30:56.464 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 193387536, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 549193, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 6, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.896608005, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.131729902, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.629048415, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.106313905, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1727, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.163811763, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000094853 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:30:56.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-05-27T10:30:56.466 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 5785, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.452130285, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000078155 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-05-27T10:30:56.468 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 39527, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 5091, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 34436, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 32254, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-05-27T10:30:56.469 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-05-27T10:30:56.470 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-05-27T10:30:56.471 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-05-27T10:30:56.472 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 0, 2024-05-27T10:30:56.473 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-05-27T10:30:56.474 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 206009, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 131, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 18, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 149851692, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 369446051, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 369435878, 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-05-27T10:30:56.475 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 206009 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 10141, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 21841, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 11700, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 397, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 485, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 88, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 39527, 2024-05-27T10:30:56.476 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 41227, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 1700, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 176956, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215440 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-05-27T10:30:56.477 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.478 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.479 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.480 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-05-27T10:30:56.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.482 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.482 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.482 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.482 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-05-27T10:30:56.482 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.482 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.484 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-05-27T10:30:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.487 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.488 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-05-27T10:30:56.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 1, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 5, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 4, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 4, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-05-27T10:30:56.490 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 7460, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:30:56.491 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:30:56.492 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 239505, 2024-05-27T10:30:56.493 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 80, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 7936, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 165, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:30:56.494 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 49741688, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 362917, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 3568, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-05-27T10:30:56.495 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 3568, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3568, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 6.254834997, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001753036 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3568, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 3569 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 3568, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 3568, 2024-05-27T10:30:56.496 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 2147, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 1419, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-05-27T10:30:56.497 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-05-27T10:30:56.498 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-05-27T10:30:56.499 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 93, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 12, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 11, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 11, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-05-27T10:30:56.500 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-05-27T10:30:56.501 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 3891, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 367712163, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 3891, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 3891, 2024-05-27T10:30:56.502 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 367712163, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:56.503 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 3568, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 365269287, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 3568, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 365269287, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-05-27T10:30:56.504 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 3568, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 3568, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 3568, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 3568, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.505 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-27T10:30:56.506 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x5650192469c0": { 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:30:56.507 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:30:56.508 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-27T10:30:56.594 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:56 smithi042 ceph-mon[31661]: pgmap v380: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:56.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:56 smithi104 ceph-mon[34812]: pgmap v380: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:30:57.392 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T10:30:57.393 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-05-27T10:30:57.393 DEBUG:tasks.check_counter:Getting stats from l 2024-05-27T10:30:57.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-27T10:30:57.840 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T10:30:58.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:30:58 smithi143 ceph-mon[35545]: pgmap v381: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:58.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:30:58 smithi042 ceph-mon[31661]: pgmap v381: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:30:58.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:30:58 smithi104 ceph-mon[34812]: pgmap v381: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:00.329 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 0, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 2, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 0, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 0, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 0, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.005658448, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.000862783, 2024-05-27T10:31:00.330 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T10:31:00.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 297, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 286, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1732112, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 176704, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 2, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.146363544, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.044602938, 2024-05-27T10:31:00.332 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.123391315, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.015235273, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 285, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.031411163, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000110214 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1732112, 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 176704 2024-05-27T10:31:00.333 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 6, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 221, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 3312, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 130784, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.031832794, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.020111881, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.005210525, 2024-05-27T10:31:00.334 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 219, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.031496529, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000143819 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-27T10:31:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 130784 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 0, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-27T10:31:00.336 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-27T10:31:00.337 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-27T10:31:00.338 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 148714, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 30, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-27T10:31:00.339 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-27T10:31:00.340 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 928, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 2, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 299, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 1690185, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-27T10:31:00.341 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 299, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 299, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1690185, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-27T10:31:00.342 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-27T10:31:00.593 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:00 smithi042 ceph-mon[31661]: pgmap v382: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:31:00.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:00 smithi104 ceph-mon[34812]: pgmap v382: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:31:00.704 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:00 smithi143 ceph-mon[35545]: pgmap v382: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T10:31:01.131 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-27T10:31:01.131 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-27T10:31:01.131 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-27T10:31:01.131 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-27T10:31:01.131 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-27T10:31:01.132 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-27T10:31:01.132 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-27T10:31:01.132 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-05-27T10:31:01.132 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-27T10:31:01.145 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-27T10:31:01.145 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:31:01.145 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T10:31:01.211 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:31:01.212 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T10:31:01.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T10:31:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:31:02.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:02 smithi042 ceph-mon[31661]: pgmap v383: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:02.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:02 smithi104 ceph-mon[34812]: pgmap v383: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:02.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:02 smithi143 ceph-mon[35545]: pgmap v383: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:04.535 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:04 smithi042 ceph-mon[31661]: pgmap v384: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:04.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 -- 172.21.15.42:0/3497377474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106ec0 msgr2=0x7fa454113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 --2- 172.21.15.42:0/3497377474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106ec0 0x7fa454113350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00b0a0 tx=0x7fa44c00b3b0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 -- 172.21.15.42:0/3497377474 shutdown_connections 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 --2- 172.21.15.42:0/3497377474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106ec0 0x7fa454113350 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 --2- 172.21.15.42:0/3497377474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45410c5b0 0x7fa45410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 --2- 172.21.15.42:0/3497377474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa454106570 0x7fa454106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 -- 172.21.15.42:0/3497377474 >> 172.21.15.42:0/3497377474 conn(0x7fa454102070 msgr2=0x7fa4541044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:31:04.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.665+0000 7fa45b400700 1 -- 172.21.15.42:0/3497377474 shutdown_connections 2024-05-27T10:31:04.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.666+0000 7fa45b400700 1 -- 172.21.15.42:0/3497377474 wait complete. 2024-05-27T10:31:04.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.667+0000 7fa45b400700 1 Processor -- start 2024-05-27T10:31:04.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.667+0000 7fa45b400700 1 -- start start 2024-05-27T10:31:04.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa45b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 0x7fa45419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:04.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa45b400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa454106ec0 0x7fa45419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:04.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa45b400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45410c5b0 0x7fa4541a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:04.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa45b400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4541165e0 con 0x7fa454106570 2024-05-27T10:31:04.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa45b400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa454116460 con 0x7fa45410c5b0 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa45b400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa454116760 con 0x7fa454106ec0 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa458a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 0x7fa45419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa458a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 0x7fa45419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41056/0 (socket says 172.21.15.42:41056) 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa458a00700 1 -- 172.21.15.42:0/2269890561 learned_addr learned my addr 172.21.15.42:0/2269890561 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa453e00700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa454106ec0 0x7fa45419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.668+0000 7fa459400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45410c5b0 0x7fa4541a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:04.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.669+0000 7fa458a00700 1 -- 172.21.15.42:0/2269890561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa454106ec0 msgr2=0x7fa45419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.669+0000 7fa458a00700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa454106ec0 0x7fa45419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.669+0000 7fa458a00700 1 -- 172.21.15.42:0/2269890561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45410c5b0 msgr2=0x7fa4541a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.669+0000 7fa458a00700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45410c5b0 0x7fa4541a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.669+0000 7fa458a00700 1 -- 172.21.15.42:0/2269890561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4541a1e00 con 0x7fa454106570 2024-05-27T10:31:04.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.669+0000 7fa458a00700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 0x7fa45419cf30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa44800d980 tx=0x7fa44800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:31:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.670+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448017070 con 0x7fa454106570 2024-05-27T10:31:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.670+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa44800fd20 con 0x7fa454106570 2024-05-27T10:31:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.670+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44800e3b0 con 0x7fa454106570 2024-05-27T10:31:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.670+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4541a2090 con 0x7fa454106570 2024-05-27T10:31:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.670+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541a2580 con 0x7fa454106570 2024-05-27T10:31:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.671+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4540512b0 con 0x7fa454106570 2024-05-27T10:31:04.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.674+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa44800f890 con 0x7fa454106570 2024-05-27T10:31:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.677+0000 7fa451600700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa43c0b2a70 0x7fa43c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.677+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fa4480e2070 con 0x7fa454106570 2024-05-27T10:31:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.677+0000 7fa453e00700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa43c0b2a70 0x7fa43c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.678+0000 7fa453e00700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa43c0b2a70 0x7fa43c0b4f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa454103c50 tx=0x7fa44000b410 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:31:04.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.682+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4480a9a10 con 0x7fa454106570 2024-05-27T10:31:04.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:04 smithi104 ceph-mon[34812]: pgmap v384: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:04.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:04 smithi143 ceph-mon[35545]: pgmap v384: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T10:31:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.936+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa45402e5e0 con 0x7fa454106570 2024-05-27T10:31:04.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.938+0000 7fa451600700 1 -- 172.21.15.42:0/2269890561 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v93) ==== 81+0+945 (secure 0 0 0) 0x7fa4480a9310 con 0x7fa454106570 2024-05-27T10:31:04.936 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1953650070 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:04.936 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2062230330 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4102007980 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2381100937 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2062230330 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3098170734 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1584316049 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3950108791 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2667703980 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3950108791 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2970531755 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:04.937 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1310078254 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:04.938 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/274704256 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:04.938 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4159042453 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:04.938 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2156747670 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:04.938 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3098170734 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:04.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa43c0b2a70 msgr2=0x7fa43c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:04.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa43c0b2a70 0x7fa43c0b4f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa454103c50 tx=0x7fa44000b410 comp rx=0 tx=0).stop 2024-05-27T10:31:04.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 msgr2=0x7fa45419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 0x7fa45419cf30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa44800d980 tx=0x7fa44800dc90 comp rx=0 tx=0).stop 2024-05-27T10:31:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 shutdown_connections 2024-05-27T10:31:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7fa43c0b2a70 0x7fa43c0b4f80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45410c5b0 0x7fa4541a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa454106570 0x7fa45419cf30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.941+0000 7fa45b400700 1 --2- 172.21.15.42:0/2269890561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa454106ec0 0x7fa45419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.942+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 >> 172.21.15.42:0/2269890561 conn(0x7fa454102070 msgr2=0x7fa4541047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:31:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.942+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 shutdown_connections 2024-05-27T10:31:04.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:04.942+0000 7fa45b400700 1 -- 172.21.15.42:0/2269890561 wait complete. 2024-05-27T10:31:04.942 INFO:teuthology.orchestra.run.smithi042.stderr:listed 16 entries 2024-05-27T10:31:05.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:05 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/2269890561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T10:31:05.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:05 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/2269890561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T10:31:05.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:05 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/2269890561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T10:31:06.331 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-27T10:31:06.331 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T10:31:06.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d shell --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T10:31:06.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:06 smithi042 ceph-mon[31661]: pgmap v385: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-27T10:31:06.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:06 smithi104 ceph-mon[34812]: pgmap v385: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-27T10:31:06.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:06 smithi143 ceph-mon[35545]: pgmap v385: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-27T10:31:06.862 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T10:31:08.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:08 smithi042 ceph-mon[31661]: pgmap v386: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-27T10:31:08.609 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:08 smithi143 ceph-mon[35545]: pgmap v386: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-27T10:31:08.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:08 smithi104 ceph-mon[34812]: pgmap v386: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-27T10:31:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 -- 172.21.15.42:0/1131820261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078077550 msgr2=0x7f5078110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 --2- 172.21.15.42:0/1131820261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078077550 0x7f5078110690 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5060009990 tx=0x7f5060009ca0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 -- 172.21.15.42:0/1131820261 shutdown_connections 2024-05-27T10:31:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 --2- 172.21.15.42:0/1131820261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5078110bd0 0x7f5078112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 --2- 172.21.15.42:0/1131820261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5078076c00 0x7f5078077010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 --2- 172.21.15.42:0/1131820261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078077550 0x7f5078110690 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.711+0000 7f507e400700 1 -- 172.21.15.42:0/1131820261 >> 172.21.15.42:0/1131820261 conn(0x7f50780ffe90 msgr2=0x7f50781022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:31:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.712+0000 7f507e400700 1 -- 172.21.15.42:0/1131820261 shutdown_connections 2024-05-27T10:31:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.712+0000 7f507e400700 1 -- 172.21.15.42:0/1131820261 wait complete. 2024-05-27T10:31:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.713+0000 7f507e400700 1 Processor -- start 2024-05-27T10:31:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.713+0000 7f507e400700 1 -- start start 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f507e400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5078076c00 0x7f507819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f507e400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5078077550 0x7f507819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f507e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 0x7f50781a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f507e400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5078116540 con 0x7f5078110bd0 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f507e400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50781163c0 con 0x7f5078077550 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f507e400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f50781166c0 con 0x7f5078076c00 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f5077e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 0x7f50781a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f5077e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 0x7f50781a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41072/0 (socket says 172.21.15.42:41072) 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f5077e00700 1 -- 172.21.15.42:0/1531060574 learned_addr learned my addr 172.21.15.42:0/1531060574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.714+0000 7f5077400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5078076c00 0x7f507819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f506ea00700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5078077550 0x7f507819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f5077e00700 1 -- 172.21.15.42:0/1531060574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5078076c00 msgr2=0x7f507819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f5077e00700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5078076c00 0x7f507819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f5077e00700 1 -- 172.21.15.42:0/1531060574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5078077550 msgr2=0x7f507819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f5077e00700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5078077550 0x7f507819d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f5077e00700 1 -- 172.21.15.42:0/1531060574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781a1e40 con 0x7f5078110bd0 2024-05-27T10:31:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.715+0000 7f5077e00700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 0x7f50781a17a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f507000d950 tx=0x7f507000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:31:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.716+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070017070 con 0x7f5078110bd0 2024-05-27T10:31:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.716+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781a2130 con 0x7f5078110bd0 2024-05-27T10:31:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.716+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5070005020 con 0x7f5078110bd0 2024-05-27T10:31:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.716+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781a9a50 con 0x7f5078110bd0 2024-05-27T10:31:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.716+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070020060 con 0x7f5078110bd0 2024-05-27T10:31:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.717+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50780512b0 con 0x7f5078110bd0 2024-05-27T10:31:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.719+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5070004a30 con 0x7f5078110bd0 2024-05-27T10:31:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.721+0000 7f5074c00700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f505c0b29f0 0x7f505c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T10:31:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.721+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f50700e2420 con 0x7f5078110bd0 2024-05-27T10:31:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.722+0000 7f5077400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f505c0b29f0 0x7f505c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T10:31:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.722+0000 7f5077400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f505c0b29f0 0x7f505c0b4f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5068009550 tx=0x7f50680091b0 comp rx=0 tx=0).ready entity=mgr.34892 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T10:31:09.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.724+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50700051d0 con 0x7f5078110bd0 2024-05-27T10:31:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.982+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f507802e5e0 con 0x7f5078110bd0 2024-05-27T10:31:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.983+0000 7f5074c00700 1 -- 172.21.15.42:0/1531060574 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v93) ==== 81+0+945 (secure 0 0 0) 0x7f50700a9dc0 con 0x7f5078110bd0 2024-05-27T10:31:09.982 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1953650070 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:09.982 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2062230330 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:09.982 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4102007980 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2381100937 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2062230330 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3098170734 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1584316049 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3950108791 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2667703980 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3950108791 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2970531755 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1310078254 2024-05-28T10:18:42.838173+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/274704256 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4159042453 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2156747670 2024-05-28T10:03:03.499453+0000 2024-05-27T10:31:09.983 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3098170734 2024-05-28T10:03:40.179244+0000 2024-05-27T10:31:09.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.986+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f505c0b29f0 msgr2=0x7f505c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:09.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.986+0000 7f507e400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f505c0b29f0 0x7f505c0b4f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5068009550 tx=0x7f50680091b0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.986+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 msgr2=0x7f50781a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.986+0000 7f507e400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 0x7f50781a17a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f507000d950 tx=0x7f507000dc60 comp rx=0 tx=0).stop 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 shutdown_connections 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:6800/3616349127,v1:172.21.15.42:6801/3616349127] conn(0x7f505c0b29f0 0x7f505c0b4f00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5078110bd0 0x7f50781a17a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5078076c00 0x7f507819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 --2- 172.21.15.42:0/1531060574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5078077550 0x7f507819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T10:31:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 >> 172.21.15.42:0/1531060574 conn(0x7f50780ffe90 msgr2=0x7f5078100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T10:31:09.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 shutdown_connections 2024-05-27T10:31:09.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-27T10:31:09.987+0000 7f507e400700 1 -- 172.21.15.42:0/1531060574 wait complete. 2024-05-27T10:31:09.987 INFO:teuthology.orchestra.run.smithi042.stderr:listed 16 entries 2024-05-27T10:31:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:10 smithi104 ceph-mon[34812]: pgmap v387: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-27T10:31:10.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:10 smithi104 ceph-mon[34812]: from='client.? 172.21.15.42:0/1531060574' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T10:31:10.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:10 smithi143 ceph-mon[35545]: pgmap v387: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-27T10:31:10.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:10 smithi143 ceph-mon[35545]: from='client.? 172.21.15.42:0/1531060574' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T10:31:10.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:10 smithi042 ceph-mon[31661]: pgmap v387: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-27T10:31:10.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:10 smithi042 ceph-mon[31661]: from='client.? 172.21.15.42:0/1531060574' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T10:31:11.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-27T10:31:11.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:31:11.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-27T10:31:11.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[88898]: fuse finished with error 0 and tester_r 0 2024-05-27T10:31:11.323 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T10:31:11.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:11 smithi042 ceph-mon[31661]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T10:31:11.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:11 smithi104 ceph-mon[34812]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T10:31:11.742 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:11 smithi143 ceph-mon[35545]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T10:31:12.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:12 smithi104 ceph-mon[34812]: pgmap v389: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T10:31:12.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:12 smithi143 ceph-mon[35545]: pgmap v389: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T10:31:12.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:12 smithi042 ceph-mon[31661]: pgmap v389: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T10:31:14.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:14 smithi042 ceph-mon[31661]: pgmap v390: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T10:31:14.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:14 smithi104 ceph-mon[34812]: pgmap v390: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T10:31:14.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:14 smithi143 ceph-mon[35545]: pgmap v390: 97 pgs: 97 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-27T10:31:17.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:16 smithi143 ceph-mon[35545]: pgmap v391: 97 pgs: 97 active+clean; 417 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-27T10:31:17.306 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:16 smithi143 ceph-mon[35545]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T10:31:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:16 smithi042 ceph-mon[31661]: pgmap v391: 97 pgs: 97 active+clean; 417 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-27T10:31:17.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:16 smithi042 ceph-mon[31661]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T10:31:17.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:16 smithi104 ceph-mon[34812]: pgmap v391: 97 pgs: 97 active+clean; 417 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-27T10:31:17.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:16 smithi104 ceph-mon[34812]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T10:31:17.425 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:31:17.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:31:17.425 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T10:31:17.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:31:17.489 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-27T10:31:17.547 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T10:31:17.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:31:17.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T10:31:17.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T10:31:17.646 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-05-27T10:31:17.762 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-27T10:31:17.774 INFO:tasks.cephadm:Teardown begin 2024-05-27T10:31:17.774 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-27T10:31:17.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:31:17.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:31:17.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:31:18.047 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-27T10:31:18.047 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T10:31:18.105 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T10:31:18.160 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T10:31:18.217 INFO:tasks.cephadm:Stopping all daemons... 2024-05-27T10:31:18.218 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-27T10:31:18.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a 2024-05-27T10:31:18.592 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:18 smithi042 systemd[1]: Stopping Ceph mon.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:19.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31657]: 2024-05-27T10:31:18.821+0000 7fd912c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:19.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a[31657]: 2024-05-27T10:31:18.821+0000 7fd912c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-27T10:31:19.092 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:18 smithi042 podman[159222]: 2024-05-27 10:31:18.996194563 +0000 UTC m=+0.374531858 container died c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T10:31:19.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:19 smithi042 podman[159222]: 2024-05-27 10:31:19.74783258 +0000 UTC m=+1.126169886 container cleanup c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:31:19.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:19 smithi042 bash[159222]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a 2024-05-27T10:31:20.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:May 27 10:31:20 smithi042 podman[159236]: 2024-05-27 10:31:20.008230756 +0000 UTC m=+0.995959162 container remove c143ba0d02a3b51d33580c4d4b6ba7295da4774f64f15fe2bc33c1ded30d7747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T10:31:20.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.a.service' 2024-05-27T10:31:20.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:20.758 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-27T10:31:20.758 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-27T10:31:20.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.b 2024-05-27T10:31:21.196 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:20 smithi104 systemd[1]: Stopping Ceph mon.b for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:21.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b[34808]: 2024-05-27T10:31:21.370+0000 7f4a9aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:21.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b[34808]: 2024-05-27T10:31:21.370+0000 7f4a9aa00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-27T10:31:21.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:21 smithi104 podman[71802]: 2024-05-27 10:31:21.658592196 +0000 UTC m=+0.417944237 container died 407d38b71fe497a029e9fbc818697d9f7cf528aeb3e45981f24aa80273551ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD) 2024-05-27T10:31:22.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:22 smithi104 podman[71802]: 2024-05-27 10:31:22.130155282 +0000 UTC m=+0.889507357 container cleanup 407d38b71fe497a029e9fbc818697d9f7cf528aeb3e45981f24aa80273551ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:31:22.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:22 smithi104 bash[71802]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b 2024-05-27T10:31:22.699 INFO:journalctl@ceph.mon.b.smithi104.stdout:May 27 10:31:22 smithi104 podman[71813]: 2024-05-27 10:31:22.456376659 +0000 UTC m=+0.785271150 container remove 407d38b71fe497a029e9fbc818697d9f7cf528aeb3e45981f24aa80273551ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-27T10:31:22.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.b.service' 2024-05-27T10:31:22.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:22.896 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-27T10:31:22.896 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-27T10:31:22.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.c 2024-05-27T10:31:23.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:23 smithi143 systemd[1]: Stopping Ceph mon.c for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c[35530]: 2024-05-27T10:31:23.427+0000 7fdde5a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c[35530]: 2024-05-27T10:31:23.427+0000 7fdde5a00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-27T10:31:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:23 smithi143 podman[73376]: 2024-05-27 10:31:23.491124264 +0000 UTC m=+0.217260108 container died 945848b5fdc3f069414537a893b4ed078933c990444d720c4b7fac5ef8edb2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T10:31:24.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:23 smithi143 podman[73376]: 2024-05-27 10:31:23.949405901 +0000 UTC m=+0.675541755 container cleanup 945848b5fdc3f069414537a893b4ed078933c990444d720c4b7fac5ef8edb2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T10:31:24.056 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:23 smithi143 bash[73376]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c 2024-05-27T10:31:24.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 27 10:31:24 smithi143 podman[73390]: 2024-05-27 10:31:24.207388795 +0000 UTC m=+0.701402279 container remove 945848b5fdc3f069414537a893b4ed078933c990444d720c4b7fac5ef8edb2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mon-c, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T10:31:24.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mon.c.service' 2024-05-27T10:31:24.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:24.825 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-27T10:31:24.825 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-27T10:31:24.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T10:31:25.342 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:31:24 smithi042 systemd[1]: Stopping Ceph mgr.x for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:25.842 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:31:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x[31889]: 2024-05-27T10:31:25.460+0000 7f95d9200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:25.842 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:31:25 smithi042 podman[159343]: 2024-05-27 10:31:25.56716877 +0000 UTC m=+0.285444319 container died f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T10:31:26.655 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:31:26 smithi042 podman[159343]: 2024-05-27 10:31:26.413758981 +0000 UTC m=+1.132034473 container cleanup f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:31:26.655 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:31:26 smithi042 bash[159343]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x 2024-05-27T10:31:27.092 INFO:journalctl@ceph.mgr.x.smithi042.stdout:May 27 10:31:26 smithi042 podman[159357]: 2024-05-27 10:31:26.654389623 +0000 UTC m=+1.069704156 container remove f8155dda74fd0405c35ccf41a54d4dbb0d456c7fdea02219ce4eeef2da492150 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True) 2024-05-27T10:31:27.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.x.service' 2024-05-27T10:31:27.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:27.411 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-27T10:31:27.411 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-27T10:31:27.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.y 2024-05-27T10:31:27.946 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:31:27 smithi104 systemd[1]: Stopping Ceph mgr.y for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:28.388 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:31:28 smithi104 podman[71922]: 2024-05-27 10:31:28.117831246 +0000 UTC m=+0.266504833 container died 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-27T10:31:28.697 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:31:28 smithi104 podman[71922]: 2024-05-27 10:31:28.585153519 +0000 UTC m=+0.733827109 container cleanup 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T10:31:28.697 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:31:28 smithi104 bash[71922]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y 2024-05-27T10:31:29.196 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:31:28 smithi104 podman[71935]: 2024-05-27 10:31:28.828071876 +0000 UTC m=+0.690510958 container remove 59bbe45dca089a45655edace4a5af53fb7c2c1259bc50571907d4e8c1ef5d956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:31:29.196 INFO:journalctl@ceph.mgr.y.smithi104.stdout:May 27 10:31:28 smithi104 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-27T10:31:29.284 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.y.service' 2024-05-27T10:31:29.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:29.393 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-27T10:31:29.394 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-27T10:31:29.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.z 2024-05-27T10:31:29.805 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:31:29 smithi143 systemd[1]: Stopping Ceph mgr.z for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:30.414 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:31:30 smithi143 podman[73497]: 2024-05-27 10:31:30.121107344 +0000 UTC m=+0.268610138 container died aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T10:31:30.806 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:31:30 smithi143 podman[73497]: 2024-05-27 10:31:30.603623282 +0000 UTC m=+0.751126113 container cleanup aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-27T10:31:30.806 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:31:30 smithi143 bash[73497]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z 2024-05-27T10:31:31.277 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:31:30 smithi143 podman[73510]: 2024-05-27 10:31:30.855169621 +0000 UTC m=+0.716367980 container remove aa8a932cc3b7b66c9623acd0e1adb9bf114c4205e9438c2fbcebbce1f1dc8d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-27T10:31:31.277 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 27 10:31:30 smithi143 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-27T10:31:31.320 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mgr.z.service' 2024-05-27T10:31:31.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:31.434 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-27T10:31:31.434 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-27T10:31:31.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.0 2024-05-27T10:31:31.841 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:31 smithi042 systemd[1]: Stopping Ceph osd.0 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:32.592 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0[46409]: 2024-05-27T10:31:32.093+0000 7fe8c2400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:32.592 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0[46409]: 2024-05-27T10:31:32.093+0000 7fe8c2400700 -1 osd.0 95 *** Got signal Terminated *** 2024-05-27T10:31:32.593 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0[46409]: 2024-05-27T10:31:32.093+0000 7fe8c2400700 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:31:38.092 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:37 smithi042 podman[159476]: 2024-05-27 10:31:37.745824343 +0000 UTC m=+5.853746549 container died 30a2e5345c6ad5a905a7c14e30468a2fa644a82fccbe7f8f81431b1ddd617c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T10:31:38.842 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:38 smithi042 podman[159476]: 2024-05-27 10:31:38.420571209 +0000 UTC m=+6.528493368 container cleanup 30a2e5345c6ad5a905a7c14e30468a2fa644a82fccbe7f8f81431b1ddd617c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-27T10:31:38.843 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:38 smithi042 bash[159476]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0 2024-05-27T10:31:38.843 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:38 smithi042 podman[159499]: 2024-05-27 10:31:38.64731664 +0000 UTC m=+0.880978194 container remove 30a2e5345c6ad5a905a7c14e30468a2fa644a82fccbe7f8f81431b1ddd617c13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:31:39.737 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:39 smithi042 podman[159588]: 2024-05-27 10:31:39.393425039 +0000 UTC m=+0.036689842 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:31:39.737 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:39 smithi042 podman[159588]: 2024-05-27 10:31:39.556918163 +0000 UTC m=+0.200182995 container create e1ac7ff6fa181f19922e31be5f4c7cecb38b8eb6c9c8e17078628cde4e6611ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:31:40.008 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:39 smithi042 podman[159588]: 2024-05-27 10:31:39.894810814 +0000 UTC m=+0.538075696 container init e1ac7ff6fa181f19922e31be5f4c7cecb38b8eb6c9c8e17078628cde4e6611ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T10:31:40.008 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:39 smithi042 podman[159588]: 2024-05-27 10:31:39.906185449 +0000 UTC m=+0.549450237 container start e1ac7ff6fa181f19922e31be5f4c7cecb38b8eb6c9c8e17078628cde4e6611ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:31:40.342 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:40 smithi042 podman[159588]: 2024-05-27 10:31:40.009908734 +0000 UTC m=+0.653173613 container attach e1ac7ff6fa181f19922e31be5f4c7cecb38b8eb6c9c8e17078628cde4e6611ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T10:31:40.715 INFO:journalctl@ceph.osd.0.smithi042.stdout:May 27 10:31:40 smithi042 podman[159623]: 2024-05-27 10:31:40.425083658 +0000 UTC m=+0.033575096 container died e1ac7ff6fa181f19922e31be5f4c7cecb38b8eb6c9c8e17078628cde4e6611ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T10:31:41.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-27T10:31:41.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:41.466 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-27T10:31:41.466 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-27T10:31:41.466 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.1 2024-05-27T10:31:41.946 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:41 smithi104 systemd[1]: Stopping Ceph osd.1 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:42.446 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1[39369]: 2024-05-27T10:31:42.045+0000 7fb526400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:42.446 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1[39369]: 2024-05-27T10:31:42.045+0000 7fb526400700 -1 osd.1 95 *** Got signal Terminated *** 2024-05-27T10:31:42.447 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1[39369]: 2024-05-27T10:31:42.045+0000 7fb526400700 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:31:47.854 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:47 smithi104 podman[72045]: 2024-05-27 10:31:47.569452196 +0000 UTC m=+5.688964317 container died 9a1fb83c415dea0ffb169ddecadd760a99c67749c8227817a0d9573963300e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-27T10:31:48.196 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:48 smithi104 podman[72045]: 2024-05-27 10:31:48.021214793 +0000 UTC m=+6.140726908 container cleanup 9a1fb83c415dea0ffb169ddecadd760a99c67749c8227817a0d9573963300e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T10:31:48.196 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:48 smithi104 bash[72045]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1 2024-05-27T10:31:48.696 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:48 smithi104 podman[72064]: 2024-05-27 10:31:48.297801641 +0000 UTC m=+0.712638225 container remove 9a1fb83c415dea0ffb169ddecadd760a99c67749c8227817a0d9573963300e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux ) 2024-05-27T10:31:49.430 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:49 smithi104 podman[72151]: 2024-05-27 10:31:49.080535933 +0000 UTC m=+0.038302047 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:31:49.430 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:49 smithi104 podman[72151]: 2024-05-27 10:31:49.223749456 +0000 UTC m=+0.181515592 container create b777fc08f4fb0ea27ebae25d6245ff671591998cbb97291f788570c044b9fc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T10:31:49.696 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:49 smithi104 podman[72151]: 2024-05-27 10:31:49.560887726 +0000 UTC m=+0.518653872 container init b777fc08f4fb0ea27ebae25d6245ff671591998cbb97291f788570c044b9fc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T10:31:49.696 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:49 smithi104 podman[72151]: 2024-05-27 10:31:49.571830759 +0000 UTC m=+0.529596915 container start b777fc08f4fb0ea27ebae25d6245ff671591998cbb97291f788570c044b9fc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:31:49.697 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:49 smithi104 podman[72151]: 2024-05-27 10:31:49.668483555 +0000 UTC m=+0.626249715 container attach b777fc08f4fb0ea27ebae25d6245ff671591998cbb97291f788570c044b9fc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:31:50.421 INFO:journalctl@ceph.osd.1.smithi104.stdout:May 27 10:31:50 smithi104 podman[72151]: 2024-05-27 10:31:50.035799626 +0000 UTC m=+0.993565775 container died b777fc08f4fb0ea27ebae25d6245ff671591998cbb97291f788570c044b9fc39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:31:50.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-27T10:31:50.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:31:50.976 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-27T10:31:50.976 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-27T10:31:50.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.2 2024-05-27T10:31:51.306 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:51 smithi143 systemd[1]: Stopping Ceph osd.2 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:31:52.056 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2[40751]: 2024-05-27T10:31:51.620+0000 7f84fe200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:31:52.056 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2[40751]: 2024-05-27T10:31:51.620+0000 7f84fe200700 -1 osd.2 95 *** Got signal Terminated *** 2024-05-27T10:31:52.056 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2[40751]: 2024-05-27T10:31:51.620+0000 7f84fe200700 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:31:57.559 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:57 smithi143 podman[73619]: 2024-05-27 10:31:57.289078086 +0000 UTC m=+5.815428282 container died e58a9248a36d005730e50b07055cc2b9a48d53cd91dc16e146506074e15976f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-27T10:31:57.992 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:57 smithi143 podman[73619]: 2024-05-27 10:31:57.775255537 +0000 UTC m=+6.301605680 container cleanup e58a9248a36d005730e50b07055cc2b9a48d53cd91dc16e146506074e15976f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-27T10:31:57.992 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:57 smithi143 bash[73619]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2 2024-05-27T10:31:58.243 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:57 smithi143 podman[73636]: 2024-05-27 10:31:57.993265736 +0000 UTC m=+0.691289871 container remove e58a9248a36d005730e50b07055cc2b9a48d53cd91dc16e146506074e15976f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-27T10:31:58.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:31:58 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:31:58.244+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:31:59.115 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:58 smithi143 podman[73725]: 2024-05-27 10:31:58.714508582 +0000 UTC m=+0.032237479 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:31:59.116 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:58 smithi143 podman[73725]: 2024-05-27 10:31:58.828285728 +0000 UTC m=+0.146014665 container create 302e9ee32e147e6fb73909fc44a7a581658ae6928dedd729f839b21fd586b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-27T10:31:59.373 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:59 smithi143 podman[73725]: 2024-05-27 10:31:59.123652919 +0000 UTC m=+0.441381896 container init 302e9ee32e147e6fb73909fc44a7a581658ae6928dedd729f839b21fd586b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T10:31:59.373 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:59 smithi143 podman[73725]: 2024-05-27 10:31:59.134455134 +0000 UTC m=+0.452184040 container start 302e9ee32e147e6fb73909fc44a7a581658ae6928dedd729f839b21fd586b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T10:31:59.373 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:59 smithi143 podman[73725]: 2024-05-27 10:31:59.206059849 +0000 UTC m=+0.523788849 container attach 302e9ee32e147e6fb73909fc44a7a581658ae6928dedd729f839b21fd586b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-27T10:31:59.373 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:31:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:31:59.205+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:31:59.373 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:31:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:31:59.114+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:31:59.674 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 27 10:31:59 smithi143 podman[73725]: 2024-05-27 10:31:59.597128354 +0000 UTC m=+0.914857296 container died 302e9ee32e147e6fb73909fc44a7a581658ae6928dedd729f839b21fd586b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:32:00.215 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:00.153+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:00.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-27T10:32:00.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:00.418 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-27T10:32:00.422 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-27T10:32:00.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.3 2024-05-27T10:32:00.555 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:00.215+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:00.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:00.268+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:00.842 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:00 smithi042 systemd[1]: Stopping Ceph osd.3 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:01.328 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:01.025+0000 7fafae200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:01.328 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:01.025+0000 7fafae200700 -1 osd.3 95 *** Got signal Terminated *** 2024-05-27T10:32:01.328 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:01.025+0000 7fafae200700 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:01.329 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:01.328+0000 7fafa9a00700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:35.019876+0000 front 2024-05-27T10:31:35.019807+0000 (oldest deadline 2024-05-27T10:32:00.920023+0000) 2024-05-27T10:32:01.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:01.313+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:01.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:01.149+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:01.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:01.185+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:02.555 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:02.161+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:02.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:02.154+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:02.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:02.358+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:02.582 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:02.312+0000 7fafa9a00700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:35.019876+0000 front 2024-05-27T10:31:35.019807+0000 (oldest deadline 2024-05-27T10:32:00.920023+0000) 2024-05-27T10:32:02.582 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:02.428+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:03.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:03 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:03.121+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:03.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:03 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:03.135+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:03.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:03 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:03.324+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:03.842 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:03.356+0000 7fafa9a00700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:35.019876+0000 front 2024-05-27T10:31:35.019807+0000 (oldest deadline 2024-05-27T10:32:00.920023+0000) 2024-05-27T10:32:03.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:03.398+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:03.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:03.586+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:03.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:03 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:03.501+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:04.555 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:04.338+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:04.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:04.100+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:04.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:04.118+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:04.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:04 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:04.389+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:04.842 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:04 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:04.397+0000 7fafa9a00700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:35.019876+0000 front 2024-05-27T10:31:35.019807+0000 (oldest deadline 2024-05-27T10:32:00.920023+0000) 2024-05-27T10:32:04.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:04 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:04.590+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:04.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:04.455+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:05.555 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:05.153+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:05.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:05.126+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:05.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:05.329+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:05.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:05 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:05.424+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:05.842 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:05 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3[53351]: 2024-05-27T10:32:05.423+0000 7fafa9a00700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:35.019876+0000 front 2024-05-27T10:31:35.019807+0000 (oldest deadline 2024-05-27T10:32:00.920023+0000) 2024-05-27T10:32:05.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:05 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:05.565+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:05.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:05.489+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:06.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:06.168+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:06.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:06.105+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:06.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:06.321+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:06.842 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:06 smithi042 podman[159702]: 2024-05-27 10:32:06.511016265 +0000 UTC m=+5.631264813 container died 2f951c3640de051bbd283368ee5d6cec3d19d392a725a4545aa4d7bb6bc1ccb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T10:32:06.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:06 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:06.447+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:06.843 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:06 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:06.528+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:06.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:06.475+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:07.327 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:07 smithi042 podman[159702]: 2024-05-27 10:32:07.327806988 +0000 UTC m=+6.448055567 container cleanup 2f951c3640de051bbd283368ee5d6cec3d19d392a725a4545aa4d7bb6bc1ccb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:32:07.555 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:07.132+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:07.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:07.150+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:07.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:07.318+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:07.592 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:07 smithi042 bash[159702]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3 2024-05-27T10:32:07.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:07 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:07.490+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:07.592 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:07 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:07.480+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:07.820 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:07.456+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:08.092 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:07 smithi042 podman[159730]: 2024-05-27 10:32:07.604829257 +0000 UTC m=+1.079526528 container remove 2f951c3640de051bbd283368ee5d6cec3d19d392a725a4545aa4d7bb6bc1ccb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:08.197 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:07.821+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:08.555 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:08.332+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:08.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:08.155+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:08.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:08.111+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:08.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:08.111+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:08.785 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:08.463+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:08.785 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:08.463+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:08.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:08 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:08.456+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:08.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:08 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:08.456+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:08.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:08 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:08.489+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:08.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:08 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:08.489+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:08.843 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:08 smithi042 podman[159816]: 2024-05-27 10:32:08.380217584 +0000 UTC m=+0.037231494 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:32:08.843 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:08 smithi042 podman[159816]: 2024-05-27 10:32:08.505795576 +0000 UTC m=+0.162809543 container create 200d76be0dc0d6718be4ad3d11b1f2f7806ec14d43e5a03408ac385d2642949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T10:32:09.174 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:09 smithi042 podman[159816]: 2024-05-27 10:32:09.050605368 +0000 UTC m=+0.707619355 container init 200d76be0dc0d6718be4ad3d11b1f2f7806ec14d43e5a03408ac385d2642949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T10:32:09.174 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:09 smithi042 podman[159816]: 2024-05-27 10:32:09.061830387 +0000 UTC m=+0.718844329 container start 200d76be0dc0d6718be4ad3d11b1f2f7806ec14d43e5a03408ac385d2642949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T10:32:09.174 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:09 smithi042 podman[159816]: 2024-05-27 10:32:09.175917956 +0000 UTC m=+0.832931864 container attach 200d76be0dc0d6718be4ad3d11b1f2f7806ec14d43e5a03408ac385d2642949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:09.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:08.785+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:09.430 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:09.430+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:09.430 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:09.430+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:09.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:09.084+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:09.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:09.084+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:09.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:09.180+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:09.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:09.180+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:09.557 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:09.297+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:09.557 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:09.297+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:09.828 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:09.462+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:09.828 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:09.462+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:09.842 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:09 smithi042 podman[159842]: 2024-05-27 10:32:09.60226198 +0000 UTC m=+0.034084943 container died 200d76be0dc0d6718be4ad3d11b1f2f7806ec14d43e5a03408ac385d2642949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, ceph=True) 2024-05-27T10:32:09.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:09.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:09.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:09 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:09.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:10.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:09.828+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:10.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:10.037+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:10.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:10.037+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:10.306 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:10.259+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:10.306 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:10.259+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:10.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:10.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:10.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:10.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:10.596 INFO:journalctl@ceph.osd.3.smithi042.stdout:May 27 10:32:10 smithi042 podman[159842]: 2024-05-27 10:32:10.566310539 +0000 UTC m=+0.998133546 container remove 200d76be0dc0d6718be4ad3d11b1f2f7806ec14d43e5a03408ac385d2642949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521) 2024-05-27T10:32:10.596 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:10 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:10.475+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:10.596 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:10 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:10.475+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:10.596 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:10 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:10.401+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:10.596 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:10 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:10.401+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:10.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-27T10:32:10.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:10.737 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-27T10:32:10.738 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-27T10:32:10.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.4 2024-05-27T10:32:10.815 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:10.483+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:10.815 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:10.483+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:11.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:10.876+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:11.196 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:10 smithi104 systemd[1]: Stopping Ceph osd.4 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:11.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:11.247+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:11.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:11.247+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:11.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:11.088+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:11.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:11.088+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:11.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:11.210+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:11.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:11.210+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:11.698 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:11.524+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:11.698 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:11.524+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:11.698 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:11.354+0000 7f8aeb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:11.699 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:11.354+0000 7f8aeb000700 -1 osd.4 95 *** Got signal Terminated *** 2024-05-27T10:32:11.699 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:11.354+0000 7f8aeb000700 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:11.757 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:11 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:11.513+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:11.758 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:11 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:11.513+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:11.758 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:11 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:11.428+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:11.758 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:11 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:11.428+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:12.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:11.900+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:12.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:12.050+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:12.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:12.050+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:12.196+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:12.196+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:12.307 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:12.262+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:12.307 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:12.262+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:12.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:12 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:12.449+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:12.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:12 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:12.449+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:12.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:12 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:12.536+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:12.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:12 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:12.536+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:12.924 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:12.633+0000 7f8ae1800700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:47.021451+0000 front 2024-05-27T10:31:47.021528+0000 (oldest deadline 2024-05-27T10:32:11.721691+0000) 2024-05-27T10:32:12.925 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:12.539+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:12.925 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:12.539+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:13.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:12.925+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:13.555 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:13.088+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:13.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:13.088+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:13.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:13.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:13.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:13.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:13.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:13.265+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:13.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:13.265+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:13.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:13 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:13.493+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:13.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:13 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:13.493+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:13.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:13 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:13.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:13.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:13 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:13.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:13.905 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:13.676+0000 7f8ae1800700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:47.021451+0000 front 2024-05-27T10:31:47.021528+0000 (oldest deadline 2024-05-27T10:32:11.721691+0000) 2024-05-27T10:32:13.905 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:13.538+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:13.905 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:13.538+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:14.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:13.905+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:14.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:14.290+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:14.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:14.290+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:14.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:14.128+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:14.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:14.128+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:14.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:14.167+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:14.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:14.167+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:14.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:14 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:14.523+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:14.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:14 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:14.523+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:14.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:14 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:14.533+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:14.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:14 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:14.533+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:14.888 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:14 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:14.562+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:14.888 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:14 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:14.562+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:14.888 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:14 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:14.722+0000 7f8ae1800700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:47.021451+0000 front 2024-05-27T10:31:47.021528+0000 (oldest deadline 2024-05-27T10:32:11.721691+0000) 2024-05-27T10:32:15.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:14 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:14.889+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:15.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:15.331+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:15.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:15.331+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:15.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:15.104+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:15.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:15.104+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:15.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:15.206+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:15.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:15.206+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:15.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:15 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:15.547+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:15.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:15 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:15.547+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:15.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:15 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:15.490+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:15.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:15 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:15.490+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:15.931 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:15 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4[46075]: 2024-05-27T10:32:15.752+0000 7f8ae1800700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:47.021451+0000 front 2024-05-27T10:31:47.021528+0000 (oldest deadline 2024-05-27T10:32:11.721691+0000) 2024-05-27T10:32:15.931 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:15 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:15.575+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:15.931 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:15 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:15.575+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:16.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:15 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:15.932+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:16.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:16 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:16.145+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:16.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:16 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:16.145+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:16.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:16 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:16.223+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:16.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:16 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:16.223+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:16.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:16 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:16.352+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:16.557 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:16 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:16.352+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:16.835 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:16 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:16.526+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:16.835 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:16 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:16.526+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:16.841 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:16 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:16.554+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:16.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:16 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:16.554+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:16.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:16 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:16.487+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:16.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:16 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:16.487+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:17.152 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:16 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:16.938+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:17.152 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:16 smithi104 podman[72254]: 2024-05-27 10:32:16.836733487 +0000 UTC m=+5.627727632 container died 46d9451537b0e0003bce1f5efe7613af5693c4ff0ca7dfe956493bb70405f18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:17.446 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:17 smithi104 podman[72254]: 2024-05-27 10:32:17.29037064 +0000 UTC m=+6.081364781 container cleanup 46d9451537b0e0003bce1f5efe7613af5693c4ff0ca7dfe956493bb70405f18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T10:32:17.446 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:17 smithi104 bash[72254]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4 2024-05-27T10:32:17.555 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:17.324+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:17.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:17.324+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:17.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:17.182+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:17.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:17.182+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:17.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:17.186+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:17.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:17.186+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:17.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:17 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:17.501+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:17.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:17 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:17.501+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:17.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:17 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:17.582+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:17.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:17 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:17.582+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:17.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:17 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:17.582+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:17.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:17 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:17.532+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:17.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:17 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:17.532+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:17.946 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:17 smithi104 podman[72271]: 2024-05-27 10:32:17.616724007 +0000 UTC m=+0.759126080 container remove 46d9451537b0e0003bce1f5efe7613af5693c4ff0ca7dfe956493bb70405f18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-27T10:32:18.376 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:17 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:17.949+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:18.555 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:18.330+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:18.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:18.330+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:18.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:18.188+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:18.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:18.188+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:18.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:18.147+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:18.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:18.147+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:18.697 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:18 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:18.572+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:18.697 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:18 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:18.572+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:18.697 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:18 smithi104 podman[72359]: 2024-05-27 10:32:18.282103495 +0000 UTC m=+0.032086002 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:32:18.697 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:18 smithi104 podman[72359]: 2024-05-27 10:32:18.427148724 +0000 UTC m=+0.177131267 container create cda050c79417f11281b66ad0aa47a7a70024cedfe6891f866324688adc2dcc35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T10:32:18.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:18.541+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:18.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:18.541+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:18.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:18.541+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:18.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:18.471+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:18.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:18 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:18.471+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:19.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:18 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:18.988+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:19.197 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:18 smithi104 podman[72359]: 2024-05-27 10:32:18.746464824 +0000 UTC m=+0.496447364 container init cda050c79417f11281b66ad0aa47a7a70024cedfe6891f866324688adc2dcc35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T10:32:19.197 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:18 smithi104 podman[72359]: 2024-05-27 10:32:18.757337875 +0000 UTC m=+0.507320386 container start cda050c79417f11281b66ad0aa47a7a70024cedfe6891f866324688adc2dcc35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:19.197 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:18 smithi104 podman[72359]: 2024-05-27 10:32:18.853348633 +0000 UTC m=+0.603331216 container attach cda050c79417f11281b66ad0aa47a7a70024cedfe6891f866324688adc2dcc35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:32:19.533 INFO:journalctl@ceph.osd.4.smithi104.stdout:May 27 10:32:19 smithi104 podman[72359]: 2024-05-27 10:32:19.217963485 +0000 UTC m=+0.967945991 container died cda050c79417f11281b66ad0aa47a7a70024cedfe6891f866324688adc2dcc35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T10:32:19.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:19.175+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:19.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:19.175+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:19.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:19.356+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:19.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:19.356+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:19.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:19.356+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:19.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:19.174+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:19.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:19.174+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:19.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:19 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:19.558+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:19.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:19 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:19.558+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:19.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:19 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:19.558+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:19.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:19 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:19.494+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:19.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:19 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:19.494+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:19.901 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:19 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:19.534+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:19.902 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:19 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:19.534+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:20.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-27T10:32:20.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:20.141 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-27T10:32:20.141 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-27T10:32:20.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.5 2024-05-27T10:32:20.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:20 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:20.011+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:20.555 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:20.194+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:20.555 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:20.194+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:20.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:20.173+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:20.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:20.173+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:20.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 systemd[1]: Stopping Ceph osd.5 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:20.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:20.346+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:20.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:20.346+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:20.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:20.346+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:20.835 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:20 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:20.561+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:20.835 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:20 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:20.561+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:20.841 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:20 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:20.518+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:20.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:20 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:20.518+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:20.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:20 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:20.539+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:20.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:20 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:20.539+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:20.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:20 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:20.539+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:21.056 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:20.726+0000 7f9e25800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:21.056 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:20.726+0000 7f9e25800700 -1 osd.5 95 *** Got signal Terminated *** 2024-05-27T10:32:21.056 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:20.726+0000 7f9e25800700 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:21.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:20 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:20.973+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:21.318 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:21.210+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:21.319 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:21.211+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:21.319 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:21.162+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:21.319 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:21.162+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:21.757 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:21.319+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:21.757 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:21.319+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:21.757 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:21.319+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:21.818 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:21 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:21.489+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:21.818 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:21 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:21.489+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:21.818 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:21 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:21.574+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:21.819 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:21 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:21.574+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:21.819 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:21 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:21.574+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:21.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:21.594+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:21.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:21.594+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:22.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:21.979+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:22.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:21 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:21.979+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:22.555 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:22.235+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:22.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:22.235+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:22.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:22.200+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:22.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:22.200+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:22.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:22.200+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:22.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:22.337+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:22.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:22.337+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:22.557 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:22.337+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:22.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:22 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:22.526+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:22.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:22 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:22.526+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:22.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:22 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:22.526+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:22.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:22 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:22.480+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:22.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:22 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:22.480+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:22.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:22.549+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:22.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:22.549+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:22.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:22.549+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:23.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:22.953+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:23.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:22 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:22.953+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:23.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:23.376+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:23.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:23.376+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:23.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:23.376+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:23.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:23.196+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:23.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:23.196+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:23.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:23.196+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:23.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:23.224+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:23.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:23.224+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:23.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:23.224+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:23.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:23 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:23.513+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:23.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:23 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:23.513+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:23.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:23 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:23.513+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:23.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:23 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:23.461+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:23.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:23 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:23.461+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:23.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:23 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:23.461+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:23.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:23 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:23.588+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:23.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:23 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:23.588+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:23.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:23 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:23.588+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:24.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:23 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:23.982+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:24.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:23 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:23.982+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:24.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:24.358+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:24.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:24.358+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:24.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:24.358+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:24.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:24.217+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:24.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:24.217+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:24.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:24.217+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:24.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:24.157+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:24.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:24.157+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:24.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:24 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:24.157+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:24.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:24 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:24.552+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:24.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:24 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:24.552+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:24.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:24 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:24.552+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:24.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:24 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:24.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:24.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:24 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:24.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:24.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:24 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:24.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:24.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:24 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:24.635+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:24.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:24 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:24.635+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:24.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:24 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:24.635+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:25.401 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:25.237+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:25.401 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:25.237+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:25.401 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:25.237+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:25.401 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:25.112+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:25.402 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:25.112+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:25.402 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:25.112+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:25.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:25.014+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:25.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:25.014+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:25.806 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:25.401+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:33.850023+0000 front 2024-05-27T10:31:33.850084+0000 (oldest deadline 2024-05-27T10:31:59.750211+0000) 2024-05-27T10:32:25.806 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:25.401+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.154388+0000 front 2024-05-27T10:31:46.154624+0000 (oldest deadline 2024-05-27T10:32:08.454233+0000) 2024-05-27T10:32:25.806 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5[47314]: 2024-05-27T10:32:25.401+0000 7f9e1c000700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.857737+0000 front 2024-05-27T10:31:54.857815+0000 (oldest deadline 2024-05-27T10:32:18.859000+0000) 2024-05-27T10:32:25.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:25.517+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:25.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:25.517+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:25.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:25.517+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:25.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:25.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:25.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:25.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:25.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:25 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:25.467+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:25.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:25.670+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:25.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:25.670+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:25.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:25.670+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:26.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:25.988+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:26.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:25 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:25.988+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:26.215+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:26.215+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:26.216+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:26 smithi143 podman[73841]: 2024-05-27 10:32:26.314841832 +0000 UTC m=+5.754029531 container died d622eec71dcc9ac9097124fa4c25a829fecb84276dc69dfa80ebdfe7d764045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:26.092+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:26.092+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:26.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:26 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:26.092+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:26.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:26 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:26.551+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:26.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:26 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:26.551+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:26.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:26 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:26.551+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:26.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:26 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:26.442+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:26.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:26 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:26.442+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:26.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:26 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:26.442+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:26.941 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:26.643+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:26.942 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:26.643+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:26.942 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:26.643+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:26.942 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:26.643+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:27.012 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:26 smithi143 podman[73841]: 2024-05-27 10:32:26.728900474 +0000 UTC m=+6.168088204 container cleanup d622eec71dcc9ac9097124fa4c25a829fecb84276dc69dfa80ebdfe7d764045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T10:32:27.012 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:26 smithi143 bash[73841]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5 2024-05-27T10:32:27.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:26.942+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:27.196 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:26 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:26.942+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:27.305 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:27 smithi143 podman[73872]: 2024-05-27 10:32:27.013333987 +0000 UTC m=+0.686885444 container remove d622eec71dcc9ac9097124fa4c25a829fecb84276dc69dfa80ebdfe7d764045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T10:32:27.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:27.198+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:27.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:27.198+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:27.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:27.198+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:27.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:27.075+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:27.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:27.075+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:27.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:27 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:27.075+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:27.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:27.515+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:27.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:27.515+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:27.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:27.515+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:27.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:27.476+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:27.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:27.476+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:27.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:27.476+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:27.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:27.476+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:27.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:27.595+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:27.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:27.595+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:27.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:27.595+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:27.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:27.595+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:28.117 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:27 smithi143 podman[73959]: 2024-05-27 10:32:27.756709056 +0000 UTC m=+0.032136787 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:32:28.117 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:28 smithi143 podman[73959]: 2024-05-27 10:32:28.031440869 +0000 UTC m=+0.306868637 container create 0d7df19d04be0cb38f0ae762cbef4f7b75b8ba9fd841d0fbd816da9672aceb81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:28.441 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:28 smithi143 podman[73959]: 2024-05-27 10:32:28.335660109 +0000 UTC m=+0.611087887 container init 0d7df19d04be0cb38f0ae762cbef4f7b75b8ba9fd841d0fbd816da9672aceb81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T10:32:28.441 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:28 smithi143 podman[73959]: 2024-05-27 10:32:28.347743753 +0000 UTC m=+0.623171513 container start 0d7df19d04be0cb38f0ae762cbef4f7b75b8ba9fd841d0fbd816da9672aceb81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:28.117+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:28.117+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:28.117+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:28.117+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:28.226+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:28.226+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:28.226+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:28.442 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:28 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:28.226+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:28.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:27.990+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:28.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:27 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:27.990+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:28.792 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:28 smithi143 podman[73959]: 2024-05-27 10:32:28.442661817 +0000 UTC m=+0.718089615 container attach 0d7df19d04be0cb38f0ae762cbef4f7b75b8ba9fd841d0fbd816da9672aceb81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:28.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:28.528+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:28.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:28.528+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:28.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:28.528+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:28.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:28.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:28.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:28.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:28.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:28.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:28.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:28 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:28.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:28.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:28.634+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:28.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:28.634+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:28.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:28.634+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:28.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:28 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:28.634+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:29.055 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:28 smithi143 podman[73984]: 2024-05-27 10:32:28.850132926 +0000 UTC m=+0.035193526 container died 0d7df19d04be0cb38f0ae762cbef4f7b75b8ba9fd841d0fbd816da9672aceb81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:32:29.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:29.026+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:29.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:29.026+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:29.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:29.026+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:29.513 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:29.255+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:29.513 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:29.255+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:29.513 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:29.255+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:29.513 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:29.255+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:29.514 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:29.140+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:29.514 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:29.140+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:29.514 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:29.140+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:29.514 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:29 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:29.140+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:29.514 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 27 10:32:29 smithi143 podman[73984]: 2024-05-27 10:32:29.513833229 +0000 UTC m=+0.698893850 container remove 0d7df19d04be0cb38f0ae762cbef4f7b75b8ba9fd841d0fbd816da9672aceb81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:29.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-27T10:32:29.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:29.677 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-27T10:32:29.678 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-27T10:32:29.678 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.6 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:29.510+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:29.510+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:29.510+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:29.480+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:29.480+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:29.480+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:29.760 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:29 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:29.480+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:29.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:29.659+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:29.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:29.659+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:29.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:29.659+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:29.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:29 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:29.659+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:30.092 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:29 smithi042 systemd[1]: Stopping Ceph osd.6 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:30.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:30.026+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:30.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:30.027+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:30.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:30.027+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:30.481 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.324+0000 7fe50ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:30.481 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.324+0000 7fe50ca00700 -1 osd.6 95 *** Got signal Terminated *** 2024-05-27T10:32:30.481 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.324+0000 7fe50ca00700 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:30.481 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.481+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:30.481 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.481+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:30.482 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.481+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:30.482 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:30.481+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:05.121375+0000 front 2024-05-27T10:32:05.121800+0000 (oldest deadline 2024-05-27T10:32:30.421561+0000) 2024-05-27T10:32:30.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:30.268+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:30.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:30.268+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:30.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:30.268+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:30.557 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:30.268+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:30.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:30.129+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:30.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:30.129+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:30.557 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:30.129+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:30.558 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:30 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:30.129+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:30.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:30.528+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:30.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:30.528+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:30.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:30.528+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:30.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:30 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:30.528+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:30.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:30.660+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:30.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:30.660+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:30.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:30.660+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:30.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:30 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:30.660+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:31.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:31.004+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:31.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:31.004+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:31.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:31.004+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:31.288+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:31.288+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:31.288+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:31.288+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:31.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:31.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:31.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:31.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:31 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:31.171+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:31.828 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:31.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:31.828 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:31.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:31.828 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:31.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:31.828 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:31.523+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:31.828 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:31.465+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:31.829 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:31.465+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:31.829 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:31.465+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:31.829 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:31 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:31.465+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:05.121375+0000 front 2024-05-27T10:32:05.121800+0000 (oldest deadline 2024-05-27T10:32:30.421561+0000) 2024-05-27T10:32:31.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:31.666+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:31.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:31.666+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:31.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:31.666+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:31.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:31 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:31.666+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:32.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:32.019+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:32.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:32.019+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:32.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:32.019+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:32.323+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:32.323+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:32.323+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:32.323+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:32.130+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:32.130+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:32.130+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:32.556 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:32 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:32.130+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:32.791 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:32.496+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:32.791 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:32.496+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:32.791 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:32.496+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:32.791 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:32.496+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:05.121375+0000 front 2024-05-27T10:32:05.121800+0000 (oldest deadline 2024-05-27T10:32:30.421561+0000) 2024-05-27T10:32:32.791 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:32.498+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:32.792 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:32.498+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:32.792 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:32.498+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:32.792 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:32 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:32.498+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:32.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:32.681+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:32.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:32.681+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:32.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:32.681+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:32.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:32 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:32.681+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:33.369 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:33.088+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:33.369 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:33.088+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:33.369 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:33.088+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:33.369 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:33.088+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:33.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:33.030+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:33.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:33.030+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:33.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:33.030+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:33.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:33.368+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:33.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:33.368+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:33.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:33.368+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:33.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:33 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:33.369+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:33.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:33.508+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:33.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:33.508+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:33.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:33.508+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:33.843 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:33.508+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:05.121375+0000 front 2024-05-27T10:32:05.121800+0000 (oldest deadline 2024-05-27T10:32:30.421561+0000) 2024-05-27T10:32:33.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:33.503+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:33.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:33.503+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:33.844 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:33.503+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:33.844 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:33.503+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:34.004 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:33.710+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:34.004 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:33.710+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:34.004 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:33.710+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:34.004 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:33 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:33.710+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:34.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:34.045+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:34.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:34.045+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:34.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:34.045+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:34.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:34.045+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:34.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:34.005+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:34.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:34.005+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:34.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:34.005+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:34.810 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:34.403+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:34.810 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:34.403+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:34.810 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:34.403+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:34.810 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:34 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:34.403+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:34.846 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:34.550+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:32.508381+0000 front 2024-05-27T10:31:32.508312+0000 (oldest deadline 2024-05-27T10:32:03.109644+0000) 2024-05-27T10:32:34.846 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:34.550+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.212207+0000 front 2024-05-27T10:31:44.212000+0000 (oldest deadline 2024-05-27T10:32:07.712217+0000) 2024-05-27T10:32:34.846 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:34.550+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.217493+0000 front 2024-05-27T10:31:55.217389+0000 (oldest deadline 2024-05-27T10:32:17.517169+0000) 2024-05-27T10:32:34.846 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6[60393]: 2024-05-27T10:32:34.550+0000 7fe503200700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:05.121375+0000 front 2024-05-27T10:32:05.121800+0000 (oldest deadline 2024-05-27T10:32:30.421561+0000) 2024-05-27T10:32:34.847 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:34.482+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:34.847 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:34.482+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:34.847 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:34.482+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:34.847 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:34 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:34.482+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:34.952 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:34.685+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:34.952 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:34.685+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:34.952 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:34.685+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:34.953 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:34 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:34.685+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:35.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:35.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:35.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:35.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:35.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:35.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:35.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:35.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:35.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:35.054+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:35.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:35.054+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:35.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:35.054+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:35.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:35.447+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:35.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:35.447+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:35.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:35.447+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:35.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:35 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:35.447+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:35.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:35.434+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:35.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:35.434+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:35.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:35.434+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:35.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:35 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:35.434+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:35.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:35.687+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:35.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:35.687+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:35.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:35.687+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:35.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:35 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:35.687+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:36.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:36.026+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:36.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:36.026+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:36.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:36.026+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:36.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:36.026+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:36.313 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:35 smithi042 podman[159921]: 2024-05-27 10:32:35.952615698 +0000 UTC m=+5.784374934 container died 49a4ff8a67369a9f92320331c4df3c2a5677d14ed88fe1136b6046e08a3fbb4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T10:32:36.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:36.094+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:36.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:36.094+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:36.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:36.094+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:36.591 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:36 smithi042 podman[159921]: 2024-05-27 10:32:36.492801461 +0000 UTC m=+6.324560660 container cleanup 49a4ff8a67369a9f92320331c4df3c2a5677d14ed88fe1136b6046e08a3fbb4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:36.592 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:36 smithi042 bash[159921]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6 2024-05-27T10:32:36.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:36.465+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:36.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:36.465+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:36.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:36.465+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:36.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:36 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:36.465+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:36.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:36.427+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:36.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:36.427+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:36.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:36.427+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:36.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:36 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:36.427+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:37.091 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:36 smithi042 podman[159953]: 2024-05-27 10:32:36.796190103 +0000 UTC m=+0.831393815 container remove 49a4ff8a67369a9f92320331c4df3c2a5677d14ed88fe1136b6046e08a3fbb4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:32:37.102 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:36.725+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:37.102 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:36.725+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:37.102 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:36.725+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:37.102 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:36 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:36.725+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:37.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:37.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:37.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:37.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:37.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:37.102+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:37.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:37.102+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:37.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:37.102+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:37.727 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:37 smithi042 podman[160039]: 2024-05-27 10:32:37.587129917 +0000 UTC m=+0.041152221 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:32:37.727 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:37.471+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:37.727 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:37.471+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:37.727 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:37.471+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:37.727 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:37.471+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:37.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:37.413+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:37.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:37.413+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:37.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:37.413+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:37.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:37.413+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:38.041 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:37 smithi042 podman[160039]: 2024-05-27 10:32:37.728578274 +0000 UTC m=+0.182600639 container create d63fbb15927984109621988dc6be1e55ba3d99b2573e18d8ad043be0626d6163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T10:32:38.041 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:38 smithi042 podman[160039]: 2024-05-27 10:32:38.042864379 +0000 UTC m=+0.496886694 container init d63fbb15927984109621988dc6be1e55ba3d99b2573e18d8ad043be0626d6163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:38.081 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:37.758+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:38.081 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:37.758+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:38.081 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:37.758+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:38.081 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:37 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:37.758+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:38.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.985+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:38.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.985+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:38.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.985+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:38.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:37 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:37.985+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:38.342 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:38 smithi042 podman[160039]: 2024-05-27 10:32:38.054689934 +0000 UTC m=+0.508712279 container start d63fbb15927984109621988dc6be1e55ba3d99b2573e18d8ad043be0626d6163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:38.342 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:38 smithi042 podman[160039]: 2024-05-27 10:32:38.149970595 +0000 UTC m=+0.603992969 container attach d63fbb15927984109621988dc6be1e55ba3d99b2573e18d8ad043be0626d6163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:32:38.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:38.082+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:38.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:38.082+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:38.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:38.082+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:38.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:38.082+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:38.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:38.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:38.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:38.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:38.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:38.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:38.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:38 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:38.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:38.842 INFO:journalctl@ceph.osd.6.smithi042.stdout:May 27 10:32:38 smithi042 podman[160065]: 2024-05-27 10:32:38.583728423 +0000 UTC m=+0.034413754 container died d63fbb15927984109621988dc6be1e55ba3d99b2573e18d8ad043be0626d6163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T10:32:38.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:38.444+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:38.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:38.444+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:38.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:38.444+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:38.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:38 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:38.444+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:39.039 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:38.737+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:39.039 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:38.737+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:39.039 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:38.737+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:39.040 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:38.737+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:39.040 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:38 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:38.737+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:39.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:39.016+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:39.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:39.016+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:39.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:39.017+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:39.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:39.017+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:39.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.6.service' 2024-05-27T10:32:39.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:39.040+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:39.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:39.040+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:39.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:39.040+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:39.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:39.040+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:39.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:39.571 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-27T10:32:39.571 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-27T10:32:39.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.7 2024-05-27T10:32:39.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:39.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:39.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:39.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:39.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:39.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:39.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:39 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:39.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:39.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:39.400+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:39.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:39.400+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:39.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:39.400+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:39.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:39 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:39.400+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:39.946 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:39 smithi104 systemd[1]: Stopping Ceph osd.7 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:39.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:39.733+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:39.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:39.733+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:39.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:39.733+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:39.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:39.733+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:39.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:39 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:39.733+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:40.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:40.036+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:40.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:40.036+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:40.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:40.036+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:40.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:40.036+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.090+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.090+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.090+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.090+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.150+0000 7f55ec000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.150+0000 7f55ec000700 -1 osd.7 95 *** Got signal Terminated *** 2024-05-27T10:32:40.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:40.150+0000 7f55ec000700 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:40.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:40.341+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:40.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:40.341+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:40.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:40.341+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:40.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:40 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:40.341+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:40.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:40.362+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:40.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:40.362+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:40.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:40.362+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:40.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:40.362+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:40.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:40 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:40.362+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:41.100 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:40.783+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:41.100 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:40.783+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:41.100 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:40.783+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:41.100 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:40.783+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:41.100 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:40 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:40.783+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:41.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:41.001+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:41.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:41.002+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:41.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:41.002+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:41.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:41.002+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:41.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:41.101+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:41.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:41.101+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:41.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:41.101+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:41.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:41.101+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:41.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:41.352+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:41.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:41.352+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:41.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:41.352+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:41.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:41.352+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:41.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:41 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:41.352+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:41.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:41.376+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:41.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:41.376+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:41.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:41.376+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:41.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:41.376+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:41.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:41 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:41.376+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:42.071 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:41.803+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:42.071 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:41.803+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:42.071 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:41.803+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:42.071 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:41.803+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:42.072 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:41 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:41.803+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:42.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:42.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:42.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:42.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:42.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:42.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:42.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:42.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:42.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:42.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:42.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:42.072+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:42.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:42.072+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:42.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:42.072+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:42.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:42.072+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:42.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:42.369+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:42.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:42.369+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:42.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:42.369+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:42.593 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:42.369+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:42.593 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:42.369+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:42.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:42.398+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:42.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:42.398+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:42.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:42.398+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:42.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:42.398+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:42.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:42 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:42.398+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:43.087 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:42.813+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:43.087 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:42.813+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:43.088 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:42.813+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:43.088 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:42.813+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:43.088 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:42 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:42.813+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:43.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:43.003+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:43.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:43.003+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:43.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:43.003+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:43.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:43.003+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:43.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:43.003+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:43.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:43.088+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:43.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:43.088+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:43.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:43.088+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:43.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:43.088+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:43.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:43.380+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:43.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:43.380+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:43.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:43.380+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:43.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:43.380+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:43.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:43 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:43.380+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:43.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:43.379+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:43.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:43.379+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:43.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:43.379+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:43.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:43.379+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:43.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:43 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:43.379+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:44.119 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:43.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:44.120 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:43.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:44.120 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:43.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:44.120 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:43.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:44.120 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:43 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:43.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:44.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:44.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:44.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:44.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:44.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:44.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:44.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:44.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:44.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:44.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:44.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:44.120+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.524567+0000 front 2024-05-27T10:31:43.524902+0000 (oldest deadline 2024-05-27T10:32:07.624802+0000) 2024-05-27T10:32:44.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:44.120+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:53.526569+0000 front 2024-05-27T10:31:53.527082+0000 (oldest deadline 2024-05-27T10:32:21.127622+0000) 2024-05-27T10:32:44.446 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:44.120+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.429530+0000 front 2024-05-27T10:32:03.429733+0000 (oldest deadline 2024-05-27T10:32:28.129809+0000) 2024-05-27T10:32:44.447 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7[52890]: 2024-05-27T10:32:44.120+0000 7f55e7800700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.433838+0000 front 2024-05-27T10:32:14.433748+0000 (oldest deadline 2024-05-27T10:32:37.333613+0000) 2024-05-27T10:32:44.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:44.339+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:44.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:44.339+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:44.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:44.339+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:44.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:44.339+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:44.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:44 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:44.339+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:44.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:44.389+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:44.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:44.389+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:44.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:44.389+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:44.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:44.389+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:44.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:44 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:44.389+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:45.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:44.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:45.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:44.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:45.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:44.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:45.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:44.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:45.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:44 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:44.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:45.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:45.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:45.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:45.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:45.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:45.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:45.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:45.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:45.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:45.006+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:45.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:45.336+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:45.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:45.336+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:45.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:45.336+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:45.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:45.336+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:45.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:45 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:45.336+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:45.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:45.433+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:45.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:45.433+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:45.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:45.433+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:45.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:45.433+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:45.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:45 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:45.433+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:45.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:45 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:45.837+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:45.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:45 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:45.837+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:45.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:45 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:45.837+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:45.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:45 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:45.837+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:45.947 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:45 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:45.837+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:45.947 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:45 smithi104 podman[72478]: 2024-05-27 10:32:45.673696368 +0000 UTC m=+5.683832291 container died d0c51dcd1de3688bb3f87ff1d8c1e4e6bb4309892051b850e21b3a236d5f0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T10:32:46.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:46.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:46.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:46.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:46.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:46.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:46.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:46.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:46.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:46.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:46.418 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:46 smithi104 podman[72478]: 2024-05-27 10:32:46.17553913 +0000 UTC m=+6.185675082 container cleanup d0c51dcd1de3688bb3f87ff1d8c1e4e6bb4309892051b850e21b3a236d5f0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-27T10:32:46.418 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:46 smithi104 bash[72478]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7 2024-05-27T10:32:46.696 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:46 smithi104 podman[72496]: 2024-05-27 10:32:46.418368887 +0000 UTC m=+0.729637948 container remove d0c51dcd1de3688bb3f87ff1d8c1e4e6bb4309892051b850e21b3a236d5f0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:32:46.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:46.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:46.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:46.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:46.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:46.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:46.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:46.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:46.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:46 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:46.377+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:46.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:46.446+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:46.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:46.446+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:46.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:46.446+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:46.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:46.446+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:46.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:46 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:46.446+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:47.199 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:46 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:46.811+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:47.199 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:46 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:46.811+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:47.199 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:46 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:46.811+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:47.199 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:46 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:46.811+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:47.199 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:46 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:46.811+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:47.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:47.023+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:47.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:47.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:47.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:47.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:47.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:47.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:47.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:47.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:47.574 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:47 smithi104 podman[72583]: 2024-05-27 10:32:47.141020404 +0000 UTC m=+0.043633904 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:32:47.574 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:47 smithi104 podman[72583]: 2024-05-27 10:32:47.261118753 +0000 UTC m=+0.163732261 container create e43780a4092931cb15a041e0e2f887284219f7701d2f6cfc17d9334d32c1b7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T10:32:47.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:47.375+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:47.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:47.375+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:47.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:47.375+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:47.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:47.375+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:47.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:47 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:47.375+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:47.835 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:47 smithi104 podman[72583]: 2024-05-27 10:32:47.586147347 +0000 UTC m=+0.488760885 container init e43780a4092931cb15a041e0e2f887284219f7701d2f6cfc17d9334d32c1b7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T10:32:47.835 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:47 smithi104 podman[72583]: 2024-05-27 10:32:47.596333122 +0000 UTC m=+0.498946637 container start e43780a4092931cb15a041e0e2f887284219f7701d2f6cfc17d9334d32c1b7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:47.835 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:47 smithi104 podman[72583]: 2024-05-27 10:32:47.696877379 +0000 UTC m=+0.599490865 container attach e43780a4092931cb15a041e0e2f887284219f7701d2f6cfc17d9334d32c1b7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T10:32:47.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:47 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:47.399+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:47.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:47 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:47.399+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:47.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:47 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:47.399+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:47.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:47 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:47.399+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:47.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:47 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:47.399+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:48.197 INFO:journalctl@ceph.osd.7.smithi104.stdout:May 27 10:32:48 smithi104 podman[72583]: 2024-05-27 10:32:48.058370137 +0000 UTC m=+0.960983609 container died e43780a4092931cb15a041e0e2f887284219f7701d2f6cfc17d9334d32c1b7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:48.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:47 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:47.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:48.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:47 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:47.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:48.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:47 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:47.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:48.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:47 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:47.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:48.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:47 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:47.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:48.198 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:47 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:47.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:48.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.004+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:48.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.004+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:48.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.004+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:48.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.004+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:48.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.004+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:48.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:48.381+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:48.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:48.381+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:48.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:48.381+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:48.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:48.381+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:48.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:48.381+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:48.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:48 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:48.356+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:48.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:48 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:48.356+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:48.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:48 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:48.356+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:48.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:48 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:48.356+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:48.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:48 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:48.356+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:48.919 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.7.service' 2024-05-27T10:32:49.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:49.033 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-27T10:32:49.033 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-27T10:32:49.033 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.8 2024-05-27T10:32:49.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:48.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:49.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:48.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:49.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:48.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:49.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:48.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:49.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:48.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:49.198 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:48.836+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:49.306 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 systemd[1]: Stopping Ceph osd.8 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:49.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.972+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:49.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.972+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:49.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.972+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:49.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.972+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:49.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.972+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:49.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:48 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:48.972+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:49.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.376+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:49.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.376+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:49.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.376+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:49.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.376+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:49.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.376+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:49.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.376+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:24.117007+0000 front 2024-05-27T10:32:24.116796+0000 (oldest deadline 2024-05-27T10:32:48.816928+0000) 2024-05-27T10:32:49.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.623+0000 7f493d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:49.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.623+0000 7f493d000700 -1 osd.8 95 *** Got signal Terminated *** 2024-05-27T10:32:49.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:49.623+0000 7f493d000700 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:49.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:49 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:49.359+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:49.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:49 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:49.359+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:49.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:49 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:49.359+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:49.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:49 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:49.359+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:49.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:49 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:49.359+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:50.051 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:49.792+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:50.051 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:49.792+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:50.051 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:49.792+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:50.052 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:49.792+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:50.052 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:49.792+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:50.052 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:49 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:49.792+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:50.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:49.949+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:50.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:49.949+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:50.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:49.949+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:50.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:49.949+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:50.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:49.949+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:50.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:49 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:49.949+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:50.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:50.310+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:50.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:50.310+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:50.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:50.310+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:50.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:50.310+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:50.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:50 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:50.310+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:50.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:50.327+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:50.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:50.327+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:50.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:50.327+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:50.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:50.327+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:50.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:50.327+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:50.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:50.327+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:24.117007+0000 front 2024-05-27T10:32:24.116796+0000 (oldest deadline 2024-05-27T10:32:48.816928+0000) 2024-05-27T10:32:51.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:50 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:50.781+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:51.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:50 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:50.781+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:51.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:50 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:50.781+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:51.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:50 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:50.781+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:51.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:50 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:50.781+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:51.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:50 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:50.781+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:51.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:50.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:51.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:50.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:51.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:50.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:51.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:50.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:51.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:50.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:51.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:50 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:50.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:51.677 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:51.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:51.677 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:51.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:51.677 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:51.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:51.678 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:51.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:51.678 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:51.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:51.678 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:51.345+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:24.117007+0000 front 2024-05-27T10:32:24.116796+0000 (oldest deadline 2024-05-27T10:32:48.816928+0000) 2024-05-27T10:32:51.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:51 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:51.349+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:51.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:51 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:51.349+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:51.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:51 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:51.349+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:51.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:51 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:51.349+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:51.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:51 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:51.349+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:52.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:51 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:51.786+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:52.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:51 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:51.786+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:52.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:51 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:51.786+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:52.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:51 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:51.786+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:52.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:51 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:51.786+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:52.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:51 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:51.786+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:52.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:51.994+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:52.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:51.994+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:52.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:51.994+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:52.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:51.994+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:52.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:51.994+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:52.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:51 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:51.994+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:52.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:52.364+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:52.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:52.364+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:52.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:52.364+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:52.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:52.364+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:52.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:52.364+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:52.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:52.364+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:24.117007+0000 front 2024-05-27T10:32:24.116796+0000 (oldest deadline 2024-05-27T10:32:48.816928+0000) 2024-05-27T10:32:52.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:52 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:52.391+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:52.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:52 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:52.391+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:52.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:52 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:52.391+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:52.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:52 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:52.391+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:52.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:52 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:52.391+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:53.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:52 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:52.768+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:53.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:52 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:52.768+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:53.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:52 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:52.768+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:53.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:52 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:52.768+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:53.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:52 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:52.768+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:53.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:52 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:52.768+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:53.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:52.992+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:53.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:52.992+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:53.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:52.992+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:53.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:52.992+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:53.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:52.992+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:53.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:52 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:52.992+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:53.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:53 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:53.404+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:53.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:53 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:53.404+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:53.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:53 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:53.404+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:53.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:53 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:53.404+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:53.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:53 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:53.404+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:53.807 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:53 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:53.404+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:24.117007+0000 front 2024-05-27T10:32:24.116796+0000 (oldest deadline 2024-05-27T10:32:48.816928+0000) 2024-05-27T10:32:53.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:53.353+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:53.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:53.353+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:53.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:53.353+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:53.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:53.353+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:53.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:53 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:53.353+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:54.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:53 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:53.800+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:54.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:53 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:53.800+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:54.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:53 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:53.800+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:54.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:53 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:53.800+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:54.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:53 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:53.800+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:54.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:53 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:53.800+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:54.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:54.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:54.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:54.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:54.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:54.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:54.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:54.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:54.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:54.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:54.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:54.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:54.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:54.356+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.802078+0000 front 2024-05-27T10:31:36.801952+0000 (oldest deadline 2024-05-27T10:31:58.501820+0000) 2024-05-27T10:32:54.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:54.356+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:44.404766+0000 front 2024-05-27T10:31:44.404440+0000 (oldest deadline 2024-05-27T10:32:07.904618+0000) 2024-05-27T10:32:54.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:54.356+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.108026+0000 front 2024-05-27T10:31:56.108296+0000 (oldest deadline 2024-05-27T10:32:22.008335+0000) 2024-05-27T10:32:54.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:54.356+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.009253+0000 front 2024-05-27T10:32:02.009830+0000 (oldest deadline 2024-05-27T10:32:27.309261+0000) 2024-05-27T10:32:54.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:54.356+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.715372+0000 front 2024-05-27T10:32:14.715045+0000 (oldest deadline 2024-05-27T10:32:40.614905+0000) 2024-05-27T10:32:54.806 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:54 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8[54139]: 2024-05-27T10:32:54.356+0000 7f4933800700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:24.117007+0000 front 2024-05-27T10:32:24.116796+0000 (oldest deadline 2024-05-27T10:32:48.816928+0000) 2024-05-27T10:32:54.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:54.381+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:54.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:54.381+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:54.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:54.381+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:54.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:54.381+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:54.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:54 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:54.381+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:55.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:54.782+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:55.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:54.782+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:55.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:54.782+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:55.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:54.782+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:55.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:54.782+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:55.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:54 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:54.782+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:55.305 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:55 smithi143 podman[74064]: 2024-05-27 10:32:55.03037018 +0000 UTC m=+5.563566207 container died 49e274f21a1c55fd730d2d9b5113e64ed42cf0fb4a4022dbd355be5145621c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:32:55.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.005+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:55.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.005+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:55.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.005+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:55.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.005+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:55.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.005+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:55.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.005+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:55.724 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:55 smithi143 podman[74064]: 2024-05-27 10:32:55.5414118 +0000 UTC m=+6.074607894 container cleanup 49e274f21a1c55fd730d2d9b5113e64ed42cf0fb4a4022dbd355be5145621c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T10:32:55.724 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:55 smithi143 bash[74064]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8 2024-05-27T10:32:55.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:55 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:55.374+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:55.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:55 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:55.374+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:55.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:55 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:55.374+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:55.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:55 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:55.374+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:55.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:55 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:55.374+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:56.055 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:56.056 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:56.056 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:56.056 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:56.056 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:56.056 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:55 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:55.957+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:56.056 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:55 smithi143 podman[74091]: 2024-05-27 10:32:55.726027759 +0000 UTC m=+0.681171643 container remove 49e274f21a1c55fd730d2d9b5113e64ed42cf0fb4a4022dbd355be5145621c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:56.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:55 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:55.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:56.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:55 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:55.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:56.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:55 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:55.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:56.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:55 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:55.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:56.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:55 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:55.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:56.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:55 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:55.826+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:56.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:56.421+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:56.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:56.421+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:56.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:56.421+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:56.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:56.421+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:56.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:56 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:56.421+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:56.873 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:56 smithi143 podman[74184]: 2024-05-27 10:32:56.474795447 +0000 UTC m=+0.032834477 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:32:56.873 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:56 smithi143 podman[74184]: 2024-05-27 10:32:56.593825068 +0000 UTC m=+0.151864138 container create cf0f69ba4003e6e8673349afc5fc1bf0ff11d9b2e3ea035cbeed52f5590590f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:32:56.873 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:56 smithi143 podman[74184]: 2024-05-27 10:32:56.873825622 +0000 UTC m=+0.431864732 container init cf0f69ba4003e6e8673349afc5fc1bf0ff11d9b2e3ea035cbeed52f5590590f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T10:32:57.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:57.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:57.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:57.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:57.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:57.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:57.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:56.839+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:32:57.305 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:56 smithi143 podman[74184]: 2024-05-27 10:32:56.888344335 +0000 UTC m=+0.446383372 container start cf0f69ba4003e6e8673349afc5fc1bf0ff11d9b2e3ea035cbeed52f5590590f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:32:57.305 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:56 smithi143 podman[74184]: 2024-05-27 10:32:56.987971509 +0000 UTC m=+0.546010583 container attach cf0f69ba4003e6e8673349afc5fc1bf0ff11d9b2e3ea035cbeed52f5590590f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:32:57.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:56.938+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:57.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:56.938+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:57.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:56.938+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:57.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:56.938+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:57.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:56.938+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:57.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:56 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:56.938+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:57.694 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:57 smithi143 podman[74210]: 2024-05-27 10:32:57.402351399 +0000 UTC m=+0.038261855 container died cf0f69ba4003e6e8673349afc5fc1bf0ff11d9b2e3ea035cbeed52f5590590f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T10:32:57.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:57.372+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:57.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:57.372+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:57.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:57.372+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:57.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:57.372+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:57.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:57 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:57.372+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:58.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.8.service' 2024-05-27T10:32:58.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:58.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:58.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:58.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:58.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:58.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:58.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:57 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:57.859+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:32:58.223 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:58 smithi143 podman[74210]: 2024-05-27 10:32:58.100713158 +0000 UTC m=+0.736623617 container remove cf0f69ba4003e6e8673349afc5fc1bf0ff11d9b2e3ea035cbeed52f5590590f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:32:58.224 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:58 smithi143 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.8.service: Deactivated successfully. 2024-05-27T10:32:58.224 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:58 smithi143 systemd[1]: Stopped Ceph osd.8 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:32:58.224 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 27 10:32:58 smithi143 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.8.service: Consumed 1min 17.319s CPU time. 2024-05-27T10:32:58.224 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:57 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:57.964+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:58.224 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:57 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:57.964+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:58.225 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:57 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:57.964+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:58.225 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:57 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:57.964+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:58.225 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:57 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:57.964+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:58.225 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:57 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:57.964+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:58.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:32:58.260 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-27T10:32:58.260 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-27T10:32:58.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.9 2024-05-27T10:32:58.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.354+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:58.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.354+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:58.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.354+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:58.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.354+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:58.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.354+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:58.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 systemd[1]: Stopping Ceph osd.9 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:32:59.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:32:59.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:32:59.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:32:59.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:32:59.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:32:59.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:32:59.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:58 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:58.862+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:32:59.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.010+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:32:59.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.010+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:32:59.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.010+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:32:59.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.010+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:32:59.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.010+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:32:59.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.010+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:32:59.342 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.872+0000 7f54fde00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:32:59.342 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.872+0000 7f54fde00700 -1 osd.9 95 *** Got signal Terminated *** 2024-05-27T10:32:59.342 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:58 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:58.872+0000 7f54fde00700 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:32:59.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:59.366+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:32:59.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:59.366+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:32:59.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:59.366+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:32:59.844 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:59.366+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:32:59.844 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:59.366+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:32:59.844 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:32:59 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:32:59.366+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:32.923096+0000 front 2024-05-27T10:32:32.923181+0000 (oldest deadline 2024-05-27T10:32:58.823076+0000) 2024-05-27T10:33:00.185 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:00.185 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:00.185 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:00.186 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:00.186 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:00.186 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:00.186 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:32:59 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:32:59.908+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:00.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.996+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:00.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.996+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:00.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.996+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:00.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.996+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:00.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.996+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:00.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:32:59 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:32:59.996+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:00.746 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:00.396+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:33:00.746 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:00.396+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:33:00.746 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:00.396+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:33:00.746 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:00.396+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:33:00.746 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:00.396+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:33:00.746 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:00 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:00.396+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:32.923096+0000 front 2024-05-27T10:32:32.923181+0000 (oldest deadline 2024-05-27T10:32:58.823076+0000) 2024-05-27T10:33:01.196 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:01.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:01.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:01.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:01.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:01.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:01.197 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:00.917+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:01.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:00.968+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:01.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:00.968+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:01.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:00.968+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:01.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:00.968+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:01.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:00.968+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:01.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:00 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:00.968+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:01.782 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:01.375+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:33:01.782 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:01.375+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:33:01.782 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:01.375+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:33:01.782 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:01.375+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:33:01.783 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:01.375+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:33:01.783 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:01 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:01.375+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:32.923096+0000 front 2024-05-27T10:32:32.923181+0000 (oldest deadline 2024-05-27T10:32:58.823076+0000) 2024-05-27T10:33:02.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:01.945+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:02.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:01.945+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:02.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:01.945+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:02.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:01.945+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:02.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:01.945+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:02.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:01 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:01.945+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:02.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:02.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:02.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:02.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:02.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:02.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:02.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:01 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:01.954+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:02.471 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:02.357+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:33:02.471 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:02.357+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:33:02.472 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:02.357+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:33:02.472 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:02.357+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:33:02.472 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:02.357+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:33:02.472 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:02 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:02.357+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:32.923096+0000 front 2024-05-27T10:32:32.923181+0000 (oldest deadline 2024-05-27T10:32:58.823076+0000) 2024-05-27T10:33:03.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:02.951+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:03.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:02.951+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:03.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:02.951+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:03.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:02.951+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:03.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:02.951+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:03.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:02 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:02.951+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:03.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:02 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:02.993+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:03.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:03.404+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.203157+0000 front 2024-05-27T10:31:34.203225+0000 (oldest deadline 2024-05-27T10:32:02.404311+0000) 2024-05-27T10:33:03.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:03.404+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:42.405612+0000 front 2024-05-27T10:31:42.405808+0000 (oldest deadline 2024-05-27T10:32:08.207416+0000) 2024-05-27T10:33:03.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:03.404+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:54.610583+0000 front 2024-05-27T10:31:54.610692+0000 (oldest deadline 2024-05-27T10:32:22.811727+0000) 2024-05-27T10:33:03.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:03.404+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.313983+0000 front 2024-05-27T10:32:03.314229+0000 (oldest deadline 2024-05-27T10:32:27.413906+0000) 2024-05-27T10:33:03.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:03.404+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:14.218722+0000 front 2024-05-27T10:32:14.218113+0000 (oldest deadline 2024-05-27T10:32:39.518209+0000) 2024-05-27T10:33:03.843 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:03 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9[67426]: 2024-05-27T10:33:03.404+0000 7f54f9600700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:32.923096+0000 front 2024-05-27T10:32:32.923181+0000 (oldest deadline 2024-05-27T10:32:58.823076+0000) 2024-05-27T10:33:04.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:03.999+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:04.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:03.999+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:04.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:03.999+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:04.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:03.999+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:04.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:03.999+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:04.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:03.999+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:04.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:04 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:04.040+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:04.592 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:04 smithi042 podman[160144]: 2024-05-27 10:33:04.302714101 +0000 UTC m=+5.603182711 container died d84a9fdc6d0d61f17526f016b02d12e06958dcbc86065235b15b6e0ca5014220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521) 2024-05-27T10:33:05.092 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:04 smithi042 podman[160144]: 2024-05-27 10:33:04.96508554 +0000 UTC m=+6.265554143 container cleanup d84a9fdc6d0d61f17526f016b02d12e06958dcbc86065235b15b6e0ca5014220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T10:33:05.092 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:04 smithi042 bash[160144]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9 2024-05-27T10:33:05.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:04.950+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:05.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:04.950+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:05.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:04.950+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:05.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:04.950+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:05.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:04.950+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:05.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:04 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:04.950+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:05.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:05 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.041+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:05.591 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:05 smithi042 podman[160173]: 2024-05-27 10:33:05.233591425 +0000 UTC m=+0.911673585 container remove d84a9fdc6d0d61f17526f016b02d12e06958dcbc86065235b15b6e0ca5014220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T10:33:06.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:05.907+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:06.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:05.907+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:06.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:05.907+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:06.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:05.907+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:06.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:05.907+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:06.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:05 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:05.907+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:06.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:06.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:06.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:06.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:06.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:06.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:06.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:05.999+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:06.455 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:06 smithi042 podman[160263]: 2024-05-27 10:33:06.011594499 +0000 UTC m=+0.033039016 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:33:06.455 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:06 smithi042 podman[160263]: 2024-05-27 10:33:06.301278188 +0000 UTC m=+0.322722742 container create 4d54214b09fc2e6c3c4a5f071d2f27176028b67819a4746509d7b6ea39ea225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-27T10:33:06.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:06 smithi042 podman[160263]: 2024-05-27 10:33:06.563307664 +0000 UTC m=+0.584752241 container init 4d54214b09fc2e6c3c4a5f071d2f27176028b67819a4746509d7b6ea39ea225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T10:33:06.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:06 smithi042 podman[160263]: 2024-05-27 10:33:06.574652722 +0000 UTC m=+0.596097282 container start 4d54214b09fc2e6c3c4a5f071d2f27176028b67819a4746509d7b6ea39ea225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T10:33:06.842 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:06 smithi042 podman[160263]: 2024-05-27 10:33:06.695283437 +0000 UTC m=+0.716727980 container attach 4d54214b09fc2e6c3c4a5f071d2f27176028b67819a4746509d7b6ea39ea225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T10:33:07.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:06.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:07.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:06.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:07.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:06.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:07.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:06.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:07.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:06.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:07.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:06 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:06.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:07.342 INFO:journalctl@ceph.osd.9.smithi042.stdout:May 27 10:33:07 smithi042 podman[160263]: 2024-05-27 10:33:07.042365522 +0000 UTC m=+1.063810096 container died 4d54214b09fc2e6c3c4a5f071d2f27176028b67819a4746509d7b6ea39ea225f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T10:33:07.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:07.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:07.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:07.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:07.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:07.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:07.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:06 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:06.996+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:08.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:08.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:08.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:08.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:08.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:08.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:08.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:07 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:07.939+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:08.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.9.service' 2024-05-27T10:33:08.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:08.437 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-27T10:33:08.437 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-27T10:33:08.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.10 2024-05-27T10:33:08.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.958+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:08.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.958+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:08.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.958+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:08.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.958+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:08.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.958+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:08.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.958+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:08.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:07.959+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:08.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 systemd[1]: Stopping Ceph osd.10 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:09.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:09.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:09.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:09.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:09.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:09.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:09.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:08 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:08.965+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:08 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:08.964+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:09.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.180+0000 7fc98d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:09.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.180+0000 7fc98d800700 -1 osd.10 95 *** Got signal Terminated *** 2024-05-27T10:33:09.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.180+0000 7fc98d800700 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:10.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:09 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:09.934+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:10.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:10.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:10.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:10.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:10.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:10.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:10.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:09 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:09.971+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:11.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:11.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:11.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:11.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:11.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:11.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:11.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:10 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:10.948+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:11.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:11.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:11.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:11.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:11.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:11.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:11.448 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:10 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:10.977+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:12.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:11 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:11.914+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:12.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:12.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:12.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:12.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:12.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:12.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:12.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:11 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:11.949+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:13.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:12.912+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:13.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:13.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:13.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:13.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:13.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:13.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:13.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:12 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:12.978+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:14.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:14.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:14.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:14.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:14.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:14.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:14.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:13 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:13.943+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:36.756335+0000 front 2024-05-27T10:31:36.756530+0000 (oldest deadline 2024-05-27T10:32:03.157691+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:43.158595+0000 front 2024-05-27T10:31:43.159385+0000 (oldest deadline 2024-05-27T10:32:08.458830+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:56.062178+0000 front 2024-05-27T10:31:56.062253+0000 (oldest deadline 2024-05-27T10:32:21.962374+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:03.064272+0000 front 2024-05-27T10:32:03.064462+0000 (oldest deadline 2024-05-27T10:32:26.564624+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.268232+0000 front 2024-05-27T10:32:15.268803+0000 (oldest deadline 2024-05-27T10:32:38.669343+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:23.872603+0000 front 2024-05-27T10:32:23.872460+0000 (oldest deadline 2024-05-27T10:32:47.372264+0000) 2024-05-27T10:33:14.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:13 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10[59805]: 2024-05-27T10:33:13.963+0000 7fc989000700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-05-27T10:32:31.474087+0000 front 2024-05-27T10:32:31.474309+0000 (oldest deadline 2024-05-27T10:32:56.173727+0000) 2024-05-27T10:33:14.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:14 smithi104 podman[72699]: 2024-05-27 10:33:14.638557998 +0000 UTC m=+5.733996180 container died 94a673679b5686a07ef06cb478d47582723aab5f1b6d09db3e68c5a487aedf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:15.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:14 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:14.975+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:15.446 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:15 smithi104 podman[72699]: 2024-05-27 10:33:15.226858959 +0000 UTC m=+6.322297181 container cleanup 94a673679b5686a07ef06cb478d47582723aab5f1b6d09db3e68c5a487aedf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T10:33:15.447 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:15 smithi104 bash[72699]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10 2024-05-27T10:33:15.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:15 smithi104 podman[72715]: 2024-05-27 10:33:15.586680776 +0000 UTC m=+0.926116927 container remove 94a673679b5686a07ef06cb478d47582723aab5f1b6d09db3e68c5a487aedf00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:16.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:15 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:15.993+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:16.539 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:16 smithi104 podman[72807]: 2024-05-27 10:33:16.128157959 +0000 UTC m=+0.034719092 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:33:16.540 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:16 smithi104 podman[72807]: 2024-05-27 10:33:16.254083278 +0000 UTC m=+0.160644438 container create 82b3fd7d0d094700508f8f2821b16bc57e797bd2c90a6184f60aba36ab9f5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-05-27T10:33:16.540 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:16 smithi104 podman[72807]: 2024-05-27 10:33:16.540767308 +0000 UTC m=+0.447328452 container init 82b3fd7d0d094700508f8f2821b16bc57e797bd2c90a6184f60aba36ab9f5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:33:16.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:16 smithi104 podman[72807]: 2024-05-27 10:33:16.551850388 +0000 UTC m=+0.458411527 container start 82b3fd7d0d094700508f8f2821b16bc57e797bd2c90a6184f60aba36ab9f5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T10:33:16.946 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:16 smithi104 podman[72807]: 2024-05-27 10:33:16.664617192 +0000 UTC m=+0.571178362 container attach 82b3fd7d0d094700508f8f2821b16bc57e797bd2c90a6184f60aba36ab9f5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T10:33:17.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:17.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:17.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:17.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:17.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:17.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:17.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:17.033+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:17.345 INFO:journalctl@ceph.osd.10.smithi104.stdout:May 27 10:33:17 smithi104 podman[72807]: 2024-05-27 10:33:17.006513486 +0000 UTC m=+0.913074686 container died 82b3fd7d0d094700508f8f2821b16bc57e797bd2c90a6184f60aba36ab9f5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T10:33:17.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.10.service' 2024-05-27T10:33:17.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:17.855 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-27T10:33:17.855 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-27T10:33:17.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.11 2024-05-27T10:33:18.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:17 smithi143 systemd[1]: Stopping Ceph osd.11 for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:18.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:18.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:18.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:18.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:18.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:18.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:18.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:18.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.039+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-27T10:32:52.449555+0000 front 2024-05-27T10:32:52.449419+0000 (oldest deadline 2024-05-27T10:33:17.749493+0000) 2024-05-27T10:33:18.806 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.458+0000 7f0731c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:18.806 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.458+0000 7f0731c00700 -1 osd.11 95 *** Got signal Terminated *** 2024-05-27T10:33:18.806 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:18 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:18.458+0000 7f0731c00700 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T10:33:19.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:19.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:19.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:19.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:19.308 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:19.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:19.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:19.309 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:19 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:19.044+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-27T10:32:52.449555+0000 front 2024-05-27T10:32:52.449419+0000 (oldest deadline 2024-05-27T10:33:17.749493+0000) 2024-05-27T10:33:20.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:20.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:20 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:20.007+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-27T10:32:52.449555+0000 front 2024-05-27T10:32:52.449419+0000 (oldest deadline 2024-05-27T10:33:17.749493+0000) 2024-05-27T10:33:21.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:21.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:21.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:21.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:21.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:21.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:21.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:21.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:21.000+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-27T10:32:52.449555+0000 front 2024-05-27T10:32:52.449419+0000 (oldest deadline 2024-05-27T10:33:17.749493+0000) 2024-05-27T10:33:22.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:22.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:22 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:22.024+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-27T10:32:52.449555+0000 front 2024-05-27T10:32:52.449419+0000 (oldest deadline 2024-05-27T10:33:17.749493+0000) 2024-05-27T10:33:23.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-05-27T10:31:34.727040+0000 front 2024-05-27T10:31:34.727235+0000 (oldest deadline 2024-05-27T10:31:58.227008+0000) 2024-05-27T10:33:23.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-05-27T10:31:46.832637+0000 front 2024-05-27T10:31:46.832727+0000 (oldest deadline 2024-05-27T10:32:09.132879+0000) 2024-05-27T10:33:23.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-27T10:31:55.536377+0000 front 2024-05-27T10:31:55.536276+0000 (oldest deadline 2024-05-27T10:32:22.536976+0000) 2024-05-27T10:33:23.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-05-27T10:32:02.538165+0000 front 2024-05-27T10:32:02.538260+0000 (oldest deadline 2024-05-27T10:32:27.838357+0000) 2024-05-27T10:33:23.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-05-27T10:32:15.441933+0000 front 2024-05-27T10:32:15.442087+0000 (oldest deadline 2024-05-27T10:32:41.341935+0000) 2024-05-27T10:33:23.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-27T10:32:25.444291+0000 front 2024-05-27T10:32:25.443986+0000 (oldest deadline 2024-05-27T10:32:48.343965+0000) 2024-05-27T10:33:23.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-05-27T10:32:43.047978+0000 front 2024-05-27T10:32:43.048086+0000 (oldest deadline 2024-05-27T10:33:07.747907+0000) 2024-05-27T10:33:23.307 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:23 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11[61070]: 2024-05-27T10:33:23.022+0000 7f072d400700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-27T10:32:52.449555+0000 front 2024-05-27T10:32:52.449419+0000 (oldest deadline 2024-05-27T10:33:17.749493+0000) 2024-05-27T10:33:24.306 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:24 smithi143 podman[74288]: 2024-05-27 10:33:24.028256084 +0000 UTC m=+5.709879098 container died aa09ab002a647f5778817df8abf34cd30f371c73bb8f0acd6b10d1d810c0bd7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-27T10:33:24.580 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:24 smithi143 podman[74288]: 2024-05-27 10:33:24.419299239 +0000 UTC m=+6.100922246 container cleanup aa09ab002a647f5778817df8abf34cd30f371c73bb8f0acd6b10d1d810c0bd7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T10:33:24.580 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:24 smithi143 bash[74288]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11 2024-05-27T10:33:24.580 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:24 smithi143 podman[74307]: 2024-05-27 10:33:24.578777529 +0000 UTC m=+0.527090905 container remove aa09ab002a647f5778817df8abf34cd30f371c73bb8f0acd6b10d1d810c0bd7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T10:33:25.615 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:25 smithi143 podman[74397]: 2024-05-27 10:33:25.269972755 +0000 UTC m=+0.034315981 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T10:33:25.615 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:25 smithi143 podman[74397]: 2024-05-27 10:33:25.379676733 +0000 UTC m=+0.144019963 container create 8019c793467f177ce6b5a2eb701af5678fec3d96af14dac4d009d083892da4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:33:25.615 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:25 smithi143 podman[74397]: 2024-05-27 10:33:25.616474787 +0000 UTC m=+0.380818044 container init 8019c793467f177ce6b5a2eb701af5678fec3d96af14dac4d009d083892da4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T10:33:26.055 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:25 smithi143 podman[74397]: 2024-05-27 10:33:25.627897905 +0000 UTC m=+0.392241124 container start 8019c793467f177ce6b5a2eb701af5678fec3d96af14dac4d009d083892da4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T10:33:26.056 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:25 smithi143 podman[74397]: 2024-05-27 10:33:25.838395291 +0000 UTC m=+0.602738584 container attach 8019c793467f177ce6b5a2eb701af5678fec3d96af14dac4d009d083892da4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5) 2024-05-27T10:33:26.437 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 27 10:33:26 smithi143 podman[74397]: 2024-05-27 10:33:26.087427543 +0000 UTC m=+0.851770813 container died 8019c793467f177ce6b5a2eb701af5678fec3d96af14dac4d009d083892da4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T10:33:26.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@osd.11.service' 2024-05-27T10:33:26.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:26.885 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-27T10:33:26.885 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-27T10:33:26.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.a 2024-05-27T10:33:27.341 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:27 smithi042 systemd[1]: Stopping Ceph mds.a for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:27.842 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a[74835]: 2024-05-27T10:33:27.458+0000 7fe091e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:27.842 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:27 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a[74835]: 2024-05-27T10:33:27.458+0000 7fe091e00700 -1 mds.a *** got signal Terminated *** 2024-05-27T10:33:31.152 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:30 smithi042 podman[160367]: 2024-05-27 10:33:30.855367781 +0000 UTC m=+3.524111118 container died 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T10:33:31.417 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:31 smithi042 podman[160367]: 2024-05-27 10:33:31.392575978 +0000 UTC m=+4.061319308 container cleanup 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:33:31.417 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:31 smithi042 bash[160367]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a 2024-05-27T10:33:32.091 INFO:journalctl@ceph.mds.a.smithi042.stdout:May 27 10:33:31 smithi042 podman[160378]: 2024-05-27 10:33:31.744755883 +0000 UTC m=+0.867952121 container remove 2c4b962aeb59879db7a9111a629bae6201b480e501f57774eab3d2014ed7133b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T10:33:32.250 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.a.service' 2024-05-27T10:33:32.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:32.370 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-27T10:33:32.370 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-27T10:33:32.370 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.d 2024-05-27T10:33:32.841 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:32 smithi042 systemd[1]: Stopping Ceph mds.d for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:33.342 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d[73462]: 2024-05-27T10:33:33.021+0000 7f7bf3c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:33.342 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:33 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d[73462]: 2024-05-27T10:33:33.021+0000 7f7bf3c00700 -1 mds.d *** got signal Terminated *** 2024-05-27T10:33:35.476 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:35 smithi042 podman[160488]: 2024-05-27 10:33:35.122490483 +0000 UTC m=+2.246182231 container died 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T10:33:35.822 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:35 smithi042 podman[160488]: 2024-05-27 10:33:35.589080057 +0000 UTC m=+2.712771829 container cleanup 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-27T10:33:35.823 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:35 smithi042 bash[160488]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d 2024-05-27T10:33:36.092 INFO:journalctl@ceph.mds.d.smithi042.stdout:May 27 10:33:35 smithi042 podman[160513]: 2024-05-27 10:33:35.823706682 +0000 UTC m=+0.685070801 container remove 868cb46713fbd5952500a57bace7d6cf06b3eda70e961c469aa8cad41c880cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T10:33:36.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.d.service' 2024-05-27T10:33:36.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:36.526 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-27T10:33:36.526 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-27T10:33:36.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.g 2024-05-27T10:33:36.842 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:36 smithi042 systemd[1]: Stopping Ceph mds.g for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:37.591 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g[74392]: 2024-05-27T10:33:37.107+0000 7f72f9600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:37.592 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:37 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g[74392]: 2024-05-27T10:33:37.107+0000 7f72f9600700 -1 mds.g *** got signal Terminated *** 2024-05-27T10:33:41.205 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:40 smithi042 podman[160620]: 2024-05-27 10:33:40.87699956 +0000 UTC m=+3.889007195 container died 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T10:33:41.592 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:41 smithi042 podman[160620]: 2024-05-27 10:33:41.393518982 +0000 UTC m=+4.405526611 container cleanup 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-27T10:33:41.592 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:41 smithi042 bash[160620]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g 2024-05-27T10:33:41.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.g.service' 2024-05-27T10:33:42.018 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:41 smithi042 podman[160634]: 2024-05-27 10:33:41.695120456 +0000 UTC m=+0.794672595 container remove 860b52c20f48432e66096bbc63fd44031a260e6dde282fd970cb04372ae2a013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:33:42.018 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:41 smithi042 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-27T10:33:42.018 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:41 smithi042 systemd[1]: Stopped Ceph mds.g for ff1a20ea-1c0f-11ef-bc9b-c7b262605968. 2024-05-27T10:33:42.019 INFO:journalctl@ceph.mds.g.smithi042.stdout:May 27 10:33:41 smithi042 systemd[1]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.g.service: Consumed 2.673s CPU time. 2024-05-27T10:33:42.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:42.056 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-27T10:33:42.056 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-27T10:33:42.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.j 2024-05-27T10:33:42.341 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:42 smithi042 systemd[1]: Stopping Ceph mds.j for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:43.092 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j[73041]: 2024-05-27T10:33:42.652+0000 7fe5cf000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:43.092 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:42 smithi042 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j[73041]: 2024-05-27T10:33:42.652+0000 7fe5cf000700 -1 mds.j *** got signal Terminated *** 2024-05-27T10:33:46.592 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:46 smithi042 podman[160739]: 2024-05-27 10:33:46.308440642 +0000 UTC m=+3.786959816 container died 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T10:33:47.005 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:46 smithi042 podman[160739]: 2024-05-27 10:33:46.756012837 +0000 UTC m=+4.234531971 container cleanup 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T10:33:47.006 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:46 smithi042 bash[160739]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j 2024-05-27T10:33:47.341 INFO:journalctl@ceph.mds.j.smithi042.stdout:May 27 10:33:47 smithi042 podman[160752]: 2024-05-27 10:33:47.006931123 +0000 UTC m=+0.684288888 container remove 9f6a3772c279096e356164d1049a6f8c734fece97f92a2150fccb614d4bf2ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T10:33:47.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.j.service' 2024-05-27T10:33:47.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:47.660 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-27T10:33:47.660 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-27T10:33:47.660 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.b 2024-05-27T10:33:48.165 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:47 smithi104 systemd[1]: Stopping Ceph mds.b for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:48.446 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b[65158]: 2024-05-27T10:33:48.214+0000 7f81e4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:48.446 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:48 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b[65158]: 2024-05-27T10:33:48.214+0000 7f81e4e00700 -1 mds.b *** got signal Terminated *** 2024-05-27T10:33:54.851 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:54 smithi104 podman[72925]: 2024-05-27 10:33:54.536134241 +0000 UTC m=+6.448352110 container died 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T10:33:55.196 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:54 smithi104 podman[72925]: 2024-05-27 10:33:54.973030423 +0000 UTC m=+6.885248289 container cleanup 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0) 2024-05-27T10:33:55.196 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:54 smithi104 bash[72925]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b 2024-05-27T10:33:55.627 INFO:journalctl@ceph.mds.b.smithi104.stdout:May 27 10:33:55 smithi104 podman[72949]: 2024-05-27 10:33:55.198893254 +0000 UTC m=+0.644702445 container remove 46fd50cc7f4120a150a700fdbb3e2e9ab8a53d8d4992442d22c6e5b0d9c52297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T10:33:55.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.b.service' 2024-05-27T10:33:55.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:33:55.763 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-27T10:33:55.763 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-27T10:33:55.763 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.e 2024-05-27T10:33:56.196 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:55 smithi104 systemd[1]: Stopping Ceph mds.e for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:33:56.696 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e[64539]: 2024-05-27T10:33:56.391+0000 7f23b6200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:33:56.696 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:56 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e[64539]: 2024-05-27T10:33:56.391+0000 7f23b6200700 -1 mds.e *** got signal Terminated *** 2024-05-27T10:33:59.097 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:58 smithi104 podman[73057]: 2024-05-27 10:33:58.790728562 +0000 UTC m=+2.536229590 container died 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T10:33:59.446 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:59 smithi104 podman[73057]: 2024-05-27 10:33:59.228015069 +0000 UTC m=+2.973516087 container cleanup 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T10:33:59.447 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:59 smithi104 bash[73057]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e 2024-05-27T10:33:59.946 INFO:journalctl@ceph.mds.e.smithi104.stdout:May 27 10:33:59 smithi104 podman[73070]: 2024-05-27 10:33:59.569389452 +0000 UTC m=+0.762487446 container remove 6315d348d2d7bcde311fecb245fca3dd680d2510929390ddb22882cfcb1e1d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-27T10:34:00.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.e.service' 2024-05-27T10:34:00.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:00.226 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-27T10:34:00.226 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-27T10:34:00.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.h 2024-05-27T10:34:00.696 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:00 smithi104 systemd[1]: Stopping Ceph mds.h for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:34:01.196 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:34:00.812+0000 7fe191600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:34:01.196 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:00 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h[64838]: 2024-05-27T10:34:00.812+0000 7fe191600700 -1 mds.h *** got signal Terminated *** 2024-05-27T10:34:06.174 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:05 smithi104 podman[73176]: 2024-05-27 10:34:05.908602664 +0000 UTC m=+5.229376088 container died 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2) 2024-05-27T10:34:06.446 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:06 smithi104 podman[73176]: 2024-05-27 10:34:06.312953897 +0000 UTC m=+5.633727340 container cleanup 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T10:34:06.446 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:06 smithi104 bash[73176]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h 2024-05-27T10:34:06.946 INFO:journalctl@ceph.mds.h.smithi104.stdout:May 27 10:34:06 smithi104 podman[73187]: 2024-05-27 10:34:06.505776032 +0000 UTC m=+0.572862876 container remove 9faec2d5ac5227b647c37d28fa1aff4927392c9003273fa32f2c91b6bad9472d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-27T10:34:07.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.h.service' 2024-05-27T10:34:07.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:07.103 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-27T10:34:07.104 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-27T10:34:07.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k 2024-05-27T10:34:07.696 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:07 smithi104 systemd[1]: Stopping Ceph mds.k for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:34:08.196 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[70346]: 2024-05-27T10:34:07.705+0000 7fe0c8200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:34:08.197 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:07 smithi104 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k[70346]: 2024-05-27T10:34:07.705+0000 7fe0c8200700 -1 mds.k *** got signal Terminated *** 2024-05-27T10:34:10.934 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:10 smithi104 podman[73296]: 2024-05-27 10:34:10.545328182 +0000 UTC m=+2.961626758 container died 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T10:34:11.196 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:10 smithi104 podman[73296]: 2024-05-27 10:34:10.935667869 +0000 UTC m=+3.351966472 container cleanup 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T10:34:11.196 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:10 smithi104 bash[73296]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k 2024-05-27T10:34:11.687 INFO:journalctl@ceph.mds.k.smithi104.stdout:May 27 10:34:11 smithi104 podman[73308]: 2024-05-27 10:34:11.224998039 +0000 UTC m=+0.663918098 container remove 24d97da2f9a027be2321d6504927e45dd45713d10c7bfc33a2915c8c77535169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T10:34:11.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.k.service' 2024-05-27T10:34:11.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:11.842 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-27T10:34:11.843 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-27T10:34:11.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.c 2024-05-27T10:34:12.305 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:11 smithi143 systemd[1]: Stopping Ceph mds.c for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:34:12.800 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c[66350]: 2024-05-27T10:34:12.389+0000 7f0ccca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:34:12.801 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:12 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c[66350]: 2024-05-27T10:34:12.389+0000 7f0ccca00700 -1 mds.c *** got signal Terminated *** 2024-05-27T10:34:15.853 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:15 smithi143 podman[74503]: 2024-05-27 10:34:15.563489873 +0000 UTC m=+3.319905607 container died 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521) 2024-05-27T10:34:16.250 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:15 smithi143 podman[74503]: 2024-05-27 10:34:15.958226298 +0000 UTC m=+3.714641995 container cleanup 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2) 2024-05-27T10:34:16.250 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:15 smithi143 bash[74503]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c 2024-05-27T10:34:16.551 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 27 10:34:16 smithi143 podman[74527]: 2024-05-27 10:34:16.250871087 +0000 UTC m=+0.665922859 container remove 98f783de499c493cf0d0af86668f882966a59554e3daee4e0840ef6f0386bc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-27T10:34:16.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.c.service' 2024-05-27T10:34:16.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:16.710 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-27T10:34:16.710 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-27T10:34:16.710 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.f 2024-05-27T10:34:17.270 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:16 smithi143 systemd[1]: Stopping Ceph mds.f for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:34:17.555 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f[66046]: 2024-05-27T10:34:17.271+0000 7f1546a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:34:17.555 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:17 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f[66046]: 2024-05-27T10:34:17.271+0000 7f1546a00700 -1 mds.f *** got signal Terminated *** 2024-05-27T10:34:19.749 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:19 smithi143 podman[74634]: 2024-05-27 10:34:19.45323737 +0000 UTC m=+2.314264228 container died ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-27T10:34:20.056 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:19 smithi143 podman[74634]: 2024-05-27 10:34:19.870497505 +0000 UTC m=+2.731524333 container cleanup ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T10:34:20.056 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:19 smithi143 bash[74634]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f 2024-05-27T10:34:20.555 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 27 10:34:20 smithi143 podman[74648]: 2024-05-27 10:34:20.105122985 +0000 UTC m=+0.636852757 container remove ec67033ec54c50cf6020222d0893c1c38369961314ac2e6951f48145aad2c1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T10:34:20.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.f.service' 2024-05-27T10:34:20.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:20.720 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-27T10:34:20.720 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-27T10:34:20.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.i 2024-05-27T10:34:21.055 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:20 smithi143 systemd[1]: Stopping Ceph mds.i for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:34:21.555 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i[66657]: 2024-05-27T10:34:21.270+0000 7f04e4200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:34:21.555 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:21 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i[66657]: 2024-05-27T10:34:21.270+0000 7f04e4200700 -1 mds.i *** got signal Terminated *** 2024-05-27T10:34:24.053 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:23 smithi143 podman[74753]: 2024-05-27 10:34:23.795644949 +0000 UTC m=+2.654138351 container died b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T10:34:24.305 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:24 smithi143 podman[74753]: 2024-05-27 10:34:24.166029872 +0000 UTC m=+3.024523205 container cleanup b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T10:34:24.305 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:24 smithi143 bash[74753]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i 2024-05-27T10:34:24.720 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 27 10:34:24 smithi143 podman[74765]: 2024-05-27 10:34:24.342157185 +0000 UTC m=+0.522499011 container remove b1f10c310b47da5af636b42e767b5776eaa743a0c30c2d13f50aa07d960d0220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T10:34:24.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.i.service' 2024-05-27T10:34:24.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:24.858 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-27T10:34:24.858 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-27T10:34:24.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.l 2024-05-27T10:34:25.305 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:25 smithi143 systemd[1]: Stopping Ceph mds.l for ff1a20ea-1c0f-11ef-bc9b-c7b262605968... 2024-05-27T10:34:25.805 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l[65738]: 2024-05-27T10:34:25.492+0000 7fd81be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T10:34:25.805 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:25 smithi143 ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l[65738]: 2024-05-27T10:34:25.492+0000 7fd81be00700 -1 mds.l *** got signal Terminated *** 2024-05-27T10:34:29.064 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:28 smithi143 podman[74872]: 2024-05-27 10:34:28.812204885 +0000 UTC m=+3.449009297 container died c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True) 2024-05-27T10:34:29.386 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:29 smithi143 podman[74872]: 2024-05-27 10:34:29.178403473 +0000 UTC m=+3.815207908 container cleanup c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T10:34:29.386 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:29 smithi143 bash[74872]: ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l 2024-05-27T10:34:29.805 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 27 10:34:29 smithi143 podman[74885]: 2024-05-27 10:34:29.387453115 +0000 UTC m=+0.553822252 container remove c29cc86ecba11a98d8bc6c272c68e21bb5acc67aa2f86585a8bf3f421e1c211d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-27T10:34:29.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff1a20ea-1c0f-11ef-bc9b-c7b262605968@mds.l.service' 2024-05-27T10:34:29.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T10:34:29.966 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-27T10:34:29.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T10:34:30.300 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:34:35.421 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T10:34:35.738 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:34:40.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T10:34:40.829 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:34:45.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:34:45.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:34:45.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T10:34:45.993 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-27T10:34:45.996 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi042/crash 2024-05-27T10:34:45.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T10:34:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T10:34:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-05-27T10:34:46.089 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi104/crash 2024-05-27T10:34:46.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T10:34:46.184 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T10:34:46.184 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-05-27T10:34:46.188 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi143/crash 2024-05-27T10:34:46.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T10:34:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:tar: /var/lib/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T10:34:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:tar: Error is not recoverable: exiting now 2024-05-27T10:34:46.283 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-27T10:34:46.284 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-27T10:34:46.395 INFO:teuthology.orchestra.run.smithi042.stdout:2024-05-27T10:21:49.588039+0000 mds.h (mds.1) 2 : cluster [WRN] Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/8b372384-d91f-4b21-9dce-13ee6ea8be63/client.0/tmp/blogbench-1.0/man) see mds.h log and `damage ls` output for details 2024-05-27T10:34:46.397 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-27T10:34:46.397 INFO:tasks.cephadm:Compressing logs... 2024-05-27T10:34:46.397 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T10:34:46.401 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T10:34:46.409 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T10:34:46.458 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T10:34:46.461 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-27T10:34:46.462 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T10:34:46.463 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T10:34:46.464 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-05-27T10:34:46.465 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-05-27T10:34:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.g.log: 87.5%gzip -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T10:34:46.467 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-27T10:34:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-27T10:34:46.468 INFO:teuthology.orchestra.run.smithi042.stderr: 90.2% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T10:34:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-05-27T10:34:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-05-27T10:34:46.473 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-05-27T10:34:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-05-27T10:34:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T10:34:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-05-27T10:34:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-05-27T10:34:46.478 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T10:34:46.479 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.a.log: -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T10:34:46.481 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T10:34:46.481 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-05-27T10:34:46.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-05-27T10:34:46.483 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T10:34:46.483 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-05-27T10:34:46.483 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-05-27T10:34:46.484 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-27T10:34:46.485 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.b.log: 87.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T10:34:46.486 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-27T10:34:46.487 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.9.log 94.4% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-05-27T10:34:46.487 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-05-27T10:34:46.488 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-05-27T10:34:46.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.4.log: 90.3% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T10:34:46.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-05-27T10:34:46.490 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.9.log: --verbose -- /var/log/ceph/cephadm.log 2024-05-27T10:34:46.490 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-05-27T10:34:46.491 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T10:34:46.494 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.10.log: /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T10:34:46.496 INFO:teuthology.orchestra.run.smithi143.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T10:34:46.496 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T10:34:46.496 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-27T10:34:46.496 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-05-27T10:34:46.498 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-05-27T10:34:46.498 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T10:34:46.498 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T10:34:46.499 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-27T10:34:46.499 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T10:34:46.500 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-05-27T10:34:46.501 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-05-27T10:34:46.502 INFO:teuthology.orchestra.run.smithi104.stderr: 92.2% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-05-27T10:34:46.503 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-05-27T10:34:46.503 INFO:teuthology.orchestra.run.smithi143.stderr: 90.4% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T10:34:46.503 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-05-27T10:34:46.504 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-05-27T10:34:46.505 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-05-27T10:34:46.505 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-05-27T10:34:46.506 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T10:34:46.507 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T10:34:46.507 INFO:teuthology.orchestra.run.smithi143.stderr: 94.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-05-27T10:34:46.507 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-05-27T10:34:46.509 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T10:34:46.510 INFO:teuthology.orchestra.run.smithi143.stderr:gzip/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.11.log: -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-05-27T10:34:46.511 INFO:teuthology.orchestra.run.smithi042.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T10:34:46.513 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T10:34:46.515 INFO:teuthology.orchestra.run.smithi042.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T10:34:46.516 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T10:34:46.517 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-05-27T10:34:46.519 INFO:teuthology.orchestra.run.smithi143.stderr: 92.2% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-05-27T10:34:46.520 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T10:34:46.524 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T10:34:46.534 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T10:34:46.538 INFO:teuthology.orchestra.run.smithi143.stderr: 94.5% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-05-27T10:34:46.551 INFO:teuthology.orchestra.run.smithi143.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T10:34:46.871 INFO:teuthology.orchestra.run.smithi143.stderr: 91.7% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-05-27T10:34:46.907 INFO:teuthology.orchestra.run.smithi042.stderr: 90.0% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-05-27T10:34:46.911 INFO:teuthology.orchestra.run.smithi143.stderr: 94.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-05-27T10:34:46.917 INFO:teuthology.orchestra.run.smithi104.stderr: 94.1% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-05-27T10:34:47.106 INFO:teuthology.orchestra.run.smithi104.stderr: 91.8% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-05-27T10:34:47.703 INFO:teuthology.orchestra.run.smithi042.stderr: 90.1% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-05-27T10:34:50.288 INFO:teuthology.orchestra.run.smithi104.stderr: 93.8% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-05-27T10:34:51.186 INFO:teuthology.orchestra.run.smithi143.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-05-27T10:34:51.443 INFO:teuthology.orchestra.run.smithi143.stderr: 91.0% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-27T10:34:51.471 INFO:teuthology.orchestra.run.smithi143.stderr: 91.3% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-05-27T10:34:51.904 INFO:teuthology.orchestra.run.smithi042.stderr: 91.2% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-27T10:34:52.718 INFO:teuthology.orchestra.run.smithi042.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-05-27T10:34:52.926 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-27T10:34:52.981 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-05-27T10:34:53.107 INFO:teuthology.orchestra.run.smithi104.stderr: 91.3% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-27T10:34:53.377 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-05-27T10:34:53.647 INFO:teuthology.orchestra.run.smithi104.stderr: 91.2% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-27T10:34:53.986 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-05-27T10:34:54.000 INFO:teuthology.orchestra.run.smithi143.stderr: 91.2% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-27T10:34:54.060 INFO:teuthology.orchestra.run.smithi104.stderr: 94.3% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-05-27T10:34:54.690 INFO:teuthology.orchestra.run.smithi143.stderr: 94.0% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-05-27T10:34:54.694 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-05-27T10:34:54.694 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m8.231s 2024-05-27T10:34:54.694 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m29.008s 2024-05-27T10:34:54.694 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m2.152s 2024-05-27T10:36:05.712 INFO:teuthology.orchestra.run.smithi042.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-27T10:36:05.715 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-05-27T10:36:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:real 1m19.281s 2024-05-27T10:36:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:user 1m38.557s 2024-05-27T10:36:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m7.090s 2024-05-27T10:36:50.380 INFO:teuthology.orchestra.run.smithi104.stderr: 94.0% -- replaced with /var/log/ceph/ff1a20ea-1c0f-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-05-27T10:36:50.384 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-05-27T10:36:50.384 INFO:teuthology.orchestra.run.smithi104.stderr:real 2m3.932s 2024-05-27T10:36:50.384 INFO:teuthology.orchestra.run.smithi104.stderr:user 2m33.070s 2024-05-27T10:36:50.384 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m10.950s 2024-05-27T10:36:50.385 INFO:tasks.cephadm:Archiving logs... 2024-05-27T10:36:50.386 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi042/log 2024-05-27T10:36:50.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T10:37:05.584 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi104/log 2024-05-27T10:37:05.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T10:37:26.633 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi143/log 2024-05-27T10:37:26.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T10:37:30.930 INFO:tasks.cephadm:Removing cluster... 2024-05-27T10:37:30.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --force 2024-05-27T10:37:31.264 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:37:31.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --force 2024-05-27T10:37:32.276 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:37:33.246 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff1a20ea-1c0f-11ef-bc9b-c7b262605968 --force 2024-05-27T10:37:33.570 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: ff1a20ea-1c0f-11ef-bc9b-c7b262605968 2024-05-27T10:37:34.220 INFO:tasks.cephadm:Removing cephadm ... 2024-05-27T10:37:34.221 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T10:37:34.282 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T10:37:34.340 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T10:37:34.397 INFO:tasks.cephadm:Teardown complete 2024-05-27T10:37:34.397 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T10:37:34.411 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-27T10:37:34.412 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T10:37:34.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T10:37:34.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T10:37:34.425 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T10:37:34.588 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T10:37:34.602 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T10:37:34.602 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T10:37:34.610 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T10:37:34.617 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T10:37:34.661 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-05-27T10:37:34.665 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-05-27T10:37:34.672 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T10:37:34.672 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-05-27T10:37:34.672 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +664us[ +678us] +/- 69ms 2024-05-27T10:37:34.672 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -1188us[-1188us] +/- 75ms 2024-05-27T10:37:34.672 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +557us[ +572us] +/- 84ms 2024-05-27T10:37:34.673 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T10:37:34.673 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-05-27T10:37:34.674 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T10:37:34.674 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-05-27T10:37:34.675 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 7 377 68 +647us[ +666us] +/- 68ms 2024-05-27T10:37:34.675 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -1255us[-1255us] +/- 75ms 2024-05-27T10:37:34.675 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +465us[ +465us] +/- 83ms 2024-05-27T10:37:34.675 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T10:37:34.683 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T10:37:34.683 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-05-27T10:37:34.683 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv01.front.sepia.ceph.com 3 7 377 73 +708us[ +693us] +/- 68ms 2024-05-27T10:37:34.683 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 5 -1146us[-1146us] +/- 75ms 2024-05-27T10:37:34.683 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +551us[ +551us] +/- 84ms 2024-05-27T10:37:34.683 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T10:37:34.684 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T10:37:34.695 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T10:37:34.696 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T10:37:34.734 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T10:37:34.969 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T10:37:35.151 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T10:37:35.347 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T10:37:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:37:35.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T10:37:35.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:37:35.544 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T10:37:35.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:37:35.636 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T10:37:35.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T10:37:35.734 INFO:teuthology.task.internal:Duration was 2730.169021 seconds 2024-05-27T10:37:35.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T10:37:35.784 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T10:37:35.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T10:37:35.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T10:37:35.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T10:37:35.928 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T10:37:35.936 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T10:37:35.944 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T10:37:36.437 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T10:37:36.438 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-05-27T10:37:36.438 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T10:37:36.502 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-05-27T10:37:36.503 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T10:37:36.576 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-05-27T10:37:36.576 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T10:37:36.639 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T10:37:36.640 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T10:37:36.648 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T10:37:36.653 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T10:37:36.738 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T10:37:36.738 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T10:37:36.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T10:37:36.749 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T10:37:37.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T10:37:37.847 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T10:37:37.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T10:37:37.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T10:37:38.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T10:37:38.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T10:37:38.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T10:37:38.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T10:37:38.146 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T10:37:38.233 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-05-27T10:37:38.235 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-05-27T10:37:38.239 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-05-27T10:37:38.296 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T10:37:38.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:37:38.351 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T10:37:38.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:37:38.406 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T10:37:38.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T10:37:38.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T10:37:38.472 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T10:37:38.473 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi042 2024-05-27T10:37:38.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T10:37:38.595 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi104 2024-05-27T10:37:38.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T10:37:38.694 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727706/remote/smithi143 2024-05-27T10:37:38.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T10:37:38.793 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T10:37:38.794 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T10:37:38.802 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T10:37:38.809 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T10:37:38.864 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T10:37:38.878 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T10:37:38.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T10:37:38.946 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T10:37:38.947 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T10:37:38.955 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T10:37:38.962 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T10:37:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 10:37 /home/ubuntu/cephtest 2024-05-27T10:37:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 10:37 /home/ubuntu/cephtest 2024-05-27T10:37:39.020 INFO:teuthology.orchestra.run.smithi143.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 10:37 /home/ubuntu/cephtest 2024-05-27T10:37:39.023 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T10:37:39.036 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T10:37:39.069 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-27T10:37:39.073 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2730.169020652771 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.c-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_vshankar@teuthology status: fail success: false 2024-05-27T10:37:39.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T10:37:39.170 INFO:teuthology.run:FAIL